Document some perf numbers

[conartist6]

I found some test results which benchmark C regex engines. It includes a corpus and some realistic expressions of varying complexity, so I think I can run the test directly.

I'd like to see how this engine stacks up for speed.

If I care enough to try to do an apples to apples comparison I'll also need to build the tests for the other engines so they can run on the same hardware. Really though if I'm within an order of magnitude I'll be happy for now, given that I've done all this in a scripting language.

[conartist6]

I should say that I did run the test suite, and this engine is really, quite slow. It was a while ago but IIRC maybe 100x slower than a native implementation. At first I thought I had the wrong computational complexity somewhere, but after I stared at the profiling output for a while (and fixed a few issues) it seemed clear to me that time was being spent in the places I expect it to be spent. I guess it's not ideal to write a regex engine in a scripting language? Who knew.

Anyway, I'm not too worried. 100x slower sounds like a lot, but this is not a tool for searching a huge corpus with a complicated expression. Those exist! This is about matching on streams, which was not previously possible in the browser. When your data is a stream, even 100x slower regex parsing may be a trivial cost compared with being unable to do any work until an input stream terminates. A stream may never terminate!

[conartist6]

I expect to see some very big perf gains from #35. Will measure the speedup soon.

[conartist6]

It might be worth getting rid of the remaining string comparison in favor of symbols, e.g. use Symbol('cont') vs 'cont'. We know that symbols can always be equality checked in a single operation, whereas string comparisons need to go into character-by-character comparison if strings are not reference-equal.

[conartist6]

The next really hot section will be getSeq. Anything I could do to optimize that would be well worth it.

[conartist6]

It might speed things up to combine adjacent non-match expressions when possible... It could be done in state.explode, probably relatively easily given that expressions have little functionality or data of their own

[conartist6]

I was able to run the benchmark. I'm cheating a bit using better hardware, but I'm much closer the the average performance now than I was before. I'd say maybe 20x slower instead of 100x, so a 5x speedup from my changes, which seems plausible to me. That's incorporating a ~2x fudge factor for the newer hardware, and assuming the average runtime of native libraries on the URI pattern and provided corpus to be ~2s (I didn't do the math).

[conartist6]

I made some more perf improvements but I forget where I but the benchmarking script : (

[conartist6]

Oh I figured out what happened. I was part way through the perf test stuff and I got distracted while I was implementing head to head perf testing with native javascript regex. The results are... not encouraging.

const corpus = readFile('corpus/howtosmall'); // 4MB of text

let exp;

// URI (protocol://server/path)
exp = /([a-zA-Z][a-zA-Z0-9]*):\/\/([^ /]+)(\/[^ ]*)?/g;
execGlobal(exp, corpus); // 3371 ms
exp.exec(corpus); // 42 ms
// slowdown: 80x

// Email (name@server)
exp = /([^ @]+)@([^ @]+)/g;
execGlobal(exp, corpus); // 3325 ms
exp.exec(corpus); // 66 ms
// slowdown: 50x

// Date (month/day/year)
exp = /([0-9][0-9]?)\/([0-9][0-9]?)\/([0-9][0-9]([0-9][0-9])?)/g;
execGlobal(exp, corpus); // 2184 ms
exp.exec(corpus); // 1 ms
// slowdown: 2184x

// URI|Email
exp = /([a-zA-Z][a-zA-Z0-9]*):\/\/([^ /]+)(\/[^ ]*)?|([^ @]+)@([^ @]+)/g;
execGlobal(exp, corpus); // 6413 ms
exp.exec(corpus); // 98 ms
// slowdown: 65x

[conartist6]

The 2000x slowdown is the eye popping one. I assume the native engine has implemented an optimization for certain kinds of extremely simple patterns which allows it to avoid the cost and overhead of a full engine. I think I could easily do something similar for this project.

[conartist6]

The rest of the patterns execute an average of 70x slower than native. Likely closing the gap on those would require a combination of algorithmic optimizations and getting closer to the hardware. Fortunately it is likely that both of these things are possible.

[conartist6]

Something to think about: Rust has a streaming regex engine based on RE2. I think I could just ship that as WASM code with bindings. I wonder how big it would be? It has pretty much all the features I want and need except for lookahead.

[conartist6]

I've been noticing depots on certain termination cases. These cases may not be hit until the engine is otherwise very hot and well optimized, at which point it depots. I think the thing to do might be to run a few small matches as a warm-up to ensure that the ICs for these lines contain data.

[conartist6]

I guess I think for the moment it's worth exploring the optimizations that are still possible in the scripted code. That includes storing nextSeq in the engine's data structure so that it can be accessed like a linked list and caching DFA representations for some parts of the graph.

[conartist6]

Also possible: keeping pools of expression and sequence nodes to avoid some GC costs? (Think React events)