perf: move indexer to the manager node

[doorgan]

I was looking at the possibility of indexing without the engine and realized there are not many reasons to keep the indexer in the engine. A smaller benefit but still part of the motivation is that on the fly engine builds can be a little bit faster and take a little bit less disk space it less code needs to be compiled.

This PR moves the indexer to the manager node, and changes a few things about how the indexer works. It's a big change because all indexer files are moved to the manager node, so I tried to split this into several commits that touch each individual step I took to implement this so it's easier to review.

• The compiler tracer for definitions is removed. Updating the beam files indexer I came to the realization that it is redundant if we index project beam files. It also added complexity to this whole idea because we can't fully move the indexer to the manager node if it exists.
• The indexer now uses Streams instead of producing all the entries, holding them in memory and then storing them. This keeps memory usage more stable, and in my measurements using this against the Sequin codebase, memory usage during indexing drops from 1.3gb down to 400mb. Should fix Huge memory usage spike on startup #862
• The indexes are dropped before a full index replace and recreated afterwards, in my tests this is way faster than having the indexes be updates constantly for every insert chunk.
• There are still parts of the source indexer that need runtime information, those were kept and a cache in the reducer prevents excessive chattiness with the engine that can slow down indexing by about 10x. One possible improvement on top of that is to make those calls optional so we can have expert work in a mode that avoids compilation, but that's out of scope for this PR.

Total time to perform a full reindex of Sequin is a little bit faster with this vs main(7 seconds with this, closer to 10 seconds with main). Memory usage is way less and more stable though, which is the main benefit of this change.

[doorgan]

cc @katafrakt, this may also help with the memory issues you were seeing