Apparently, our benchmarks take up a lot of the community runner's time. #8387 is one mitigation, but we should probably also go through and see where we could trim back benchmarks to lower the runner time without losing too much signal.
This should include:
- Provide a breakdown of where benchmark time is spent across the repo.
- List the top 10 benchmarks in terms of benchmark time.
- Identify any overlap or redundancy we have between benchmarks.
- Propose benchmark cases to remove, or other ways to simplify the benchmark suite.
Apparently, our benchmarks take up a lot of the community runner's time. #8387 is one mitigation, but we should probably also go through and see where we could trim back benchmarks to lower the runner time without losing too much signal.
This should include: