When benching on a single machine we need to finely control the number of threads used by the services spun up by windsock. This includes the bench itself, shotover and the DB.
If services are sharing cores it will give less consistent results as a service performing more work will slow down other services.
In a realistic production use case these services will be running on machines with more than just one core, so we should use as many cores as we have available in order to create a realistic scenario.
Additionally we will get more useful results if we always ensure shotover is the bottleneck, that way we will actually be able to measure changes to shotovers performance.
To achieve this, there should be hardcoded logic per Bench implementation that derives the core counts per service from the number of cores on the machine it runs on.
When benching on a single machine we need to finely control the number of threads used by the services spun up by windsock. This includes the bench itself, shotover and the DB.
If services are sharing cores it will give less consistent results as a service performing more work will slow down other services.
In a realistic production use case these services will be running on machines with more than just one core, so we should use as many cores as we have available in order to create a realistic scenario.
Additionally we will get more useful results if we always ensure shotover is the bottleneck, that way we will actually be able to measure changes to shotovers performance.
To achieve this, there should be hardcoded logic per
Benchimplementation that derives the core counts per service from the number of cores on the machine it runs on.