Include HFS in Health Samurai FHIR server performance benchmark (separate GH Action + internal test)

[smunini]

Summary

Health Samurai recently published a public FHIR Server Performance Benchmark and comparative report:

• Repo: https://github.com/HealthSamurai/fhir-server-performance-benchmark
• Report: https://healthsamurai.github.io/fhir-server-performance-benchmark

Today it evaluates Aidbox, HAPI FHIR, and Medplum. HFS should be included so we can measure ourselves against these servers and publish comparable numbers.

We already established the pattern for this with the HTS terminology benchmark (.github/workflows/hts-benchmark.yml, which drives the HealthSamurai tx-benchmark k6 suite against HTS on our self-hosted runner). We want to repeat that approach for the general FHIR server benchmark.

What the benchmark covers

• Load generator: k6
• Environment: Docker Compose, with Prometheus + Grafana dashboards
• Operations exercised: CRUD (create/read/update/delete), bulk import via FHIR bundles, search (simple / complex / full-text), and validation (schema + terminology)
• Bootstrap: ./runner.sh bootstrap brings the stack up (Aidbox :13080, HAPI :13090, Grafana :13000)

Scope of this work

1. Wire HFS into the benchmark harness — add HFS as a target server in the HealthSamurai benchmark (Docker Compose service + runner config), analogous to how Aidbox/HAPI/Medplum are configured. Determine which storage backend(s) to benchmark (SQLite, PostgreSQL) as we did for HTS.
2. Create a separate GitHub Action — a manual workflow_dispatch benchmark workflow (e.g. .github/workflows/fhir-benchmark.yml) modeled on hts-benchmark.yml: backend matrix input, VUs/duration/test selection inputs, self-hosted runner, remote Docker daemon wiring, k6 auto-install.
3. Run an internal test — execute the benchmark against HFS internally, capture results, and identify any gaps (unsupported operations, failing scenarios) that block full participation.
4. Follow up on submission — once HFS runs clean, coordinate with Health Samurai on including HFS in the public comparative report.

Notes / open questions

• Confirm which FHIR operations HFS must implement to fully participate (bulk import, full-text search, validation) and file follow-ups for any gaps.
• Reuse the self-hosted runner + remote Docker (DOCKER_HOST / DOCKER_HOST_IP) pattern already established by the HTS and audit-events workflows.

Reference

• Existing pattern: .github/workflows/hts-benchmark.yml