Skip to content

feat: implement end-to-end runtime profiling and telemetry - #119

Open
rapheljr wants to merge 4 commits into
drumih:mainfrom
rapheljr:feat/runtime-profiling
Open

feat: implement end-to-end runtime profiling and telemetry#119
rapheljr wants to merge 4 commits into
drumih:mainfrom
rapheljr:feat/runtime-profiling

Conversation

@rapheljr

Copy link
Copy Markdown

Summary

Adds a zero-overhead runtime telemetry and end-to-end profiling subsystem to TurboFieldfare without changing inference behavior, tensor layouts, or kernel performance.

Features Included

  • RuntimeTelemetry Core (RuntimeTelemetry.swift): Thread-safe metrics collector capturing:
    • End-to-end timing (TTFT, prefill tok/s, decode tok/s, total tokens)
    • CPU phase breakdown (token prep, scheduling, cache lookup, router handoff, SSD prep, sampling, GPU sync wait)
    • Expert cache efficiency (slots/layer, hits, misses, hit rate %, evictions, loads per layer & expert)
    • SSD I/O stats (read ops, total bytes, avg/min/max read latency, peak concurrency, effective MB/s)
    • Metal GPU interval timings (cb1, cb2, shared expert FFN, LM head)
    • System memory footprint (Process RSS, common weights, expert cache capacity, KV cache, prefill scratch)
  • CLI Options (TurboFieldfareCLI):
    • --profile: Formatted ASCII text profiling summary table printed to stdout after generation.
    • --profile-json: Machine-readable JSON metrics report printed to stdout.
  • Documentation (docs/PROFILING.md): Complete profiling architecture, CLI guide, schema definitions, and benchmarking methodology.
  • Unit Tests (ProfilingTests.swift): 100% test coverage verifying metrics accumulation, thread safety, zero-overhead guards, and format outputs.

Verification

  • ./Scripts/test.sh: Passed 14/14 unit tests cleanly.
  • swift build -c release: Built release executables successfully.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant