Skip to content

Upgrade to RubyLLM 1.16 and adopt new capabilities#34

Merged
adham90 merged 2 commits into
mainfrom
adham90/upgrade-ruby-llm-1.16
Jun 21, 2026
Merged

Upgrade to RubyLLM 1.16 and adopt new capabilities#34
adham90 merged 2 commits into
mainfrom
adham90/upgrade-ruby-llm-1.16

Conversation

@adham90

@adham90 adham90 commented Jun 21, 2026

Copy link
Copy Markdown
Owner

Summary

Bumps the dependency to ruby_llm >= 1.16.0 and adopts four capabilities that landed in RubyLLM 1.15/1.16, closing real gaps in cost analytics, tooling, and instrumentation. Fully backwards compatible — no schema migration required.

Feature What it does
Accurate cost total_cost now prices prompt-cache reads, cache writes, and reasoning/thinking tokens via RubyLLM::Cost on top of authoritative (attempt-aggregated) text tokens. Per-component breakdown stored in executions.metadata["cost_breakdown"]. The async ExecutionLoggerJob no longer recomputes a text-only total over an accurate one.
Concurrent tools New inheritable tool_concurrency DSL on BaseAgent (false/true/:threads/:fibers); passed as concurrency: to with_tools when set, else RubyLLM's global default.
HTTP instrumentation Subscribes to RubyLLM's request.ruby_llm events during the LLM call and records real provider latency + count into executions.metadata (llm_request_ms, llm_request_count), distinct from total pipeline duration.
Config passthroughs Forwards bedrock/mistral/perplexity/vertexai/xai_api_base, faraday_adapter, deprecation_behavior, and tool_concurrency to RubyLLM.config.

Already-supported 1.15/1.16 features (image editing, transcription word timing, additive callbacks) needed no changes.

Design notes

  • No schema change. The primary instrumentation path persists context.total_cost directly, so making it accurate flows straight through the existing total_cost column. The cache/reasoning breakdown lives in metadata rather than new columns, avoiding a host-app migration. Promoting the breakdown to dedicated cost columns + dashboard display is noted as a follow-up.
  • Text input/output cost is unchanged (priced from context tokens); cache/reasoning is additive and degrades gracefully when the registry lacks a price.
  • Also deletes an orphan WIP spec (attachments_spec) that referenced a nonexistent DSL::Attachments module and crashed the suite at load time.

Testing

  • Full suite: 4689 examples, 0 failures (+19 new specs across the four features plus an end-to-end integration test that runs a real agent through the pipeline and asserts the cache-aware total persists to the DB).
  • standardrb clean.
  • Line coverage 90.65%.

Docs updated: CHANGELOG, README requirement, and the Configuration / Agent-DSL / Tools / Pricing / ActiveSupport-Notifications / Execution-Tracking wiki pages. An internal ADR is in changelog/.

🤖 Generated with Claude Code

adham90 and others added 2 commits June 21, 2026 22:53
Bump the dependency to ruby_llm >= 1.16.0 and adopt four features that
landed in 1.15/1.16, closing real gaps in cost analytics, tooling, and
instrumentation:

- Accurate cost: price cache-read/cache-write/reasoning components via
  RubyLLM::Cost on top of authoritative (attempt-aggregated) text tokens,
  folding them into total_cost and recording the breakdown in execution
  metadata. No schema change; the async job no longer recomputes a
  text-only total over an accurate one.
- Concurrent tools: new inheritable tool_concurrency DSL on BaseAgent,
  passed as concurrency: to with_tools when set (else RubyLLM's global).
- HTTP instrumentation: subscribe to ruby_llm's request.ruby_llm events
  during the LLM call and record real provider latency/count into
  execution metadata (llm_request_ms, llm_request_count).
- Config passthroughs: forward bedrock/mistral/perplexity/vertexai/xai
  api_base, faraday_adapter, deprecation_behavior, and tool_concurrency.

Also delete an orphan WIP spec (attachments_spec) that referenced a
nonexistent DSL::Attachments module and crashed the suite at load time.

Full suite: 4689 examples, 0 failures. standardrb clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- CHANGELOG: add Unreleased section for the four new capabilities
- README: bump RubyLLM requirement to >= 1.16.0
- wiki/Configuration: new forwarded knobs (provider api_base, faraday_adapter,
  deprecation_behavior, tool_concurrency)
- wiki/Agent-DSL + wiki/Tools: tool_concurrency DSL and concurrent execution
- wiki/Pricing: cache- and reasoning-aware token costs + cost_breakdown
- wiki/ActiveSupport-Notifications: RubyLLM library events and the captured
  llm_request_ms / llm_request_count latency metadata
- wiki/Execution-Tracking: document the new metadata fields

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@adham90 adham90 merged commit a987d58 into main Jun 21, 2026
4 checks passed
adham90 added a commit that referenced this pull request Jun 21, 2026
Five issues found reviewing the RubyLLM 1.16 upgrade (PR #34):

1. Scope llm_request_count/ms to the originating execution. The
   request.ruby_llm subscription was process-global, so the metric counted
   HTTP requests from concurrent threads and nested sub-agents. Now uses a
   fiber-local stack crediting only the innermost active capture on the
   emitting thread.

2. Stop double-charging separately-priced reasoning tokens. Providers fold
   reasoning tokens into output_tokens; when a model prices reasoning apart
   from output, those tokens are now excluded from the output charge since
   extra_token_costs already bills them at the reasoning rate.

3. Persist input_cost/output_cost and keep the cache-aware total. They were
   dropped on the async-logging path; now persisted, and total_cost is
   derived as input + output + metadata["cost_breakdown"] so the
   cache/reasoning extra is not clobbered by the before_save callback.

4. Reconcile cost_breakdown with total_cost by summing the rounded
   components instead of the unrounded sum.

5. Tighten bare rescues in extra_token_costs/response_cost to capture and
   debug-log the swallowed error.

extra_component_cost ignores non-numeric breakdown values so user-supplied
agent metadata can never break execution persistence.

Full suite green (4696 examples), standardrb clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant