Capacity Blocks for ML: instance-count gating, CB-only types, eligibility allowlist, 24h floor, utilization break-even

[scttfrdmn]

Summary

truffle capacity-blocks / truffle capacity --blocks surface Capacity Block
offerings, but a few behaviours of DescribeCapacityBlockOfferings are easy to
get wrong and produce misleading output. Filing what I verified live so the
handling can be hardened (and because the numbers are useful reference data).

All verified 2026-07-27, account 942542972736.

1. --instance-count silently gates results (highest-impact)

I initially concluded p5.48xlarge and p6-b200.48xlarge had zero capacity
block offerings. That was an artifact of asking for --instance-count 8. At
--instance-count 1, all four accelerator types had offerings:

p5.48xlarge      us-east-2 / us-west-2   24h  $ 996.67  = $41.53/inst-hr
p5e.48xlarge     us-west-2               24h  $1146.24  = $47.76/inst-hr
p5en.48xlarge    us-east-2 / us-west-2   24h  $1318.08  = $54.92/inst-hr
p6-b200.48xlarge us-west-2               24h  $2372.16  = $98.84/inst-hr

An empty result for a large count reads as "no capacity exists" when the truth is
"not that many at once." Probing count=1 first, or noting the count in the
"no offerings" message, would avoid a wrong conclusion. (us-east-1 genuinely has
0 offerings at every count I tried — region matters independently of count.)

2. Capacity blocks routinely UNDERCUT on-demand

Type	CB $/inst-hr	On-demand $/hr	CB discount
p5.48xlarge (8×H100)	$41.53	$55.04	−24.5%
p5en.48xlarge (8×H200)	$54.92	$63.30	−13.2%
p6-b200.48xlarge (8×B200)	$98.84	$113.93	−13.2%
p5e.48xlarge (8×H200)	$47.76	no on-demand price exists	—

The discount is per-family and reportedly drifts over time, so it shouldn't be
hardcoded anywhere — worth computing from the two live sources when both exist.

3. Some accelerator types are Capacity-Block-ONLY

p5e.48xlarge has zero AWS Price List rows of any kind — no OnDemand, no
CapacityBlock — yet DescribeInstanceTypeOfferings lists it in us-east-2
(2a/2b/2c) and us-west-2 (2c), and capacity blocks are purchasable at
$47.76/inst-hr. Any code path that resolves a price via the Price List first will
report this type as unpriced/unavailable when it is in fact buyable — and at a
rate that beats p5en.48xlarge for the same 8×H200 hardware.

4. "Capacity Blocks for ML" is a restricted per-region allowlist

GPU does not imply eligible. Rejected in us-west-2:

g7e.4xlarge     -> InvalidParameterValue: The instance type 'g7e.4xlarge' is not
                   supported for Capacity Blocks in 'us-west-2'
c7i.4xlarge     -> same
trn2.48xlarge   -> same   (Trainium, also rejected)

So the API distinguishes ineligible (error) from eligible but sold out
(empty list). Worth surfacing differently — "this type can't use capacity blocks
here" is a different answer for a user than "none available right now."

5. Duration constraints

• 24h minimum, 24h granularity. 1/4/6/8/12h are all rejected with
  InvalidParameterValue: The duration is not valid. 24/48/72h valid. 168h
  returned no offerings.
• Booking horizon ~9 days. DescribeCapacityBlockOfferings rejects end dates
  beyond it: The end date is not valid. End date must not be later than Wed Aug 05 14:47:53 UTC 2026. A helpful error message when a user asks for a
  window further out would beat passing the raw API error through.

6. Price List marketoption=CapacityBlock rows are always $0.0000

p6-b200.48xlarge  marketopt=CapacityBlock  capstatus=Used  $0.0000
p6-b200.48xlarge  marketopt=OnDemand       capstatus=Used  $113.9328

Real capacity block cost lives only in DescribeCapacityBlockOfferings as
UpfrontFee. Anything reading the Price List CapacityBlock rows will report free
H200s.

7. Don't assume the returned block matches the request

AWS can return shorter, partially-used blocks that a previous holder didn't
consume — potentially the best value on offer. I could not reproduce this in the
current window (every offering came back exactly as requested), but the safe
handling is to read CapacityBlockDurationHours, StartDate, EndDate and
InstanceCount off each offering and compute from those rather than from the
request parameters.

8. Worth surfacing: utilization break-even

Because the block is prepaid and non-refundable, effective cost is
UpfrontFee ÷ hours actually used. For p5.48xlarge:

used of 24h	effective $/used-hr	vs on-demand
24h	$41.53	−25%
18h	$55.37	+1%
12h	$83.06	+51%
8h	$124.59	+126%
2h	$498.36	+805%

Break-even is 18.1h of 24h (75% utilization.) Below that on-demand is cheaper
despite the headline discount. A short burst job on a capacity block can pay 5–8×
the on-demand rate per useful hour. Given truffle's cost-advisory role, a note or
--utilization hint alongside the upfront fee would help users avoid this.

Context

Found while building a Hugging Face model → AWS deployment tool that compares
Bedrock vs SageMaker vs EC2 (on-demand / spot / capacity block) for serving the
same model. Related: #107 (SageMaker Hosting component), #108 (obtainability).

[scttfrdmn]

Follow-up on finding #7 (shorter partially-used blocks): confirmed with a live
offering. I could not reproduce this when filing; it reproduced today.

us-west-2, p5e.48xlarge, --instance-count 1 --capacity-duration-hours 24:

cb-0630487666239ed35  p5e.48xlarge  us-west-2c  dur=24h  fee=$1146.2400  start=2026-07-29T11:30:00Z
cb-093d91a1f5abd75fb  p5e.48xlarge  us-west-2c  dur=19h  fee=$935.3000   start=2026-07-27T15:55:00Z

A 19-hour block returned for a 24-hour request, starting ~25 minutes out. So
CapacityBlockDurationHours must be read off each offering; dividing the fee by
the requested duration overstates the rate by 26% here.

One correction to what I wrote in the original issue — I assumed partials would be
the bargain case. They are not, at least not on rate:

offering	duration	fee	$/instance-hr
cb-0630487666239ed35	24h	$1146.24	$47.76
cb-093d91a1f5abd75fb	19h	$935.30	$49.23

The partial is 3% more expensive per hour. Its real advantages are a lower
absolute outlay ($935 vs $1,146) and a near-immediate start (25 minutes vs ~2
days) — which for a short benchmark or a deadline is often worth more than the
rate. Either way, partials should be surfaced as distinct candidates with their own
computed rate, start time, and total, rather than ranked on $/hr alone or filtered
out for not matching the requested duration.

This also means duration granularity is not the "24h increments" I described:
19h is neither a multiple of 24 nor of 12. The 24h minimum applies to what you can
request; what you can be offered is arbitrary. Worth asserting in tests that a
non-multiple-of-24 duration round-trips correctly.

(Also filed #110 — the offerings call drops per-region errors, so an ineligible or
AccessDenied region currently looks identical to one with no offerings.)

[scttfrdmn]

Status check across this issue's 8 findings before deciding what (if anything) to implement this pass — 3 are already resolved by earlier/this-session work, the rest are real but larger than a single-session fix, so leaving them open rather than doing a partial, rushed pass at all five.

Already resolved:

• Library API: on-demand pricing + a HourlyRate(type, region, model) convenience for embedders #1 (--instance-count silently gates results) — fixed in feat(available): report obtainability; name the count when capacity blocks are empty (#108, #109) #113: noOfferingsMessage (cmd/capacity_blocks.go) now states the requested count and region explicitly and suggests retrying with --count 1 when count > 1, instead of the old generic "no reservations found" message.
• fix(cli): only print human summary for table output #4 (ineligible vs. sold-out) — fixed in GetCapacityBlockOfferings drops per-region errors (regression of #63) — 'no capacity' is indistinguishable from AccessDenied or ineligible #110 (just merged): GetCapacityBlockOfferings now returns a distinct ErrCapacityBlockIneligible (checkable via errors.Is) when AWS's InvalidParameterValue: ... not supported for Capacity Blocks ... means the type can never use Capacity Blocks here, vs. a real query failure or genuinely-zero-right-now. Verified live against the exact three types this issue names as rejected (g7e.4xlarge, c7i.4xlarge, trn2.48xlarge) — all classify correctly.
• ci: add coverage ratchet floor (28%) #7 (don't assume the returned block matches the request) — already correct in the current code: getRegionCapacityBlockOfferings reads CapacityBlockDurationHours, StartDate, EndDate, and InstanceCount directly off each o in the AWS response, never from the request parameters. No change needed; noting it here since the issue flagged it as worth verifying.

Still open, not attempted this pass (each is its own real scope):

• feat(pricing): live on-demand pricing, wire ShowSavings, add HourlyRate #2 (live discount vs. on-demand) — needs a computed comparison between GetCapacityBlockOfferings' UpfrontFee and the existing on-demand pricer for the same type, surfaced as a column/note. Straightforward once scoped, but touches the CLI output shape (cmd/capacity_blocks.go + pkg/output), so deserves its own issue-sized PR rather than folding into a batch.
• Human-readable summary pollutes --output json/csv/yaml on spot/az/capacity #3 (CB-only types with zero Price List rows) — p5e.48xlarge-class types need any pricing-resolution code path to fall back to "ask GetCapacityBlockOfferings directly" rather than reporting unpriced/unavailable when the Price List has nothing. This is a design decision about where that fallback lives (the on-demand pricer? a new combined resolver?) more than a small patch.
• docs: document on-demand pricing API in README #5 (duration/booking-horizon errors) — roundCapacityBlockDuration already normalizes an arbitrary --duration-hours to a valid step, but the ~9-day booking-horizon rejection (The end date is not valid...) still passes through as a raw AWS error rather than a helpful message. Small, but wants its own verification of the current horizon (it may drift from "~9 days").
• Raise test coverage — cmd/ untested, aggregate 30.8% #6 (Price List CapacityBlock rows are always $0) — a correctness trap for any future code that reads Price List rows filtered by marketoption=CapacityBlock expecting a real price. Nothing in the current codebase does this today (confirmed via grep), so there's no live bug to fix — recording it here as a landmine for whoever builds feat(pricing): live on-demand pricing, wire ShowSavings, add HourlyRate #2/Human-readable summary pollutes --output json/csv/yaml on spot/az/capacity #3 above, so they don't reach for the wrong price source.
• capacity --gpu-only misses p4d/p5 (family map keys on p4/p5, not p4d/p5) #8 (utilization break-even) — a --utilization hint or note showing effective $/used-hour vs. on-demand, given the block is prepaid and non-refundable. This is genuinely useful (the break-even math in the issue, ~75% utilization, is a real and non-obvious threshold) but is a new advisory feature, not a bug fix, and belongs in its own PR with its own design discussion (e.g. does it default to showing utilization at 100%/50%/etc., or take an explicit hours-used flag).

Leaving this issue open for #2/#3/#5/#8; #6 is a note rather than an action item. Happy to pick any one of these up as a focused follow-on if prioritized.