diff --git a/CHANGELOG.md b/CHANGELOG.md index 1a31350..88e7170 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,23 @@ # Changelog +## 1.1.0 — unreleased candidate + +- Scoped typed reads with cumulative reports, native model projection and guarded + cursor invalidation; explicit numeric fidelity and absent/null presence. +- Native JSON value-coding conveniences and immutable exact Version 1 payload. +- Metered adaptive exact-key metadata and incremental duplicate recognition; + ordered members and exact collision confirmation remain authoritative. +- macOS/iOS 27 deployment minima and corrected schema bundle layout; public + SDK/simulator and disk-backed SwiftData consumers. Full physical-device + qualification follows the approved 1.1 matrix: physical iPhone required, iPad + SDK/simulator acceptance with physical iPad deferred. No tag/publication follows. +- Correct native ISO-8601 calendar fields and strict Base64 unused padding bits. + See API migration guidance; no limits/profile weights or dependencies raised. +- Release failed Base64 scratch from logical retention while preserving consumed + work and terminal operational errors. Add 19 acceptance/conformance tests, + public mobile recovery checks and a focused debug/release/ASAN QA lane. + + ## 1.0.0 — 2026-10-11 First stable release. diff --git a/Documentation/Architecture/API.md b/Documentation/Architecture/API.md index 453c05e..9df8e3c 100644 --- a/Documentation/Architecture/API.md +++ b/Documentation/Architecture/API.md @@ -35,7 +35,7 @@ order by default. Exact duplicates fail checked construction and parsing. | `JSONObject.build(limits:_:)` | Exclusive noncopyable member accumulator; checks before append | | `JSON.buildArray(limits:_:)` | Exclusive array accumulator; nested arrays remain one element | | `append(contentsOf:)` on array accumulator | Explicit flattening with charged work | -| Object subscripts by String/JSONKey | Unmetered linear exact-key lookup; nil means absence | +| Object subscripts by String/JSONKey | Unmetered exact-key lookup using immutable metadata; nil means absence | | `JSON.requiredMember(_:limits:)` | Bounded lookup; throws on wrong kind or missing member | | `JSON.presence(of:limits:)` | Bounded lookup separating absent, null and non-null value | | `JSONObject.presence(of:)` | Unmetered convenience with the same three states | @@ -168,3 +168,25 @@ subsystem ceilings independently. WorkUnits includes subsystem work, so subtract once. Failure/cancellation terminates the composition because no successful report accounts for failed work. The [actual consumer interface](../ConsumerInterface.md) and runnable consumer demonstrate this contract. + +## Additive 1.1 candidate + +JSON and JSONDocument provide `read(limits:_:)` and `readWithReport(limits:_:)` +with synchronous non-Sendable JSONReadCursor. Throwing String/JSONKey/Int brackets, +required kind/integer access, explicit Double/Decimal policies, raw/scalar/model +presence and native projection share one active meter. Paths are typed and bounded; +JSONAccessFailure carries the complete path and structured cause. Scope exit +invalidates handles and releases root/source ownership. Resource/cancellation +failures remain terminal when caught by a caller. See [reading](Release1.1/Reading.md). + +JSON `init(encoding:coding:)`, `decode(_:coding:)` and `decodeWithReport(_:coding:)` +delegate to existing native facades. JSONCodablePayload provides +`init(preserving:limits:)`, `preservingWithReport(_:limits:)`, `value(limits:)` +and `valueWithReport(limits:)`. Its explicit Codable Version 1 stores validated +UTF-8 only and shares native parent budgets. See [persistence](Release1.1/Persistence.md). + +The native ISO-8601 Date strategy now includes calendar fields as well as fractional +UTC time. Prior time-only stored values cannot recover their original calendar +date; application migration needs an independent source of that information. +Noncanonical Base64 with nonzero unused padding bits now rejects. These corrections +restore the documented strict/native contracts; they do not add a permissive mode. diff --git a/Documentation/Architecture/Release1.1/BenchmarkReview.md b/Documentation/Architecture/Release1.1/BenchmarkReview.md new file mode 100644 index 0000000..9e0b159 --- /dev/null +++ b/Documentation/Architecture/Release1.1/BenchmarkReview.md @@ -0,0 +1,56 @@ +# Initial exact-key lookup review + +This increment filters the tail of a linear lookup using existing immutable key +fingerprints. Objects below 32 members retain the scalar loop. Wide objects compare +the first eight members scalarly, then hash the query once under the active meter. +Every candidate hit still confirms exact UTF-8 identity. No object representation, +publication path, duplicate detector, capacity weight or public API changes here. +The array-backed AVL and controlled hash prototypes remain experiment subjects. + +The matched production-stage comparison used MacBookPro18,2, Apple M1 Max, 32 GiB, +macOS 27.0, Xcode 27.2 / SDK 27.2 and Swift 6.4 in language mode 6, release `-O`. +`Tools/Release1_1Probes/LookupBenchmark.swift` covers ten widths, three key shapes, +first/middle/last/missing queries and the first filtered member. Each run has three +warmups and nine measured samples, twenty lookups with result checks per sample: +129 query cells and 1161 recorded samples. The balanced/hash prototype measurements +are a separate corpus; their timing is not presented as production improvement. + +Two sequential matched production pairs were retained, together with an earlier +candidate measurement. The table uses the repeated pair. Times are medians in +milliseconds for twenty lookups, including identical result checks; work counts +are owned package units, excluding the benchmark's result-check work. + +| Shape / width / query | Scalar ms | Filter ms | Change | Scalar / filter work | +| --- | ---: | ---: | ---: | ---: | +| ASCII / 8 / last | 0.018041 | 0.018083 | +0.23% | 1620 / 1620 | +| ASCII / 16 / middle | 0.020166 | 0.020167 | approximately 0% | 1820 / 1820 | +| ASCII / 32 / first filtered | 0.020209 | 0.023750 | +17.52% | 1820 / 2020 | +| ASCII / 32 / miss | 0.017584 | 0.019375 | +10.19% | 1280 / 1400 | +| Prefix / 32 / first filtered | 0.230042 | 0.252583 | +9.80% | 24320 / 27020 | +| ASCII / 1024 / last | 1.657416 | 0.525750 | -68.28% | 144420 / 42140 | +| Prefix / 1024 / last | 24.807875 | 0.848333 | -96.58% | 2704420 / 67140 | +| ASCII / 10000 / last | 16.837083 | 5.013375 | -70.22% | 1422220 / 401180 | +| Prefix / 10000 / last | 248.313000 | 6.156417 | -97.52% | 26422220 / 426180 | + +The unchanged 10% review trigger applies. Accept the observed boundary cost for +this focused draft increment: query hashing adds ten work units to the first +filtered ASCII hit and six to the short missing key, approximately 0.18 and 0.09 +microseconds per lookup in the repeated pair. Later wide hits materially reduce +owned comparisons. This is an explicit tradeoff, not an assertion that every +lookup improves. Keep the scalar path and qualify adaptive metadata separately. + +Timings are host observations. An initial width-16 ASCII timing outlier disappeared +in the unchanged-code repeat, and wide scalar timings varied between runs; retain +both raw sets and variance instead of attributing those changes to the algorithm. +Empty-loop timings are at timer/call-overhead scale and establish no throughput +claim. This stage retains no owned buffer and reports zero logical retained bytes; +it measures neither process RSS nor allocator counts. Construction, whole-root +admission, native keyed encoding, thermal/energy effects and mobile crossovers +remain unqualified R1/R5 work. Algorithm costs may change resource failure points; +all finite profiles and counter version/weights remain unchanged. + +The historical stream median 1.0749626803 seconds versus 0.968493986 seconds +(+10.9932%) retains its existing accepted disposition and 10% trigger. This lookup +stage gives no stream attribution or fresh full-workload release qualification. +Raw samples, source/input hashes, command receipts and medians/variance are retained +in the milestone return bundle, with compiler, SDK, CI and fixture evidence. diff --git a/Documentation/Architecture/Release1.1/Decisions.md b/Documentation/Architecture/Release1.1/Decisions.md new file mode 100644 index 0000000..39517b5 --- /dev/null +++ b/Documentation/Architecture/Release1.1/Decisions.md @@ -0,0 +1,139 @@ +# Current 1.1 decisions + +The sections below preserve the initial R0/first-filter stage history. Current +successors are [Index](Index.md), [IndexMeasurements](IndexMeasurements.md), +[Reading](Reading.md), [Persistence](Persistence.md) and [Mobile](Mobile.md). +Production selects the controlled adaptive hash index with exact confirmation, +32-member threshold and first-eight scalar lookup path. Every published object +owns valid immutable metadata; construction/native keyed storage maintain it. +The guarded reader fallback, one cumulative meter and fixed Version 1 payload +are implemented. Resource packaging is corrected under the default build engine. + +SIMD and independent-work parallel batches remain deferred conditional experiments. +The scalar path and checked Sendable immutable sharing remain. No measured mobile +crossover/energy evidence supports shipping a new accelerator, and the selected +wide-object change supplies the demonstrated workload benefit. This is a scope +decision, not completed SIMD/parallel qualification. E11/E12 remain deferred. +The maintainer's 2026-10-12 decision accepts SDK and iPad simulator qualification +for 1.1.0. Physical iPad execution is deferred and does not block R5/R8; simulator +results remain distinct from physical qualification. Physical iPhone evidence and +the three protected Native check names remain required. The exact amended scope +is [QualificationPolicy.json](QualificationPolicy.json). + +# R0 architecture decisions + +## Reader ownership and declarations + +On Apple Swift 6.4 (swiftlang-6.4.0.34.1), ordinary returned `~Escapable` cursors +are rejected. Explicit `@_lifetime` annotations require the experimental Lifetimes +feature. Preserve the current language configuration. Select a guarded synchronous +scope as the reference implementation; no lifetime flag or unsafe annotation is +added to the libraries. See the compiler fixtures in Tools/Release1_1Probes. + +The actual compiling reference declarations are `ProbeReadCursor` with throwing +String, JSONKey and Int subscripts, `requiredString()`, generic +`requiredInteger(as:)`, `presence(of: JSONKey)` and `decode(_:)`, plus generic +`probeReadWithReport(_:limits:_:) -> JSONOperationReport`. These are compiled +into a separate ReaderProbe module and Swift Testing target, not the JSON product. +The consumer imports the public JSON model and ReaderProbe; it imports no meter. +No complete production cursor API is frozen by these subset declarations. + +One JSONCodingState owns one noncopyable JSONWorkMeter in domain core. Selection, +path work, integer conversion and JSONNativeDecoder projection borrow that state; +root admission occurs once. A cursor retains only its scope and typed path. Scope +exit invalidates access and clears root/document/source and coding-state references. +Resource and cancellation failures latch, including when the callback catches them. +Ordinary missing-member failures may be caught without refunding charged work. + +Swift 6 rejects Sendable closure capture and actual detached-task transfer. The +latter must be tested with object compilation: typecheck alone does not execute +region-based ownership analysis. Escaped synchronous handles are permitted by the +fallback shape but all later operations fail before root/meter access. + +R2/R3 must still qualify the full typed presence/conversion surface, mapped-source +diagnostics, live retention/refunds and stricter projection limits against already +consumed counters. The current meter's immutable limits require a reviewed +restriction adapter; copying/replacing the meter is prohibited. No fresh decoder +facade is permitted inside a scoped projection. The prototype intentionally does +not claim these unimplemented behaviors. + +## Index candidates and publication + +Tests exercise a scalar reference, fingerprint-assisted linear reference, +array-backed AVL tree and controlled open-address hash table. All confirm exact +UTF-8 identity after candidate filtering, including injected same-fingerprint +collisions. Tree rotations, searches, table scans/rebuilds, key bytes and spare +capacity use the actual core meter. New capacities are admitted while old storage +coexists; owned loops check cancellation. These are test-only candidates. + +The release-native corpus covers widths 0, 1, 4, 8, 16, 32, 64, 256, 1024 and 10000, +ASCII, 128-byte common prefixes and scalar-distinct Unicode. It records three +warmups, nine samples, one construction per sample and three sets of four lookup +queries. Construction and lookup work/timing are separate. A resource refusal +stops that comparison and is never counted as a successful fast sample. + +No adaptive threshold or wide representation is selected yet. Mac measurements +cannot select a mobile crossover. The next small increment may filter a linear +lookup by its existing cached fingerprint, with exact comparison retained. R1 +must audit every publication path, incremental duplicate timing, nonthrowing finish, +keyed encoder traversal/allocation, failure cleanup and immutable Sendable metadata. +Existing 32-byte index-slot reservations include grammar-owned duplicate slots, +materialized object/member slots and admitted graph slots; these are distinct +ownerships. They do not automatically cover new simultaneous buffers. Version 1 +weights and all 49 metric caps remain unchanged. + +## Version 1 persistence and SDK boundary + +Freeze the logical envelope: String format exactly `swift-json-value`, integer +version exactly 1, String utf8Base64 strict standard Base64 of one complete strict +UTF-8 JSON document. Inner writing is compact, insertion order, preserved admitted +number lexemes with the existing fallback. The 0.1 fixture is `MC4x`; present JSON +null is `bnVsbA==` and differs from an optional absent payload. + +SwiftDataShape explicitly implements these fixed Codable fields and compiles +nonoptional and optional `@Attribute(.codable)` attributes on macOS, iPhoneOS and +iPhoneSimulator SDK 27.2. It is an SDK shape subject, not the exact owned codec: +strict Base64/inner JSON validation, native nested budgets and fidelity remain R4. +The later persistence test must save to a unique disk URL, leave the scope holding +all models/context/container, recreate the container, fetch and check scalar bytes, +member order and original lexemes. Repeat update/reopen and query/sort ordinary +fields. In-memory rereading is not evidence; no cloud-sync claim is intended. + +## Mobile platform and CI + +The mobile feature manifest declares `.macOS("27.0")` and `.iOS("27.0")`; +iPadOS shares iOS. See [mobile qualification](Mobile.md) for the packaging fix +and the distinction between deployment minima and executed support evidence. All four products, private +regex and schema resources compile for iPhoneOS with the default build engine. +The simulator default engine fails while signing the schema resource bundle; its +`Resources` directory layout needs qualification. The documented deprecated native +SwiftPM backend compiles the same simulator consumer without that signing step. +This alternate build is compile evidence, not resolution of the default-engine +failure. The saved inventory lists iOS 27.0 (24A5390f) and eight available +simulator devices. Runtime behavior has not been executed in this milestone; +availability is not runtime qualification. + +Keep the three protected check names and strict/no-bypass rules unchanged. The +seven authored prototype cases extend JSONTests from 54 to 61 functions: all other +bundle cardinalities, three targeted Thread Sanitizer cases, 49 exact cap probes, +independent corpus counts and exclusions remain intact. R0's filtered run requires +exactly seven cases. New platform declarations must later update the exact +macOS-only manifest expectation in the same focused change. + +## First production increment + +Select only the narrow cached-fingerprint filter described in +[BenchmarkReview](BenchmarkReview.md). The below-32-member scalar path and first +eight scalar probes remain available. A query hash, each member probe, fingerprint +comparison and exact candidate comparison consume the same noncopyable meter; +there is no new table, cached mutable state or owned index allocation. The two +additional behavior cases exercise public presence/Pointer/equality, insertion +order, forced collision confirmation in the production comparator, inclusive and +cumulative work exhaustion and cancellation during actual lookup work. + +The current 10% performance review trigger is unchanged; accept the measured +first-filtered-hit and short-miss cost for this limited draft change. Full adaptive +index publication/construction and keyed-builder/mobile measurement remain R1/R5, +not completed by these tests. CI now expects JSONTests 63 and total 131 functions; +the seven-case R0 subset, baseline parameterized invocations and all other exact +cardinality/49-cap checks remain unchanged. diff --git a/Documentation/Architecture/Release1.1/Index.md b/Documentation/Architecture/Release1.1/Index.md new file mode 100644 index 0000000..4b6c057 --- /dev/null +++ b/Documentation/Architecture/Release1.1/Index.md @@ -0,0 +1,48 @@ +# Exact-key construction and publication + +Ordered members remain the public collection and writer authority. Each published +object also owns immutable exact-key metadata. The index stores decoded JSONKey +identity and insertion positions, never canonically equating String dictionary keys. + +The R0 release-native comparisons favored controlled open addressing for wide +objects. The current candidate uses a linear index below 32 keys, then a power-of-two +table with occupancy at most one half. Lookups keep the scalar path for small +objects and the first eight members. A wide lookup hashes its query once, probes +the table and confirms exact UTF-8 on every candidate hit. Collision scans are +finite and metered; this is no adversarial constant-time guarantee. + +Each metadata capacity slot retains the existing Version 1 `indexSlot` weight of +32 logical bytes. Unused capacity is included. New capacity is admitted while old +capacity remains live; copying, initialization, rehashing and exact probes consume +the same meter. Rejected growth releases its replacement reservation and leaves +prior metadata valid. Logical capacities do not assert allocator or RSS ceilings. + +The shared grammar maintains its own duplicate index at decoded key recognition. +A duplicate still fails at the second encoded key's start before its colon/value. +Materialized objects maintain separate metadata as keys arrive. Checked +construction and the accumulator maintain their index before publication; consuming +nonthrowing `finish` transfers it without search, allocation or a fresh budget. +Editing, native materialization and schema inspection/output build metered metadata +before returning a value. Empty Merge Patch targets use an explicit empty index. + +Grammar duplicate capacity and DOM metadata are distinct live owners. Full owned +graph admission includes the published index's actual logical capacity and member +slots separately. The metric and weight meanings stay at Version 1; capacity/work +counts legitimately differ with the algorithm, and defaults are not raised. + +Native keyed encoding now keeps one invocation-owned mutable storage object and +maintained exact index. Updating a field no longer extracts and rewrites the entire +member array. Member growth admits old/new capacities together. Materialization +charges key traversal and builds immutable output metadata under the same meter. +This source observation alone makes no claim of one allocation per field. + +`IndexPublicationTests` covers all JSON publication routes, scalar-distinct keys, +forced collisions, failed rehash, duplicate timing and cancellation. The separate +release construction benchmark measures checked, parser, accumulator and native +keyed paths over 120 cells, using the matched finite offline profile. It records +resource refusals separately, and allocator snapshots separately from timing and +logical retention. A successful SDK build does not establish a mobile crossover. + +Full R1 closure requires the actual final native/safety/consumer measurements and +review receipts. The test-only AVL/scalar alternatives remain qualification +references; no table, platform or throughput result is inferred from this prose. diff --git a/Documentation/Architecture/Release1.1/IndexMeasurements.md b/Documentation/Architecture/Release1.1/IndexMeasurements.md new file mode 100644 index 0000000..e741757 --- /dev/null +++ b/Documentation/Architecture/Release1.1/IndexMeasurements.md @@ -0,0 +1,93 @@ +# Production index stage measurements + +The matched predecessor core is `5d01f88` (mobile branch `bf0ff41` changes only +resource packaging/platform documentation). The metadata/keyed-storage core is +`bf37941`. Swift 6.4, SDK 27.2, macOS 27.0, M1 Max/32 GiB, release optimization, +finite offline profile, three warmups and nine measured samples apply to both. +Exact benchmark source and static-library digests accompany the private native +receipts. Documentation/test-only successors reuse these bytes explicitly. + +Construction measures 120 cells: four paths, ten widths 0–10000, three key shapes. +The predecessor has 1044 successful samples and four explicit long-prefix +width-10000 work refusals. The candidate has 1080 successful samples. Never use a +refusal's elapsed time as successful throughput. Accumulator timing is available; +its public facade has no accounting report, so its work/retention is unavailable. + +| Stage / ASCII width | Before median ms | After median ms | Change | Before / after logical peak bytes | +| --- | ---: | ---: | ---: | ---: | +| Checked 8 | 0.0060 | 0.0039 | -35.9% | 808 / 808 | +| Checked 32 | 0.0510 | 0.0155 | -69.5% | 3136 / 5096 | +| Checked 1024 | 39.5464 | 0.4029 | -99.0% | 99360 / 132128 | +| Parse 1024 | 39.8769 | 1.1994 | -97.0% | 132160 / 197696 | +| Native keyed 1024 | 79.3653 | 1.4056 | -98.2% | 238624 / 336928 | +| Native keyed 10000 | 6930.3080 | 15.7802 | -99.8% | 2330032 / 4464688 | + +Lookup measures 129 query cells with 1161 samples per side, 20 lookups/sample. +The final serialized pair, after the overlapping compiler job completed, gives: + +| Query | Before / after median ms per 20 lookups | Change | Before / after work | +| --- | ---: | ---: | ---: | +| ASCII 8 last | 0.018042 / 0.018791 | +4.15% | 1620 / 1620 | +| ASCII 32 first filtered | 0.023833 / 0.022792 | -4.37% | 2020 / 2020 | +| ASCII 32 miss | 0.019375 / 0.007542 | -61.07% | 1400 / 580 | +| ASCII 1024 last | 0.526334 / 0.018541 | -96.48% | 42140 / 1540 | +| Prefix 10000 last | 6.195458 / 0.247250 | -96.01% | 426180 / 26580 | + +Retain and review every cell above the unchanged 10% trigger, including tiny +measurements near clock/call overhead. The disposition selects this candidate for +wide construction/lookup benefit with explicit additional spare-capacity retention. +It does not claim an improvement for every tiny case or a mobile crossover. +Construction/root admission/ordered rebuilding remain separate from lookup. + +Nine separately instrumented native-keyed cells record libmalloc live blocks/bytes +at selected field boundaries. Per-sample allocator observations and cumulative +`ru_maxrss` are process-wide; they are not one allocation per field or isolated +workload allocation counts. Logical weights remain Version 1 and are not RSS. + +First lookup samples were lost to a local receipt/output filename collision and +were rerun with distinct sample paths. An earlier retained pair overlapped compiler +work; the final serialized pair is used above. All actual failure/command records +remain evidence. These stage results precede the reader's shared-meter and ISO +correction; affected integrated performance must be refreshed on the final source. +The historical approximately 11% streaming disposition remains separate. + +## Complete trigger review + +All 11 construction and 16 lookup cells above 10% are retained below. +Construction absolute increases are at most 1.541 microseconds per operation; +lookup increases are at most 1.625 microseconds per twenty queries. These are +observed regressions, without a statistical noise or causal attribution claim. +Accept the small-object metadata/ownership cost for the measured wide-object +benefit, retaining the scalar lookup path below 32. Empty measurements establish +no throughput claim. Four predecessor refusals remain excluded from timing ratios. +The threshold stays 10%; mobile crossovers remain unqualified. + +| Operation / shape / width / query | Before / after microseconds | Change | Disposition | +| --- | ---: | ---: | --- | +| accumulator / prefix / 0 / — | 0.584 / 0.666 | +14.04% | No throughput claim; retain observation | +| checked / prefix / 0 / — | 0.792 / 0.875 | +10.48% | No throughput claim; retain observation | +| checked / unicode / 0 / — | 0.709 / 0.833 | +17.49% | No throughput claim; retain observation | +| checked / ascii / 1 / — | 1.125 / 1.417 | +25.96% | Accept measured small-case tradeoff | +| native-keyed / unicode / 0 / — | 2.125 / 2.416 | +13.69% | No throughput claim; retain observation | +| native-keyed / ascii / 1 / — | 4.041 / 4.542 | +12.40% | Accept measured small-case tradeoff | +| native-keyed / unicode / 1 / — | 3.667 / 4.042 | +10.23% | Accept measured small-case tradeoff | +| parse / ascii / 1 / — | 2.625 / 2.958 | +12.69% | Accept measured small-case tradeoff | +| parse / prefix / 1 / — | 11.792 / 13.333 | +13.07% | Accept measured small-case tradeoff | +| parse / unicode / 1 / — | 2.542 / 2.834 | +11.49% | Accept measured small-case tradeoff | +| parse / unicode / 4 / — | 6.500 / 7.500 | +15.38% | Accept measured small-case tradeoff | +| lookup / ascii / 0 / miss | 0.083 / 0.167 | +101.20% | No throughput claim; retain observation | +| lookup / prefix / 0 / miss | 0.042 / 0.167 | +297.62% | No throughput claim; retain observation | +| lookup / unicode / 0 / miss | 0.042 / 0.166 | +295.24% | No throughput claim; retain observation | +| lookup / ascii / 1 / miss | 0.625 / 0.875 | +40.00% | Accept measured small-case tradeoff | +| lookup / unicode / 1 / first | 3.542 / 3.916 | +10.56% | Accept measured small-case tradeoff | +| lookup / unicode / 1 / last | 3.542 / 3.916 | +10.56% | Accept measured small-case tradeoff | +| lookup / unicode / 1 / middle | 3.542 / 3.916 | +10.56% | Accept measured small-case tradeoff | +| lookup / unicode / 1 / miss | 0.583 / 0.875 | +50.09% | Accept measured small-case tradeoff | +| lookup / ascii / 4 / first | 2.875 / 3.208 | +11.58% | Accept measured small-case tradeoff | +| lookup / ascii / 4 / miss | 2.334 / 2.834 | +21.42% | Accept measured small-case tradeoff | +| lookup / unicode / 4 / miss | 2.333 / 2.833 | +21.43% | Accept measured small-case tradeoff | +| lookup / ascii / 8 / miss | 4.542 / 5.292 | +16.51% | Accept measured small-case tradeoff | +| lookup / unicode / 8 / miss | 4.542 / 5.292 | +16.51% | Accept measured small-case tradeoff | +| lookup / ascii / 16 / first | 2.916 / 3.208 | +10.01% | Accept measured small-case tradeoff | +| lookup / ascii / 16 / miss | 8.625 / 10.250 | +18.84% | Accept measured small-case tradeoff | +| lookup / unicode / 16 / miss | 8.916 / 9.959 | +11.70% | Accept measured small-case tradeoff | diff --git a/Documentation/Architecture/Release1.1/IntegratedPerformance.md b/Documentation/Architecture/Release1.1/IntegratedPerformance.md new file mode 100644 index 0000000..ba48326 --- /dev/null +++ b/Documentation/Architecture/Release1.1/IntegratedPerformance.md @@ -0,0 +1,68 @@ +# Integrated 1.1 performance disposition + +The unchanged twelve-workload public harness compares the released 1.0 core at +`a2733dd` (identical Sources/benchmark harness to tag 1.0.0) with the 1.1 core plus +capacity-history correction. M1 Max/32 GiB, macOS27.0, Swift6.4, Xcode27.2/SDK27.2, +release optimization, finite balanced profile, three warmups, nine samples and +three invocations/sample apply to both. Inputs/configuration hashes match exactly. +Measurements are serial, with compilation/safety workloads idle. + +An initial implementation recorded maximum admission history on every meter check +and regressed most workloads substantially. Retain those results. The correction +records history only in native coding/read scopes that can tighten limits; work is +already cumulative and live retention uses current ownership. Fixed-policy codecs +skip unused history writes. Refusal/cancellation/overflow remains terminal. The +capacity-history refusal test protects against silently weakening later ceilings. + +| Workload | Before / after ms | Change | Before / after logical peak bytes | +| --- | ---: | ---: | ---: | +| parse_small_exact | 0.018653 / 0.020917 | +12.14% | 649 / 721 | +| parse_owned_data | 0.023042 / 0.025722 | +11.63% | 0 / 0 | +| parse_large_exact_array | 21.826889 / 22.917903 | +5.00% | 320064 / 320064 | +| write_large_unicode_string | 100.354583 / 101.423028 | +1.06% | 1040004 / 1040004 | +| native_value_codable | 16.128958 / 16.260583 | +0.82% | 166332 / 166468 | +| exact_number_parse_compare | 1.459167 / 1.448500 | -0.73% | 0 / 0 | +| schema_compile_repeated_patterns | 36.223806 / 35.530042 | -1.92% | 103074 / 104642 | +| shared_plan_regex_evaluation | 0.352625 / 0.358167 | +1.57% | 1412 / 1412 | +| schema_exact_evaluation | 0.022833 / 0.023208 | +1.64% | 647 / 647 | +| staged_rfc6902 | 1.522097 / 1.531833 | +0.64% | 97100 / 97100 | +| rfc7396 | 0.052417 / 0.054500 | +3.97% | 1191 / 1255 | +| generated_root_array_stream | 1179.382305 / 1365.204694 | +15.76% | 52 / 52 | + +The unchanged 10% review trigger applies. Accept the observed small exact parse +(+12.14%, about 2.26 microseconds) and owned-Data parse (+11.63%, about 2.68 +microseconds) costs, including controlled duplicate/index bookkeeping and terminal +accounting. Small-parse logical peak rises 649 to721 (+11.09%): explicit additional +index capacity is covered by unchanged weights/limits. Accept the stream observation +(+15.76%, about1.86 microseconds/record) for this correctness/ownership increment, +with no throughput improvement claim or isolated causal attribution. Raw samples +and variance remain material evidence. No profile/10% threshold is raised. + +The historical stream 1.0749626803 versus0.968493986 seconds (+10.9932%) retains +its earlier disposition independently. The new matched stream pair is1.3652046943 +versus1.1793823053 seconds; do not compare either new run with a historical median +as if inputs/environment were a fresh matched pair. + +The separate wide-index construction/lookup corpus retains its full trigger review +and spare-capacity tradeoff in IndexMeasurements.md. This integrated corpus supplies +no new index crossover, SIMD/parallel speed claim or SwiftyJSON comparison. + +Physical iPhone14ProMax/iOS27.2 current-candidate public-product/payload work has +nine release samples after three warmups: median54.406542 ms and process peak RSS +24395776 bytes. This is a whole app/SDK peak, including UIKit, without isolated +allocation or hard-RSS claims, and no matched mobile optimization comparison. +Physical iPad measurements remain unexecuted and, under the subsequent +[1.1 qualification policy](QualificationPolicy.json), deferred and nonblocking. +Logical bytes, allocator observations, +process RSS and caller computation remain distinct. + +The additional QA wave changes only Base64 failure cleanup in the runtime. Affected +public Data/payload paths have three matched, alternating release-native process +pairs (27 samples per source/workload, 100 invocations/sample). Valid Data medians +range from −1.28% to +4.98%; payload decoding is +1.16%, corruption recovery −0.09%. +Input hashes/work counters match; no new 10% timing trigger is crossed. The corrected +recovery ledger reports final live zero instead of 300 and peak 16 instead of 300. +This is truthful accounting of freed scratch, without an RSS/allocator improvement +claim. The unchanged index and twelve-workload dispositions above are retained. +The expanded physical QA harness has different work and cannot be compared with +the earlier phone median as a matched speed result. See [QA](QA.md). diff --git a/Documentation/Architecture/Release1.1/Mobile.md b/Documentation/Architecture/Release1.1/Mobile.md new file mode 100644 index 0000000..95772c9 --- /dev/null +++ b/Documentation/Architecture/Release1.1/Mobile.md @@ -0,0 +1,41 @@ +# Apple mobile qualification + +The 1.1 feature manifest declares macOS 27 and iOS 27 minimum deployment versions. +iPadOS uses the iOS declaration. The native host still requires macOS/SDK 27 and +Swift tools/compiler 6.4, language mode 6. SDK compilation alone does not qualify +mobile runtime or physical-device behavior. + +The schema resource rule copies `Resources/Draft2020-12` as `Draft2020-12` at the +resource root. It retains every pinned schema byte and provenance/license record. +The loader uses the same path relative to `Bundle.module.resourceURL` on each +platform. A flat iOS bundle with a root directory literally named `Resources` and +a root Info.plist failed native codesign. A temporary-copy experiment reproduced +the failure and signed successfully after removing that extra directory level. +No signing bypass, custom plist, resolver or dependency is introduced. + +The mobile probe builds a public consumer of all four products, the private regex +implementation and packaged schemas with the default Swift Build engine for both +iPhoneOS and iPhoneSimulator. Required subsequent evidence includes iPhone/iPad +simulator behavior/cancellation, physical-device smoke and measurements, and real +disk-backed SwiftData teardown/reopen using the exact payload. Keep every not-run +row explicit; this document does not certify those later runs. + +The public runtime harness now executes exact numeric/scalar conversion, offline +plans/native regex, integrity-checked packaged schemas, Patch/Merge Patch, payload +fixtures, stream completion/terminal pulls, awaited sinks and cancellation. Mac +Swift Testing and isolated iPhone/iPad iOS 27 simulators execute three-process +disk save/reopen/update checks. The physical iPhone is connected with iOS 27.2 +and developer mode enabled; signing/runtime/performance are separately qualified. +The original R5 scope required a physical iPad. The maintainer's 2026-10-12 +[qualification policy](QualificationPolicy.json) now accepts the executed iPad +simulator public/disk checks for 1.1.0 and defers physical iPad testing without +blocking release preparation. Physical iPhone evidence remains required. + +A development-signed qualification wrapper now executes the exact public harness +on physical iPhone14ProMax/iOS27.2. Public smoke, eight fixtures, three separate app +processes for disk save/reopen/update and nine release measurements are materialized +as run-ID-bound files retrieved from its own container. The temporary app is +removed afterward. Core consumers/source remain independent of UIKit/signing; +no public UI product or server semantics are added. Physical iPad runtime, +performance, memory and energy remain untested. Simulator evidence qualifies the +agreed 1.1 iPadOS acceptance scope and makes no physical-device measurement claim. diff --git a/Documentation/Architecture/Release1.1/Persistence.md b/Documentation/Architecture/Release1.1/Persistence.md new file mode 100644 index 0000000..469d414 --- /dev/null +++ b/Documentation/Architecture/Release1.1/Persistence.md @@ -0,0 +1,46 @@ +# Exact Version 1 persistence + +The implemented public wrapper is immutable `JSONCodablePayload: Sendable, Codable`. +`init(preserving:limits:)`, `preservingWithReport(_:limits:)`, `value(limits:)` +and `valueWithReport(limits:)` use the existing owned codec. Private storage is +validated UTF-8 Data only, without a DOM cache, public byte mutation or equality. + +Version 1 requires format `swift-json-value`, integer version 1 and strict standard +Base64 `utf8Base64`. Compact insertion-order writing preserves admitted number +lexemes, with normalized fallback. Scalar sequences, exact decimals and array/member +order survive fixed-primitive foreign transport. Whitespace/escape/source-document +fidelity is outside this contract. Eight committed golden fixtures pin the envelope +and inner hashes; empty arrays/objects are additional behavior cases. + +Construction preflights prospective Base64, fixed envelope output and text sizes. +Explicit limits intersect balanced, never widen it. Decode admits text before +decoding, checks decoded size, certifies complete inner grammar/EOF and releases the +temporary DOM. Opening is a fresh bounded parse. Copies, simultaneous byte/text/DOM +ownership, work and cancellation share the enclosing native meter when nested. +Foreign callbacks/allocations remain caller work. No meter is borrowed across an +Encoder/Decoder callback. Logical retention is not an allocator or RSS promise. + +Required field types use ordinary Codable errors. Unknown marker/version and +malformed inner syntax/Base64 map to bounded corruption with a static description; +resource/configuration failures and cancellation retain their types. A caller that +catches an owned operational failure cannot certify successful native completion. +The shared Base64 decoder now rejects nonzero unused padding bits, closing a real +strict-codec gap. No legacy heuristic or corrupt-data-to-null fallback is added. + +`Tools/MobileQualification` imports the four public products and SwiftData outside +the core package. Three Swift Testing cases qualify public behavior, stream/sink +completion/cancellation and disk persistence. The disk case saves eight exact +fixtures, executes each save/reopen/update phase in a separate process, reopens at the +same URL, checks scalar bytes/order/lexemes, queries/sorts ordinary fields, updates, +releases again and verifies a third reopen. Optional nil and present JSON null are +distinct. SwiftData owns its database representation; embedded JSON predicates and +cloud synchronization are not claimed. Mac, simulator and physical-device outcomes +are separate receipts. Under the [amended 1.1 policy](QualificationPolicy.json), +macOS, both simulator families and physical iPhone disk phases remain required; +physical iPad disk execution is deferred and nonblocking. + +The first simulator release run failed the additional weak-container assertion: +SwiftData retained a container beyond lexical scope. The debug Mac run passed it. +Retain both outcomes; do not claim that SDK lifetime is controlled by this package. +The final disk harness uses three independently exited processes on Mac/simulators, +so reopening cannot read prior model/context/container memory or SDK caches. diff --git a/Documentation/Architecture/Release1.1/Plan.md b/Documentation/Architecture/Release1.1/Plan.md new file mode 100644 index 0000000..af5ee42 --- /dev/null +++ b/Documentation/Architecture/Release1.1/Plan.md @@ -0,0 +1,54 @@ +# Swift-JSON 1.1 additive release plan + +The released 1.0 API and behavior contracts remain authoritative. This plan records +reviewable work toward 1.1; prototype types under Tests and Tools are qualification +subjects, not additions to the four public libraries. + +| Branch | Review scope | Acceptance / gate | +| --- | --- | --- | +| chore/1.1-plan-and-qualification | Ownership and SDK probes, index corpus, finite experiment meters, test discovery adaptation | R0 | +| perf/1.1-exact-key-index | Measured exact-key candidate filtering, then adaptive metadata and incremental duplicate detection | I01–I08 / R1 | +| feat/1.1-scoped-read-access | Scoped typed navigation, bounded paths, guarded lifetime, terminal failure and reports | A01–A13 / R2 | +| feat/1.1-codable-value-bridge | Existing native coder conveniences and projection under the active scope meter | C01–C06 / R3 | +| feat/1.1-exact-codable-payload | Strict Version 1 envelope, finite owned codec and fixed compatibility fixtures | P01–P08 / R4 | +| feat/1.1-apple-mobile-qualification | macOS/iOS 27 minima, resources, simulator/device consumers, disk-backed SwiftData | S01–S04, M01–M06 / R5 | +| docs/1.1-release-readiness | Executed examples, migration/DocC, final integrated qualification and candidate | W01–W05 / R7–R8 | + +Create successors from reviewed prerequisites; disclose any stacked base. Each +change must preserve checked Sendable, strict UTF-8, exact numbers/scalar identity, +member order, cumulative finite accounting and zero root dependencies. Root +products stay JSON, JSONSchema, JSONStreaming and JSONPatch; regex stays private. +SwiftData belongs to a qualification example, never a core dependency. + +SIMD byte kernels and independent-work batches are conditional R6 experiments. +They remain unselected until scalar parity, ownership, aggregate quotas and native +measurements justify inclusion. E11 projection and E12 text sequences stay deferred. +The existing approximately 11% streaming regression retains its accepted disposition +and unchanged 10% review trigger. This milestone supplies no causal attribution. + +[Decisions](Decisions.md) records the executable prototype boundaries. Run: + +```sh +python3 Scripts/ci.py --lane r0 +python3 Scripts/ci.py --lane native +python3 Scripts/ci.py --lane safety +python3 Scripts/ci.py --lane docs +``` + +Native includes the ownership consumer and compiler-negative cases. R0 adds the +three SwiftData SDK shape probes. Mobile product builds and controlled benchmark +runs are explicit, separately reported operations: + +```sh +python3 Scripts/qualify_release_1_1.py --mode mobile --output .build/ci/mobile/receipt.json +python3 Scripts/qualify_release_1_1.py --mode index --output .build/ci/index/receipt.json +``` + +Keep actual logs/receipts under ignored build/evidence directories. Each records +source inputs, environment, command exits and counts. A failed or absent runtime +cannot advertise support. Inspect simulator resource signing separately from SDK +compilation. Physical iPhone observations and real disk teardown/reopen remain +required. The original plan also required a physical iPad; the explicit 2026-10-12 +[policy decision](QualificationPolicy.json) now accepts iPad simulator execution +for 1.1.0 and defers physical iPad qualification without blocking the release train. +No merge, tag or publication follows from this plan. diff --git a/Documentation/Architecture/Release1.1/Progress.json b/Documentation/Architecture/Release1.1/Progress.json new file mode 100644 index 0000000..2e4a647 --- /dev/null +++ b/Documentation/Architecture/Release1.1/Progress.json @@ -0,0 +1,80 @@ +{ + "schema_version": 1, + "target_release": "1.1.0", + "milestone": "maintainer-approved iPad simulator qualification policy", + "tested_core_sha": "5d655f2beab569ac804890493bfafcf6045caabe", + "preserved_history": [ + "1.0.0", + "W00-W12", + "G01-G12", + "RC01-RC08", + "F01-F06", + "J01-J04", + "R0 turn 001" + ], + "root_function_counts": { + "JSON": 97, + "JSONStreaming": 14, + "JSONSchema": 41, + "JSONRegexRuntime": 12, + "JSONPatch": 11 + }, + "root_function_total": 175, + "parameterized_invocations": 173, + "cap_probes": 49, + "readme_examples": 10, + "authored_public_docc_symbols": 582, + "payload_fixtures": 8, + "r5_matrix": { + "macOS": "executed", + "iPhone_simulator": "executed", + "iPad_simulator": "executed", + "physical_iPhone": "executed", + "physical_iPad": "deferred; not required for 1.1 release; no physical-device claims" + }, + "conditional_features": { + "SIMD": "deferred", + "parallel_batches": "deferred", + "E11": "deferred", + "E12": "deferred" + }, + "evidence_policy": "Turn003 materializes actual native/safety/QA/docs/SDK/simulator/iPhone receipts, raw measurements and reproducible source archive. Turn002 and original history remain immutable. Source bindings identify unchanged runtime bytes and expanded test cases.", + "publication_performed": false, + "remaining_release_blockers": [ + "Complete maintainer review and protected PR integration before final release readiness" + ], + "performance_review": "IntegratedPerformance.md; unchanged 10% trigger, retain initial regression measurements and final accepted dispositions", + "prior_milestones": [ + { + "turn": 2, + "tested_core_sha": "a48eb337b70eb075ce37c99dc3d150032f720d00", + "root_function_total": 156, + "source_manifest_sha256": "2b69a683002eaa4e112bd005b3eb06e63dc8e06bfcd2b5c8485f076870f26797", + "return_sha256": "151ce56037763e1ca3d6d0439521020f34ca368ef77b50c31b8ef4e27fb282b6" + }, + { + "turn": 3, + "source_sha": "c40c5ffba28ce7aeae926efea21e6c3be76c4787", + "return_sha256": "a50465cbfd30ae8a699117d589676bd5d9e3964c2584268d251d230911d5a823", + "physical_iPad_status": "blocked under the original scope; historical evidence retained" + } + ], + "qa_wave": { + "added_functions": 19, + "baseline_swift_test_files_retained": 48, + "scope": "QA.md", + "native_full": "passed on tested_core_sha", + "safety_full": "passed on tested_core_sha; TSAN5", + "mobile": "Mac/both SDKs/both simulators/physical iPhone executed; physical iPad deferred, nonblocking", + "supplemental_boundary_review": "passed at e6eb46da346bee9892119fdc414abd7783e66b2c: 19 exact cases each in debug/release/ASAN", + "docs": "passed at e6eb46da346bee9892119fdc414abd7783e66b2c: four catalogues, 582 authored public symbols, 10 README examples; refresh final documentation successor", + "accounting_fix": "Failed Base64 scratch released; work/terminal health retained; caps/weights/version unchanged" + }, + "qualification_policy": "QualificationPolicy.json", + "gate_disposition": { + "R5": "Required evidence satisfied by unchanged runtime/consumer source and materialized Turn003 receipts under amended policy", + "M03": "Satisfied for required physical iPhone; physical iPad deferred", + "M05": "Satisfied on Mac/both simulators/physical iPhone; physical iPad deferred", + "R8": "Local qualification can proceed; maintainer review/protected integration and publication authorization remain separate" + } +} diff --git a/Documentation/Architecture/Release1.1/QA.md b/Documentation/Architecture/Release1.1/QA.md new file mode 100644 index 0000000..2331192 --- /dev/null +++ b/Documentation/Architecture/Release1.1/QA.md @@ -0,0 +1,73 @@ +# Additional 1.1 acceptance and conformance wave + +The Turn003 wave adds 19 Swift Testing behavior functions to the 156-function +Turn002 candidate. Exact discovery is JSON97, Streaming14, Schema41, Regex12 and +Patch11: 175 functions, with all 173 prior parameterized invocations and all 48 +earlier Swift test files retained. No independent policy exclusion is reclassified. + +| Subject | Additional evidence | +| --- | --- | +| Exact transport | Fixed seed `0x534A1100A11CE001`, 192 documents across six root kinds; compact token bytes independently check native projection, Version 1 opening and native/Foundation JSON/property-list transport | +| Numeric projections | 772 integral spellings, independently checked signed/unsigned 8-bit ranges, including equivalent fractional/exponent spellings | +| Keys/index publication | Checked construction, accumulator, parser, edits and reusable scopes; actual member boundaries 7/8/9, 31/32/33, 63/64/65 and 127/128/129; ordered scalar-key oracle | +| Base64 | 259 canonical byte fixtures; 1,152 mutated unused-padding-bit cases; repeated caught corruption, prefix ownership, quota failure and in-loop cancellation | +| Scoped coding | Permanent ceilings after ordinary failure, prior unselected-text admission, caller-error scratch release and mapped Unicode UTF-8 byte coordinates | +| Payload budgets | Six roots, creation/opening work and live-retention inclusive/minus-one matrix; identical preserved bytes | +| Shared state | 16 workers ×20 openings/read scopes over shared immutable inputs; checked Sendable and five-case TSAN selection | +| Schema | Explicitly rooted definitions and negative cases; wide persisted instances/evaluated scalar locations, plan reuse after invalidity, cumulative regex search boundary | +| Patch | Ordered scalar-key edits and real 32→31→32 stages; cumulative quota refusal publishes no partial result | +| Streaming | Every byte split of a wide scalar-key/lexeme document and malformed duplicate; stable second-key location, terminal pulls, cancellation after three accepted sink bytes | + +The first executed wave reproduced a logical-retention accounting defect: shared +Base64 decoding left partial scratch charged after ordinary corruption. A caller +catching that error could accumulate freed-byte reservations and later get a false +resource refusal. Decode now releases only its invocation's failed reservation. +Consumed work and terminal resource/cancellation state remain charged/latched; +successful Data ownership retains its prior accounting. This corrects Version 1 +accounting without changing weights, caps, grammar or the public API. It is not an +allocator-memory leak or an RSS improvement claim. Original failure receipts remain. + +A new projection test initially calibrated its limit using native rounded decoding +then added a larger exactness-proof operation. Its correction measures the same +complete operation on both sides. Compiler harness corrections use an explicit +JSONKey initializer and an actor-owned Sendable sink; no unsafe annotation or +compiler bypass is introduced. A separate review expanded generated field widths +to include actual member boundaries after adding the special keys. + +Use `Scripts/ci.py --lane native`, `--lane safety`, `--lane docs` and the supplemental +`--lane qa` on the supported host. Full native execution retains JSONTestSuite +318 discovered/287 evaluated/31 policy and Schema2337/2300/37, 82 authored vectors, +nine budget vectors, 4,396 malformed replays, 31,824 regex cases/442 patterns and +18,853 Unicode cases/94,265 comparisons. All49 public cap outcomes remain required. +The rooted public consumer retains20 cases and distinct operational errors. + +The public mobile harness additionally checks100 caught malformed Data projections +under32 logical bytes and wide indexed payload/Schema/Patch behavior. Both SDKs, +isolated iPhone/iPad simulators and the physical iPhone execute the updated harness; +disk save/reopen/update uses three independently exited processes and eight frozen +fixtures. This QA wave originally left physical iPad qualification blocked. The +subsequent [maintainer policy](QualificationPolicy.json) accepts iPad simulator +evidence for 1.1.0 and defers physical iPad execution without blocking the release. +Simulator execution does not establish physical-device performance. + +The affected Base64 paths have three serial alternating release-native pairs, +each with3 warmups,9 samples and100 public invocations/sample, finite balanced caps, +matched input hashes and identical work. Across27 samples/side, valid decode +medians range from−1.28% to+4.98%; native payload decode is+1.16% and corruption +recovery−0.09%. No new10% timing trigger is crossed. Recovery's corrected logical +peak is16 rather than300, with final live0 rather than300; work remains506. Keep +raw variance/ranges and process-wide RSS separate from these logical counters. +The earlier wide-index and integrated streaming dispositions remain unchanged. + +Physical iPhone14ProMax/iOS27.2 updated public QA measures9 release samples after3 +warmups: median50.457167ms, process-wide peak25,133,056 bytes, thermal state0 at +both observations. This harness adds work relative to Turn002; the two medians +are not a matched speed comparison. SDK/UIKit/process costs are included. + +Actual receipts, failed-source reproduction, raw samples, source bindings, clean +archive rebuilding and compiled catalogues are materialized in the local Turn003 +return. Hosted Native job artifacts supply remote source/command receipts. This +wave does not merge the protected PR stack, qualify an MCP server, publish1.1, +select SIMD/parallel batches, or claim physical iPad qualification. The later +release-scope decision supersedes the physical iPad blocker while preserving these +historical receipts and outcomes. diff --git a/Documentation/Architecture/Release1.1/QualificationPolicy.json b/Documentation/Architecture/Release1.1/QualificationPolicy.json new file mode 100644 index 0000000..0f43399 --- /dev/null +++ b/Documentation/Architecture/Release1.1/QualificationPolicy.json @@ -0,0 +1,49 @@ +{ + "schema_version": 1, + "target_release": "1.1.0", + "decision_id": "ipad-simulator-acceptance-2026-10-12", + "approved_on": "2026-10-12", + "approval": "Explicit maintainer release-scope decision", + "supersedes": "Physical iPad requirement in the original R5/M03/M05/R8 scope", + "baseline": { + "swift_tools_compiler_minimum": "6.4", + "swift_language_mode": "6", + "macos_sdk_minimum": "27", + "ios_ipados_minimum": "27" + }, + "required_evidence": [ + "macOS native/safety/docs", + "iPhoneOS and iPhoneSimulator SDK builds", + "iPhone simulator public behavior and three-process disk persistence", + "iPad simulator public behavior and three-process disk persistence", + "physical iPhone public behavior, three-process disk persistence and release observations" + ], + "physical_devices": { + "iPhone": { + "required_for_release": true, + "evidence": "Separate physical smoke, disk persistence and release observations" + }, + "iPad": { + "required_for_release": false, + "status": "deferred", + "qualification": "SDK and iPad simulator acceptance for 1.1.0", + "limitation": "Physical iPad behavior, performance, memory and energy are unmeasured; simulator data makes no physical-device claim" + } + }, + "gate_mapping": { + "R5": "Required matrix excludes physical iPad; all other platform/storage requirements remain", + "M03": "Physical iPhone smoke and release observations required; physical iPad deferred and nonblocking", + "M05": "Three-process disk persistence required on macOS, both simulator families and physical iPhone", + "M06": "Distinguish SDK builds, simulator runtime, physical iPhone observations and untested physical iPad", + "R8": "Physical iPad unavailability does not block local release preparation; review/integration and publication authorization remain separate" + }, + "unchanged": [ + "Four public products/private regex", + "Empty root dependencies", + "Apache-2.0/all notices", + "All 49 finite caps/accounting", + "Strict semantics and every existing test/corpus assertion", + "10% performance trigger and dispositions", + "Deferred SIMD/parallel batches and E11/E12" + ] +} diff --git a/Documentation/Architecture/Release1.1/Reading.md b/Documentation/Architecture/Release1.1/Reading.md new file mode 100644 index 0000000..23808e7 --- /dev/null +++ b/Documentation/Architecture/Release1.1/Reading.md @@ -0,0 +1,59 @@ +# Scoped reads and native coding + +The additive public surface is JSON/JSONDocument `read(limits:_:)` and +`readWithReport(limits:_:)`, JSONReadCursor and JSONAccessFailure. Throwing read-only +String/JSONKey/Int brackets preserve typed selection. Required value/string/Boolean/ +number/object/array access never coerces, defaults or manufactures null. Integer +conversion is exact; Double/Decimal require an explicit `.exact` or `.rounded` +choice. Both key overloads provide raw/scalar/model typed presence. + +The supported compiler requires experimental features for the intended returned +nonescapable shape. The public implementation uses the qualified guarded fallback. +A handle owns only its bounded typed path and a synchronous non-Sendable scope. +Scope exit invalidates it and releases root/document/source/coding-state references. +A caller-thread guard runs before accessing mutable state; a cross-thread attempt +fails with configuration/invalidOperation. This introduces no actor isolation or +Sendable escape hatch. Public compiler probes and guarded misuse tests are required. + +One noncopyable core-domain meter admits the full root once. Navigation, equivalent +Pointer path text, overlapping path copies, text extraction, numeric work, source +lookup and native model projection use that ledger. Dead path ownership is refunded +at the next owned boundary; work is never refunded. Paths retain literal scalar keys +and integer indices; missing intermediate parents fail at their first missing hop. +Grouping calls amortizes root admission. It does not selectively parse the input or +make all deep navigation, root admission, edits or arbitrary caller work constant-time. + +Coding options temporarily select native strategies and userInfo while their limits +permanently tighten the scope's ceilings. Prior cumulative usage and already admitted +capacities must fit that restriction; live retention is checked against current live +ownership. No state/meter copy or new decoder facade resets the budget. Numeric +temporary storage is released after primitive projection. Returned values and caller +model computation retain the ordinary caller-work and retention boundaries. + +Resource/cancellation failure is latched at the owned meter/scope. Even a caller +Decodable that catches an owned conversion refusal cannot certify successful scope +or native facade completion. Ordinary access/type/conversion errors can be caught +explicitly. JSONAccessFailure holds a complete bounded typed path and structured +cause, with no rejected value or document. Mapped missing members use their parent +span; replaced/unmapped roots have no invented location. Diagnostic classification +uses the cause and never renders arbitrary caller descriptions. + +JSON `init(encoding:coding:)`, `decode(_:coding:)` and `decodeWithReport(_:coding:)` +delegate to the existing native facades. Natural foreign JSON Codable keeps its +representation and fidelity limitations. Exact persistence uses a separate explicit +Version 1 wrapper. No new kind, dictionary authority, defaulting, dynamic-member +lookup, mutation bracket or projection parser is introduced. + +The new modern-date compatibility case exposed an existing ISO-8601 strategy bug: +the formatter selected only time fields. Both native writer and reader now explicitly +select year/month/day plus fractional UTC time. Baseline epoch-only tests could not +detect losing the calendar date. Record this as a real affected coding qualification +failure and refresh its native/safety/consumer/DocC evidence; do not redefine it as +permitted rounding or silently reduce the assertion. + +Native coding and read scopes explicitly enable accepted-capacity history because +projection/payload limits may tighten after prior admission. Fixed-policy operations +retain terminal health/work accounting but skip history they cannot consume. Work +counters and live ownership supply their own restriction checks; no meter copy or +quota reset is introduced. The final integrated performance review records this +measured correction and retained regressions. diff --git a/Documentation/Architecture/Release1.1/Review.md b/Documentation/Architecture/Release1.1/Review.md new file mode 100644 index 0000000..b08a321 --- /dev/null +++ b/Documentation/Architecture/Release1.1/Review.md @@ -0,0 +1,65 @@ +# Separate candidate review pass + +Reviewer: Codex, the implementing agent, in a separate static/evidence review pass +on 2026-10-12. This is not an independent human review or a GitHub approval. +Reviewed core: a48eb337b70eb075ce37c99dc3d150032f720d00 against main a2733dd. +Documentation successors retain those core bytes. Read the actual source, compiler +receipts, test bodies, SDK/runtime results and raw measurements; no summary-only audit. + +| Finding | Severity | Resolution and evidence | +| --- | --- | --- | +| Native ISO-8601 omitted calendar fields | high | Explicit year/month/day UTC style in both directions; exact modern-date regression plus original coding cases | +| Base64 accepted nonzero unused padding bits | medium | Shared strict decoder rejects AB==/AAB= and noncanonical inner-null spellings | +| Caught owned failure could certify callback completion | high | Terminal meter health and final checkpoints; caught numeric/payload/cancellation/retired-counter overflow cases | +| Simulator bundle had unsuitable Resources layout | high | Direct Draft2020-12 resource root; default-engine signing, integrity loader and both SDK/runtime consumers | +| SDK retained a container beyond lexical scope | qualification | Preserve failed simulator weak-reference proof; three separate exited processes certify true disk reopening on Mac/simulators/physical iPhone | +| Exact CI count labels were mismatched | qualification | Inspect actual bundles; exactly JSON86/Streaming11/Schema38/Regex12/Patch9, 173 parameterized invocations; retain initial failures | +| Initial capacity-history writes regressed broad workloads | performance | Enable history only where later restriction is supported, exclude existing work/live counters; retain initial/final matched samples and every 10% disposition | +| Physical launch gave no inspectable results | qualification | Explicit UIKit entry/scene lifecycle and run-ID-bound result files retrieved from its own app container; normal child exits verified | +| Benchmark absolute output appended to cwd | tooling | Byte-equal relocation/path map; use the documented relative output thereafter, retain original commands | + +Inspect every object publication route, exact collision confirmation, pre-growth +spare/overlap reservations, cancellation within loops and nonthrowing finish. +Review cursor owner checks before mutable access, source-map parent spans, complete +bounded paths, scope invalidation/root release, numeric fidelity and the single +projection ledger. Review payload validated-bytes-only storage, complete grammar/EOF, +balanced intersections, checked expansion, corruption versus operational errors +and absence versus null. Native/foreign callbacks hold no active inout meter borrow. +No blanket unsafe/actor annotations, permissive parser or regex fallback is added. + +Retain 1.0/W00–W12/G01–G12/RC/F/J and R0 history. Native/safety/docs, independent +policies, all49 caps, public consumers and physical/simulator receipts identify +actual commands/source and nonzero counts. Source archives exclude private input, +Git/build trees, signing profiles and device identifiers. At this review's original +scope, physical iPad remained blocked; conditional SIMD/parallel batches stay +deferred, with no unqualified +performance claim. No merge/tag/publication is implied by this review. + +## Additional Turn003 review + +The same implementing agent performed a separate source, fixture, receipt and +test-body review against parent `9e61f158`. This remains distinct from independent +human review or GitHub approval. Native/full safety ran at `5d655f2`; the reviewed +boundary additions ran through the exact 19-case debug/release/ASAN QA lane at +`e6eb46d`. All runtime library bytes are unchanged between those two stages. + +- Medium: shared Base64 failure retained freed scratch in the logical ledger. + Preserve the 400 failed recovery assertions; restore only the failed decoder's + live reservation. Successful ownership, work and terminal health remain intact. + Full native/ASAN, quota/cancellation-prefix tests and a public 32-byte consumer + verify the correction. Three matched benchmark pairs retain identical work. +- Qualification: special keys added to generated widths obscured actual final + index boundaries. Expanded the corpus to actual 7/8/9, 31/32/33, 63/64/65 and + 127/128/129 members; Patch now explicitly visits 32→31→32. All added cases pass + debug/release/ASAN without changing any prior test file or policy exclusion. +- Documentation: the API table still described direct object lookup as linear. + It now describes immutable exact-key metadata. README distinguishes released + 1.0's historical stream regression from the candidate's separate +15.76% pair. + +The final source binding, clean reproduction, actual catalogue outputs and hosted +CI receipts accompany the local Turn003 return. Both SDKs, both simulator families +and the updated physical iPhone public/disk harness pass. This review originally +left physical iPad blocked. The subsequent 2026-10-12 +[maintainer scope decision](QualificationPolicy.json) removes that release blocker +and accepts iPad simulator qualification while retaining untested hardware as a +limitation. Protected integration and publication require separate maintainer action. diff --git a/Documentation/Qualification/README.md b/Documentation/Qualification/README.md index b66be96..3f6de2a 100644 --- a/Documentation/Qualification/README.md +++ b/Documentation/Qualification/README.md @@ -5,11 +5,30 @@ Each receipt identifies the source content, command exits and environment. CI artifacts contain raw logs, consumer outcomes and per-case corpus results. Safety and DocC lanes run separately. Unsupported hosts fail preflight. -The unchanged library has 122 Swift Testing functions across five bundles and -173 parameterized invocations. One public cap test exercises all 49 metrics; +The 1.0 baseline has 122 Swift Testing functions across five bundles and +173 parameterized invocations. The first 1.1 qualification milestone adds seven +authored prototype cases in JSONTests (54 to 61), for 129 functions, with the existing 173 parameterized invocations retained +and seven additional nonparameterized cases. The first production lookup increment adds two further cases (JSONTests 63, total 131 functions). No baseline case or corpus outcome is removed. One public cap test exercises all 49 metrics; the independent runner adds nine budget/report vectors. There are three separate JSON/JSONSchema consumer tests. All discoveries must be nonzero. +The Turn002 integrated 1.1 candidate checked exactly 156 root functions: +JSON 86, JSONStreaming 11, JSONSchema 38, JSONRegexRuntime 12 and JSONPatch 9. +All 173 parameterized invocations remain. The earlier counts above describe their +historical stages, not this source. Three additional public/mobile Swift Testing +functions exercise eight pinned payload fixtures and three independently exited +SwiftData disk phases. The seven-function R0 subset, all 49 cap probes, rooted +20-case consumer and independent policies retain their exact checks. + +The additional QA wave adds 19 behavior functions, for exactly 175: +JSON 97, JSONStreaming 14, JSONSchema 41, JSONRegexRuntime 12 and JSONPatch 11. +All 48 earlier Swift test files and 173 parameterized invocations are retained. +The focused `python3 Scripts/ci.py --lane qa` checks exact counts 11/3/3/2 in +debug, release and Address Sanitizer. It supplements the full native/safety lanes +and does not replace independent qualification or any of the 49 cap probes. +Thread Sanitizer checks five shared-state cases, including concurrent payload +opening and separate read scopes over one immutable root. See [QA](../Architecture/Release1.1/QA.md). + | Independent input | Observed policy | | --- | --- | | JSONTestSuite, pinned in FixtureRegistry.json | 318 discovered: 101 accepted, 186 rejected, 31 strict/finite-policy rejections | @@ -36,3 +55,8 @@ stream completion/backpressure, reports, cancellation and concurrent plan reuse are tested. Native safety includes Address Sanitizer, Thread Sanitizer and strict memory safety compilation. [CI](../CI/README.md) describes executed lanes; [performance](../Performance/README.md) retains the reviewed streaming disposition. + +The [1.1 platform policy](../Architecture/Release1.1/QualificationPolicy.json) +accepts iPad SDK/simulator behavior and three-process disk persistence for this +release. Physical iPad qualification is deferred and nonblocking. Physical iPhone +evidence remains required; no test, corpus policy or resource assertion is removed. diff --git a/Examples/Consumer/Sources/Consumer/READMEExamples.swift b/Examples/Consumer/Sources/Consumer/READMEExamples.swift index 3e18e28..9c78854 100644 --- a/Examples/Consumer/Sources/Consumer/READMEExamples.swift +++ b/Examples/Consumer/Sources/Consumer/READMEExamples.swift @@ -1,6 +1,7 @@ // SPDX-License-Identifier: Apache-2.0 // Compiled README examples; checked by the documentation lane. import JSON +import Foundation import JSONSchema import JSONPatch import JSONStreaming @@ -55,6 +56,21 @@ func runREADMEExamples() throws { let decoded = try JSONParser().decode(Reading.self, from: bytes) precondition(decoded == reading) + let scoped = try value.readWithReport { root in + let exactCount = try root["count"].requiredInteger(as: UInt64.self) + let optionalNote = try root.stringPresence(of: "note") + return (exactCount, optionalNote) + } + precondition(scoped.result.0 == 9_007_199_254_740_993) + precondition(scoped.result.1 == .null) + precondition(scoped.statistics[.workUnits] > 0) + + let payload = try JSONCodablePayload(preserving: value) + let transported = try PropertyListEncoder().encode(payload) + let restored = try PropertyListDecoder().decode(JSONCodablePayload.self, from: transported) + let restoredValue = try restored.value() + precondition(restoredValue == value) + let schema = try JSONSchemaDocument(parser.parse( #"{"type":"object","properties":{"count":{"type":"integer"}},"required":["count"]}"# )) diff --git a/Package.swift b/Package.swift index c83a644..c372695 100644 --- a/Package.swift +++ b/Package.swift @@ -4,7 +4,7 @@ import PackageDescription let package = Package( name: "Swift-JSON", - platforms: [.macOS("27.0")], + platforms: [.macOS("27.0"), .iOS("27.0")], products: [ .library(name: "JSON", targets: ["JSON"]), .library(name: "JSONSchema", targets: ["JSONSchema"]), @@ -14,7 +14,7 @@ let package = Package( targets: [ .target(name: "JSON"), .target(name: "JSONRegexRuntime", dependencies: ["JSON"]), - .target(name: "JSONSchema", dependencies: ["JSON", "JSONRegexRuntime"], resources: [.copy("Resources")]), + .target(name: "JSONSchema", dependencies: ["JSON", "JSONRegexRuntime"], resources: [.copy("Resources/Draft2020-12")]), .target(name: "JSONStreaming", dependencies: ["JSON"]), .target(name: "JSONPatch", dependencies: ["JSON"]), .testTarget(name: "JSONTests", dependencies: ["JSON"], exclude: ["ImplementationPlan.md"]), diff --git a/README.md b/README.md index 4c2f9db..a3a52c0 100644 --- a/README.md +++ b/README.md @@ -161,6 +161,51 @@ by default; explicit ISO-8601 uses bounded UTC text with fractional seconds and four-digit years. Foreign Codable encoders/decoders determine their own numeric and key fidelity. Caller callbacks remain caller-owned work. +## Development toward 1.1 + +The 1.1 feature branches add scoped reads, native value-coding conveniences, +exact Version 1 persistence and a measured private wide-object index. The released +1.0 package remains available through the installation declaration below. +For 1.1, iPadOS acceptance uses the SDK and iPad simulator, including disk persistence; +physical iPad qualification is deferred and nonblocking. Physical iPhone checks +remain required. Simulator results make no physical-device performance claim. +See the [release plan](Documentation/Architecture/Release1.1/Plan.md) and +[qualification policy](Documentation/Architecture/Release1.1/QualificationPolicy.json). + +```swift +let scoped = try value.readWithReport { root in + let exactCount = try root["count"].requiredInteger(as: UInt64.self) + let optionalNote = try root.stringPresence(of: "note") + return (exactCount, optionalNote) +} +precondition(scoped.result.0 == 9_007_199_254_740_993) +precondition(scoped.result.1 == .null) +precondition(scoped.statistics[.workUnits] > 0) +``` + +Throwing String/JSONKey/Int brackets preserve literal scalar keys and typed array +indices. Required access never coerces or defaults. Presence distinguishes absent, +null and a non-null value; native model projection shares the scope's cumulative +meter. Double/Decimal getters require an explicit exact or rounded policy. +Escaped cursors invalidate at scope exit and retain no selected root/source. + +```swift +import Foundation + +let payload = try JSONCodablePayload(preserving: value) +let transported = try PropertyListEncoder().encode(payload) +let restored = try PropertyListDecoder().decode(JSONCodablePayload.self, from: transported) +let restoredValue = try restored.value() +precondition(restoredValue == value) +``` + +The explicit wrapper transports a fixed ASCII envelope with strict Base64 of +complete JSON, preserving exact decimals, scalar sequences, order and admitted +number lexemes. It retains validated bytes only. Automatic coding uses finite +balanced limits; explicit limits can tighten them. Foreign coder work remains +caller work. For SwiftData, use `@Attribute(.codable)` in your application target; +see the [executed disk example](Tools/MobileQualification/README.md). + ## Compile and validate schemas offline ```swift @@ -323,12 +368,16 @@ documentation. Receipts record actual commands, toolchains, source hashes and ca counts. Policy exclusions are reported separately from evaluated successes. See [qualification](Documentation/Qualification/README.md) and [CI](Documentation/CI/README.md). -Performance is measured separately on native release builds. The current reviewed +Performance is measured separately on native release builds. The released 1.0 streaming comparison is approximately **11% slower**, exceeding the unchanged 10% review trigger, with its accepted disposition and measurement limits documented in [performance notes](Documentation/Performance/README.md). Logical retention remains separate from process RSS. Scalar scanning and plan-local immutable regex sharing are the qualified choices. Projection and JSON text-sequence framing remain deferred. +The 1.1 candidate's separate matched stream comparison is **15.76% slower**; its +[integrated review](Documentation/Architecture/Release1.1/IntegratedPerformance.md) +retains the accepted disposition. The [additional QA wave](Documentation/Architecture/Release1.1/QA.md) +records conformance coverage and the Base64 accounting correction. ## Contributing, support and licensing diff --git a/RELEASING.md b/RELEASING.md index 36ec447..283e921 100644 --- a/RELEASING.md +++ b/RELEASING.md @@ -32,3 +32,10 @@ or releases. Published version tags are protected against updates and deletion. Creating a version tag and its GitHub release are deliberate maintainer actions after this review. A successful CI run alone does not authorize them. + +For 1.1.0, apply the maintainer-approved +[qualification policy](Documentation/Architecture/Release1.1/QualificationPolicy.json): +both mobile SDKs and iPhone/iPad simulator behavior/disk phases remain required, +along with physical iPhone checks. Physical iPad qualification is deferred and +does not block this release. Preserve that limitation in release notes; never +present simulator data as physical-device performance or memory evidence. diff --git a/Scripts/ci.py b/Scripts/ci.py index d92c9c2..4022e19 100644 --- a/Scripts/ci.py +++ b/Scripts/ci.py @@ -6,7 +6,8 @@ from preflight import collect ROOT = pathlib.Path(__file__).resolve().parents[1] -SHARED_TESTS = 'independentCodecFacadesAndExactValuesCanBeSharedAcrossTasks|concurrentSchemaEvaluationsHaveExclusiveDynamicRegexAndAnnotationState|sharedRegexProgramHasExclusiveConcurrentSearchState' +SHARED_TESTS = 'independentCodecFacadesAndExactValuesCanBeSharedAcrossTasks|concurrentSchemaEvaluationsHaveExclusiveDynamicRegexAndAnnotationState|sharedRegexProgramHasExclusiveConcurrentSearchState|immutableIndexedObjectCanBeSharedAcrossTasks|qaSharedPayloadAndRootHaveIndependentConcurrentReadScopes' +R0_TESTS = 'readerProbe|indexProbes|duplicateRecognitionPrecedesValueAndClosingSyntax' def snapshot(): return [{'path': p.relative_to(ROOT).as_posix(), 'sha256': hashlib.sha256(p.read_bytes()).hexdigest()} for p in files()] @@ -14,6 +15,12 @@ def snapshot(): def commands(lane, output): # The qualification executable resolves output paths from its repository root. output = output.relative_to(ROOT) + if lane == 'qa': + return [ + ['swift', 'test', '--filter', 'qa'], + ['swift', 'test', '-c', 'release', '--filter', 'qa'], + ['swift', 'test', '-c', 'release', '--sanitize', 'address', '--scratch-path', '.build/ci-address', '--filter', 'qa'], + ] if lane == 'native': return [ ['python3', '-B', 'Scripts/public_check.py'], @@ -21,7 +28,11 @@ def commands(lane, output): ['swift', 'package', 'show-dependencies', '--format', 'json'], ['swift', 'build', '-c', 'release'], ['swift', 'test'], ['swift', 'test', '-c', 'release'], + ['python3', '-B', 'Scripts/qualify_release_1_1.py', '--mode', 'ownership', '--output', str(output / 'ownership.json')], + ['python3', '-B', 'Scripts/qualify_release_1_1.py', '--mode', 'reader', '--output', str(output / 'reader.json')], ['swift', 'run', '-c', 'release', '--package-path', 'Examples/Consumer', 'Consumer'], + ['swift', 'test', '--package-path', 'Tools/MobileQualification'], + ['swift', 'run', '-c', 'release', '--package-path', 'Tools/MobileQualification', 'MobileQualification'], ['swift', 'test', '--package-path', 'Tools/MCPReadiness'], ['swift', 'run', '-c', 'release', '--package-path', 'Tools/MCPReadiness', 'MCPReadiness', str(output / 'consumer.json')], ['swift', 'test', '--package-path', 'Tools/Qualification'], @@ -33,6 +44,12 @@ def commands(lane, output): ['swift', 'test', '-c', 'release', '--sanitize', 'thread', '--scratch-path', '.build/ci-thread', '--filter', SHARED_TESTS], ['swift', 'build', '--scratch-path', '.build/ci-strict', '-Xswiftc', '-strict-memory-safety', '-Xswiftc', '-warnings-as-errors'], ] + if lane == 'r0': + return [ + ['swift', 'test', '--filter', R0_TESTS], + ['python3', '-B', 'Scripts/qualify_release_1_1.py', '--mode', 'ownership', '--output', str(output / 'ownership.json')], + ['python3', '-B', 'Scripts/qualify_release_1_1.py', '--mode', 'sdk', '--output', str(output / 'sdk.json')], + ] return [['python3', '-B', 'Scripts/build_documentation.py']] def materialized_outcomes(output): @@ -60,7 +77,7 @@ def check_result(command, code, log): counts = [int(n) for n in re.findall(r'Test run with (\d+) tests? in[^\n]*passed', log)] passed = code == 0 if command[:2] == ['swift', 'test']: - expected = [3] if 'Tools/MCPReadiness' in command else [1] if 'Tools/Qualification' in command else [1, 1, 1] if 'thread' in command else [8, 11, 12, 37, 54] + expected = [2, 3, 3, 11] if 'qa' in command else [7] if R0_TESTS in command else [3] if 'Tools/MCPReadiness' in command or 'Tools/MobileQualification' in command else [1] if 'Tools/Qualification' in command else [1, 1, 3] if 'thread' in command else [11, 12, 14, 41, 97] passed = passed and sorted(counts) == expected if 'Tools/Qualification' in command: probes = re.findall(r'^CAP_PROBE (\w+) (\S+)$', log, re.MULTILINE) @@ -69,14 +86,16 @@ def check_result(command, code, log): passed = passed and len(probes) == 49 and dict(probes) == expected_probes if command == ['swift', 'package', 'dump-package'] and passed: d = json.loads(log) - passed = sorted(p['name'] for p in d['products']) == ['JSON', 'JSONPatch', 'JSONSchema', 'JSONStreaming'] and not d['dependencies'] and d['swiftLanguageVersions'] == ['6'] and d['toolsVersion']['_version'] == '6.4.0' and d['platforms'] == [{'options': [], 'platformName': 'macos', 'version': '27.0'}] + passed = sorted(p['name'] for p in d['products']) == ['JSON', 'JSONPatch', 'JSONSchema', 'JSONStreaming'] and not d['dependencies'] and d['swiftLanguageVersions'] == ['6'] and d['toolsVersion']['_version'] == '6.4.0' and d['platforms'] == [{'options': [], 'platformName': 'macos', 'version': '27.0'}, {'options': [], 'platformName': 'ios', 'version': '27.0'}] if command == ['swift', 'package', 'show-dependencies', '--format', 'json'] and passed: passed = json.loads(log)['dependencies'] == [] + if command[:2] == ['swift', 'run'] and 'Tools/MobileQualification' in command: + passed = passed and all(line in log.splitlines() for line in ['PUBLIC_BEHAVIOR passed fixtures=8', 'STREAM_CANCELLATION passed', 'DISK_PERSISTENCE passed containers=3 rows=8', 'MOBILE_QUALIFICATION passed cases=3']) return passed, counts def main(): parser = argparse.ArgumentParser(description=__doc__) - parser.add_argument('--lane', choices=['native', 'safety', 'docs'], required=True) + parser.add_argument('--lane', choices=['native', 'safety', 'docs', 'r0', 'qa'], required=True) parser.add_argument('--output') args = parser.parse_args() output = ROOT / (args.output or '.build/ci/' + args.lane) diff --git a/Scripts/qualify_mobile_runtime.py b/Scripts/qualify_mobile_runtime.py new file mode 100644 index 0000000..ba6c005 --- /dev/null +++ b/Scripts/qualify_mobile_runtime.py @@ -0,0 +1,90 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Build both mobile SDKs and run isolated iPhone/iPad simulator consumers.""" +import argparse, datetime, hashlib, json, pathlib, subprocess, sys +from archive_candidate import files +from preflight import collect +ROOT = pathlib.Path(__file__).resolve().parents[1] +EXPECTED = ['PUBLIC_BEHAVIOR passed fixtures=8', 'STREAM_CANCELLATION passed', + 'DISK_PERSISTENCE passed containers=3 rows=8', 'MOBILE_QUALIFICATION passed cases=3'] + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--output', default='.build/ci/mobile-runtime/receipt.json') + args = parser.parse_args() + output = ROOT / args.output + output.parent.mkdir(parents=True, exist_ok=True) + policy_path = ROOT / 'Documentation/Architecture/Release1.1/QualificationPolicy.json' + policy = json.loads(policy_path.read_text()) + source = [{'path': p.relative_to(ROOT).as_posix(), 'sha256': hashlib.sha256(p.read_bytes()).hexdigest()} for p in files()] + receipt = {'schema_version': 1, 'status': 'running', 'environment': collect(), + 'git_head': subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=ROOT, text=True).strip(), + 'source_inputs': source, 'commands': [], 'simulators': [], + 'qualification_policy': {'path': policy_path.relative_to(ROOT).as_posix(), + 'sha256': hashlib.sha256(policy_path.read_bytes()).hexdigest(), + 'decision_id': policy['decision_id']}, + 'physical_devices': policy['physical_devices'], + 'started_utc': datetime.datetime.now(datetime.timezone.utc).isoformat()} + def save(): output.write_text(json.dumps(receipt, indent=2) + '\n') + def run(name, command): + print('Running ' + name, flush=True) + result = subprocess.run(command, cwd=ROOT, capture_output=True, text=True, timeout=900) + path = output.parent / (name + '.log') + path.write_text((result.stdout + result.stderr).replace(str(ROOT), '.')) + receipt['commands'].append({'name': name, 'command': [x.replace(str(ROOT), '.') for x in command], + 'exit_code': result.returncode, 'log': path.relative_to(ROOT).as_posix(), + 'log_sha256': hashlib.sha256(path.read_bytes()).hexdigest()}) + save() + if result.returncode: raise RuntimeError(name + ' failed') + return result.stdout + save() + try: + if receipt['environment']['status'] != 'passed': raise FileNotFoundError('Supported native host required') + for sdk, triple, scratch in [('iphoneos', 'arm64-apple-ios27.0', '.build/mobile-runtime-device'), + ('iphonesimulator', 'arm64-apple-ios27.0-simulator', '.build/mobile-runtime-sim')]: + sdkpath = run(sdk + '-sdk', ['xcrun', '--sdk', sdk, '--show-sdk-path']).strip() + run(sdk + '-build', ['swift', 'build', '-c', 'release', '--package-path', 'Tools/MobileQualification', + '--scratch-path', scratch, '--triple', triple, '--sdk', sdkpath]) + # Resolve the output actually produced by the default Swift Build engine. + candidates = [p for p in (ROOT / '.build/mobile-runtime-sim/out/Products').glob('Release*/MobileQualification') if p.is_file()] + if len(candidates) != 1: raise RuntimeError('Expected one simulator executable') + binary = candidates[0] + receipt['binary_sha256'] = hashlib.sha256(binary.read_bytes()).hexdigest() + inventory = json.loads(run('available-simulators', ['xcrun', 'simctl', 'list', 'devices', 'available', '--json']))['devices'] + runtime = 'com.apple.CoreSimulator.SimRuntime.iOS-27-0' + devices = inventory.get(runtime, []) + for family in ['iPhone', 'iPad']: + model = next((d for d in devices if d['name'].startswith(family) and d['isAvailable']), None) + if model is None: raise FileNotFoundError('No supported ' + family + ' simulator model/runtime') + device = run(family + '-create', ['xcrun', 'simctl', 'create', 'Swift-JSON-1.1-' + family, + model['deviceTypeIdentifier'], runtime]).strip() + try: + run(family + '-boot', ['xcrun', 'simctl', 'boot', device]) + run(family + '-bootstatus', ['xcrun', 'simctl', 'bootstatus', device, '-b']) + result = run(family + '-consumer', ['xcrun', 'simctl', 'spawn', device, str(binary)]) + if not all(line in result.splitlines() for line in [EXPECTED[0], EXPECTED[1], 'MOBILE_QUALIFICATION passed cases=2']): raise RuntimeError('Missing exact runtime outcomes') + disk = output.parent / ('disk-' + family) + disk.mkdir(parents=True, exist_ok=True) + url = disk / 'snapshots.store' + for phase in ['save', 'reopen-update', 'verify-update']: + result = run(family + '-disk-' + phase, ['xcrun', 'simctl', 'spawn', device, str(binary), '--disk-phase', phase, str(url)]) + if 'DISK_PHASE passed phase=' + phase + ' rows=8' not in result.splitlines(): + raise RuntimeError('Missing disk phase outcome') + if not url.is_file(): raise RuntimeError('No actual disk store') + receipt['simulators'].append({'family': family, 'model': model['name'], 'runtime': runtime, + 'status': 'passed', 'executed_cases': 3, 'fixtures': 8, 'disk_containers': 3}) + save() + finally: + run(family + '-delete', ['xcrun', 'simctl', 'delete', device]) + receipt['status'] = 'passed' + except FileNotFoundError as error: + receipt['status'] = 'blocked'; receipt['error'] = str(error) + except (OSError, RuntimeError, ValueError, subprocess.TimeoutExpired) as error: + receipt['status'] = 'failed'; receipt['error'] = str(error) + receipt['source_inputs_unchanged'] = source == [{'path': p.relative_to(ROOT).as_posix(), 'sha256': hashlib.sha256(p.read_bytes()).hexdigest()} for p in files()] + if not receipt['source_inputs_unchanged']: receipt['status'] = 'failed' + receipt['finished_utc'] = datetime.datetime.now(datetime.timezone.utc).isoformat() + save(); print(json.dumps({k:v for k,v in receipt.items() if k not in ['source_inputs', 'environment']}, indent=2)) + return 0 if receipt['status'] == 'passed' else 2 if receipt['status'] == 'blocked' else 1 + +if __name__ == '__main__': sys.exit(main()) diff --git a/Scripts/qualify_release_1_1.py b/Scripts/qualify_release_1_1.py new file mode 100644 index 0000000..2022260 --- /dev/null +++ b/Scripts/qualify_release_1_1.py @@ -0,0 +1,137 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Executable R0 compiler/SDK probes; prototypes are not shipping JSON APIs.""" +import argparse, datetime, hashlib, json, math, os, pathlib, subprocess, sys +from archive_candidate import files +from preflight import collect + +ROOT = pathlib.Path(__file__).resolve().parents[1] +PROBES = ROOT / 'Tools/Release1_1Probes' + +def verify_index_samples(samples): + """Require the complete measured matrix, or an explicit finite-resource refusal.""" + algorithms = {'scalar', 'fingerprint-linear', 'array-AVL', 'controlled-hash'} + widths = {0,1,4,8,16,32,64,256,1024,10000} + shapes = {'ascii', 'prefix', 'unicode'} + expected = {(a,w,s) for a in algorithms for w in widths for s in shapes} + cells, digests = {}, {} + for sample in samples: + key = (sample['algorithm'], sample['width'], sample['shape']) + if key not in expected: raise ValueError('Unexpected index corpus cell') + if not math.isfinite(sample['seconds']) or sample['seconds'] < 0: raise ValueError('Invalid timing') + if sample['status'] not in {'passed', 'refused'}: raise ValueError('Invalid sample status') + digest_key = key[1:] + digest = sample['inputSHA256'] + if digest_key in digests and digests[digest_key] != digest: raise ValueError('Unmatched algorithm inputs') + digests[digest_key] = digest + cells.setdefault(key, []).append(sample) + if set(cells) != expected: raise ValueError('Incomplete index corpus matrix') + for rows in cells.values(): + identities = [(r['operation'],r['sample']) for r in rows] + if len(identities) != len(set(identities)): raise ValueError('Duplicate measured sample') + refusals = [r for r in rows if r['status'] == 'refused'] + if refusals: + if len(refusals) != 1 or refusals[0]['operation'] != 'construction' or not refusals[0].get('failure'): + raise ValueError('Unidentified resource refusal') + if any(r['sample'] >= refusals[0]['sample'] for r in rows if r['status'] == 'passed'): + raise ValueError('Success after a refused construction') + elif set(identities) != {(op,n) for op in ['construction','12-lookups'] for n in range(9)}: + raise ValueError('Missing nine construction/lookup samples') + return len(cells), sum(r['status'] == 'refused' for r in samples) + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--mode', choices=['ownership', 'reader', 'sdk', 'mobile', 'index'], required=True) + parser.add_argument('--output', default='.build/ci/r0/probe-receipt.json') + args = parser.parse_args() + output = ROOT / args.output + directory = output.parent / args.mode + directory.mkdir(parents=True, exist_ok=True) + inputs = [{'path': p.relative_to(ROOT).as_posix(), 'sha256': hashlib.sha256(p.read_bytes()).hexdigest()} for p in files()] + receipt = {'schema_version': 1, 'mode': args.mode, 'status': 'running', 'environment': collect(), + 'git_head': subprocess.check_output(['git', 'rev-parse', 'HEAD'], cwd=ROOT, text=True).strip(), + 'source_inputs': inputs, 'commands': [], 'runtime_qualification': 'not_run', + 'started_utc': datetime.datetime.now(datetime.timezone.utc).isoformat()} + def save(): + output.parent.mkdir(parents=True, exist_ok=True) + output.write_text(json.dumps(receipt, indent=2) + '\n') + def run(name, command, expected=0, diagnostic=None): + result = subprocess.run(command, cwd=ROOT, capture_output=True, text=True, timeout=900) + log = directory / (name + '.log') + log.write_text((result.stdout + result.stderr).replace(str(ROOT), '.')) + passed = result.returncode == expected and (diagnostic is None or diagnostic in log.read_text()) + receipt['commands'].append({'name': name, 'command': [str(c).replace(str(ROOT), '.') for c in command], + 'exit_code': result.returncode, 'expected_exit': expected, 'expected_diagnostic': diagnostic, + 'status': 'passed' if passed else 'failed', 'log': log.relative_to(ROOT).as_posix(), + 'log_sha256': hashlib.sha256(log.read_bytes()).hexdigest()}) + save() + if not passed: raise RuntimeError('Probe failed: ' + name) + return result.stdout + save() + try: + if receipt['environment']['status'] != 'passed': raise FileNotFoundError('Supported native baseline required') + # SwiftPM 6.4's default Swift Build engine publishes native modules/libraries here. + products = ROOT / '.build/out/Products/Debug' + swift = ['xcrun', 'swiftc', '-swift-version', '6'] + if args.mode == 'ownership': + if not (products / 'libJSON.a').is_file(): raise FileNotFoundError('Run swift test first to build the real testable JSON module') + module = directory / 'ReaderProbe.swiftmodule' + run('reader-module', swift + ['-parse-as-library', '-emit-module', '-emit-library', '-module-name', 'ReaderProbe', + '-I', str(products), '-L', str(products), '-lJSON', str(ROOT / 'Tests/JSONTests/Release1_1/ReaderPrototype.swift'), + '-o', str(directory / 'libReaderProbe.dylib'), '-emit-module-path', str(module)]) + imports = ['-I', str(directory), '-I', str(products)] + binary = directory / 'reader-consumer' + run('consumer-compile', swift + imports + ['-L', str(directory), '-lReaderProbe', '-Xlinker', '-rpath', '-Xlinker', str(directory), + str(PROBES / 'Consumer.swift'), '-o', str(binary)]) + result = run('consumer-run', [str(binary)]) + if 'READER_CONSUMER passed' not in result: raise RuntimeError('Missing executed public consumer assertion') + run('sendable-negative', swift + imports + ['-typecheck', str(PROBES / 'TaskEscape.swift')], 1, "capture of 'cursor' with non-Sendable type") + # Region-based transfer checking happens during SIL ownership analysis, not typecheck alone. + run('task-transfer-negative', swift + imports + ['-c', str(PROBES / 'TaskTransfer.swift'), '-o', str(directory / 'illegal.o')], 1, + "passing closure as a 'sending' parameter risks causing data races") + run('nonescapable-unannotated', swift + ['-typecheck', str(PROBES / 'Nonescapable.swift')], 1, + 'an initializer cannot return a ~Escapable result') + run('lifetime-feature-boundary', swift + ['-typecheck', str(PROBES / 'LifetimeAnnotated.swift')], 1, + "'@_lifetime' attribute is only valid when experimental feature Lifetimes is enabled") + receipt['positive_commands'] = 3; receipt['negative_compiler_cases'] = 4 + elif args.mode == 'reader': + binary = directory / 'public-reader-consumer' + imports = ['-I', str(products)] + run('public-consumer-compile', swift + imports + ['-L', str(products), '-lJSON', str(PROBES / 'PublicReaderConsumer.swift'), '-o', str(binary)]) + result = run('public-consumer-run', [str(binary)]) + if 'PUBLIC_READER_CONSUMER passed' not in result: raise RuntimeError('Missing public reader outcomes') + run('public-sendable-negative', swift + imports + ['-typecheck', str(PROBES / 'PublicTaskEscape.swift')], 1, "capture of 'cursor' with non-Sendable type") + run('public-task-transfer-negative', swift + imports + ['-c', str(PROBES / 'PublicTaskTransfer.swift'), '-o', str(directory / 'illegal-public.o')], 1, + "passing closure as a 'sending' parameter risks causing data races") + receipt['positive_commands'] = 2; receipt['negative_compiler_cases'] = 2 + elif args.mode == 'sdk': + for sdk, target in [('macosx', 'arm64-apple-macosx27.0'), ('iphoneos', 'arm64-apple-ios27.0'), ('iphonesimulator', 'arm64-apple-ios27.0-simulator')]: + path = run(sdk + '-path', ['xcrun', '--sdk', sdk, '--show-sdk-path']).strip() + run(sdk + '-codable-shape', swift + ['-sdk', path, '-target', target, '-typecheck', str(PROBES / 'SwiftDataShape.swift')]) + receipt['sdk_shapes_compiled'] = 3 + elif args.mode == 'mobile': + for sdk, target in [('iphoneos', 'arm64-apple-ios27.0'), ('iphonesimulator', 'arm64-apple-ios27.0-simulator')]: + path = run(sdk + '-path', ['xcrun', '--sdk', sdk, '--show-sdk-path']).strip() + run(sdk + '-consumer', ['swift', 'build', '-c', 'release', '--package-path', str(PROBES / 'MobileConsumer'), + '--scratch-path', str(directory / sdk), '--triple', target, '--sdk', path]) + receipt['sdk_consumers_built'] = 2 + else: + products = ROOT / '.build/out/Products/Release' + binary = directory / 'index-benchmark' + run('compile', swift + ['-O', '-parse-as-library', '-I', str(products), '-L', str(products), '-lJSON', + str(ROOT / 'Tests/JSONTests/Release1_1/IndexPrototype.swift'), str(PROBES / 'IndexBenchmark.swift'), '-o', str(binary)]) + run('measure', [str(binary), str(directory / 'samples.json')]) + samples = json.loads((directory / 'samples.json').read_text()) + receipt['corpus_cells'], receipt['resource_refusals'] = verify_index_samples(samples) + receipt['samples'] = len(samples) + receipt['samples_sha256'] = hashlib.sha256((directory / 'samples.json').read_bytes()).hexdigest() + receipt['status'] = 'passed' + except FileNotFoundError as error: + receipt['status'] = 'blocked'; receipt['error'] = str(error) + except (OSError, ValueError, RuntimeError, subprocess.TimeoutExpired) as error: + receipt['status'] = 'failed'; receipt['error'] = str(error) + receipt['finished_utc'] = datetime.datetime.now(datetime.timezone.utc).isoformat() + save(); print(json.dumps({k:v for k,v in receipt.items() if k not in ['source_inputs','environment']}, indent=2)) + return 0 if receipt['status'] == 'passed' else 2 if receipt['status'] == 'blocked' else 1 + +if __name__ == '__main__': sys.exit(main()) diff --git a/Sources/JSON/Codec/JSONTokenMachine.swift b/Sources/JSON/Codec/JSONTokenMachine.swift index 6836a1f..4702b13 100644 --- a/Sources/JSON/Codec/JSONTokenMachine.swift +++ b/Sources/JSON/Codec/JSONTokenMachine.swift @@ -11,7 +11,7 @@ package struct JSONMachineEvent: Sendable { package struct JSONTokenMachine { package enum State: Sendable { case ready, reading, complete, failed, finished } private enum Expectation { case objectFirstKey, objectKey, colon, objectValue, objectEnd, arrayFirstValue, arrayValue, arrayEnd } - private struct Frame { var expectation: Expectation; var count: UInt64 = 0; var keys: [String] = [] } + private struct Frame { var expectation: Expectation; var count: UInt64 = 0; var keys = JSONExactKeyIndex() } private var lexer = JSONLexer() private let topLevelArrayStreaming: Bool package init(topLevelArrayStreaming: Bool = false, offset: UInt64 = 0) { @@ -76,17 +76,17 @@ package struct JSONTokenMachine { } frames.removeLast(); try meter.release(JSONLogicalWeights.frame) // Key payload remains in a materialized value or in the caller's event; index slots are grammar-owned. - for _ in frame.keys { try meter.work(); try meter.release(JSONLogicalWeights.indexSlot) } + try meter.release(frame.keys.retainedBytes) if frames.isEmpty { rootComplete = true; state = .complete } return .init(kind: object ? .endObject : .endArray, start: token.start, end: token.end) case .string(let text) where expectingKey: let index = frames.count - 1 - for previous in frames[index].keys { - if try JSONComparison.textEquals(previous, text, meter: &meter) { throw failure(token, .duplicateKey) } + let key = try JSONKey(text, meter: &meter) + guard try frames[index].keys.insert(key, position: Int(frames[index].count), meter: &meter) else { + throw failure(token, .duplicateKey) } frames[index].count = try meter.adding(.objectMembers, frames[index].count, 1) - try meter.retain(JSONLogicalWeights.indexSlot) - frames[index].keys.append(text); frames[index].expectation = .colon + frames[index].expectation = .colon return .init(kind: .key(text), start: token.start, end: token.end) case .leftObject, .leftArray: try beginValue(token, meter: &meter) @@ -138,7 +138,7 @@ package struct JSONTokenMachine { } package struct JSONValueAssembler { - private struct Frame { var object: Bool; var key: String?; var members: [JSONMember] = []; var values: [JSON] = [] } + private struct Frame { var object: Bool; var key: JSONKey?; var members: [JSONMember] = []; var values: [JSON] = []; var index = JSONExactKeyIndex() } private var stack: [Frame] = [] package init() {} package mutating func consume(_ event: JSONMachineEvent, meter: inout JSONWorkMeter) throws -> JSON? { @@ -149,10 +149,14 @@ package struct JSONValueAssembler { let object: Bool if case .startObject = event.kind { object = true } else { object = false } stack.append(Frame(object: object)); return nil - case .key(let key): stack[stack.count - 1].key = key; return nil + case .key(let key): + let last = stack.count - 1 + let key = try JSONKey(key, meter: &meter) + guard try stack[last].index.insert(key, position: stack[last].members.count, meter: &meter) else { throw JSONFailure(.syntax, code: .duplicateKey) } + stack[last].key = key; return nil case .endObject, .endArray: let frame = stack.removeLast(); try meter.release(JSONLogicalWeights.frame) - let value: JSON = frame.object ? .object(JSONObject(admitted: frame.members)) : .array(frame.values) + let value: JSON = frame.object ? .object(JSONObject(admitted: frame.members, index: frame.index)) : .array(frame.values) return try add(value, meter: &meter) case .scalar(let value): try meter.retain(JSONLogicalWeights.node) @@ -164,8 +168,8 @@ package struct JSONValueAssembler { let index = stack.count - 1 if stack[index].object { guard let key = stack[index].key else { throw JSONFailure(.syntax, code: .invalidOperation) } - try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot) - stack[index].members.append(JSONMember(try JSONKey(key, meter: &meter), value: value)); stack[index].key = nil + try meter.retain(JSONLogicalWeights.member) + stack[index].members.append(JSONMember(key, value: value)); stack[index].key = nil } else { try meter.retain(JSONLogicalWeights.arraySlot); stack[index].values.append(value) } return nil } diff --git a/Sources/JSON/Coding/JSONBase64.swift b/Sources/JSON/Coding/JSONBase64.swift index a0e5e21..63922b3 100644 --- a/Sources/JSON/Coding/JSONBase64.swift +++ b/Sources/JSON/Coding/JSONBase64.swift @@ -30,28 +30,36 @@ package enum JSONBase64 { return (text,count) } package static func decode(_ string: String,meter: inout JSONWorkMeter) throws -> Data { - var iterator = string.utf8.makeIterator(), bytes: [UInt8] = [] - func malformed() -> JSONFailure { .init(.syntax,code: .invalidSyntax) } - func emit(_ byte: UInt8,meter: inout JSONWorkMeter) throws { - try meter.work(); try meter.retain(1); bytes.append(byte) - } - while let a = iterator.next() { - try meter.work() - guard let b = iterator.next(),let c = iterator.next(),let d = iterator.next(),let x = value(a),let y = value(b) else { throw malformed() } - try emit(x << 2 | y >> 4,meter: &meter) - if c == 61 { - guard d == 61, iterator.next() == nil else { throw malformed() }; break + let before = meter.used(.retainedBytes) + do { + var iterator = string.utf8.makeIterator(), bytes: [UInt8] = [] + func malformed() -> JSONFailure { .init(.syntax,code: .invalidSyntax) } + func emit(_ byte: UInt8,meter: inout JSONWorkMeter) throws { + try meter.work(); try meter.retain(1); bytes.append(byte) } - guard let z = value(c) else { throw malformed() } - try emit(y << 4 | z >> 2,meter: &meter) - if d == 61 { guard iterator.next() == nil else { throw malformed() }; break } - guard let w = value(d) else { throw malformed() } - try emit(z << 6 | w,meter: &meter) + while let a = iterator.next() { + try meter.work() + guard let b = iterator.next(),let c = iterator.next(),let d = iterator.next(),let x = value(a),let y = value(b) else { throw malformed() } + try emit(x << 2 | y >> 4,meter: &meter) + if c == 61 { + guard d == 61, y & 15 == 0, iterator.next() == nil else { throw malformed() }; break + } + guard let z = value(c) else { throw malformed() } + try emit(y << 4 | z >> 2,meter: &meter) + if d == 61 { guard z & 3 == 0, iterator.next() == nil else { throw malformed() }; break } + guard let w = value(d) else { throw malformed() } + try emit(z << 6 | w,meter: &meter) + } + for _ in bytes { try meter.work() } + try meter.retain(UInt64(bytes.count)); try meter.checkpoint() + let result = Data(bytes) + try meter.checkpoint(); try meter.release(UInt64(bytes.count)) + return result + } catch { + // No result owns failed scratch. Refund live bytes, never consumed + // work or terminal health, so ordinary corruption can be handled. + try meter.release(meter.used(.retainedBytes) - before) + throw error } - for _ in bytes { try meter.work() } - try meter.retain(UInt64(bytes.count)); try meter.checkpoint() - let result = Data(bytes) - try meter.checkpoint(); try meter.release(UInt64(bytes.count)) - return result } } diff --git a/Sources/JSON/Coding/JSONCodablePayload.swift b/Sources/JSON/Coding/JSONCodablePayload.swift new file mode 100644 index 0000000..476a94b --- /dev/null +++ b/Sources/JSON/Coding/JSONCodablePayload.swift @@ -0,0 +1,158 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation + +/// Exact JSON value persistence through a fixed, versioned Codable envelope. +/// +/// Version 1 carries `format: "swift-json-value"`, integer `version: 1` and +/// `utf8Base64`: strict standard Base64 of one complete strict UTF-8 document. +/// The inner writer preserves member order, scalar identity and admitted number +/// lexemes. Whitespace, escape spelling and source documents are not persisted. +/// Only validated bytes are stored; opening parses afresh without a hidden DOM. +/// Automatic coding uses finite balanced limits. Foreign coder work is caller +/// work; nesting in the native coder shares its cumulative meter. +public struct JSONCodablePayload: Sendable, Codable { + private let utf8: Data + private enum Keys: String, CodingKey { case format, version, utf8Base64 } + private init(validated bytes: Data) { utf8 = bytes } + + /// Writes an exact value with Version 1's compact, insertion-order policy. + /// - Parameters: + /// - value: The admitted JSON value, including a present JSON null. + /// - limits: Finite ceilings intersected with the balanced persistence profile. + /// - Throws: Resource/configuration failure or cancellation; no partial payload. + public init(preserving value: JSON, limits: JSONLimits = .balanced) throws { + self = try Self.preservingWithReport(value, limits: limits).result + } + /// Creates a payload and reports owned writing, copies and prospective envelope checks. + /// - Parameters: + /// - value: The exact JSON value to preserve. + /// - limits: Tightening ceilings; a larger profile cannot widen automatic persistence. + /// - Throws: Resource/configuration failure or cancellation. + public static func preservingWithReport(_ value: JSON, limits: JSONLimits = .balanced) throws -> JSONOperationReport { + var meter = try JSONWorkMeter(limits: JSONLimits.balanced.intersection(limits), domain: .coding) + let bytes = try JSONWriter(options: .init(numberStyle: .preserved)).write(value, meter: &meter) + try envelopeCapacity(UInt64(bytes.count), meter: &meter) + for _ in bytes { try meter.work(); try meter.retain(1) } + let data = Data(bytes) + try meter.release(UInt64(bytes.count)); try meter.checkpoint() + return .init(result: Self(validated: data), statistics: meter.statistics()) + } + /// Opens a validated payload through a new complete strict parse operation. + /// - Parameter limits: Finite ceilings intersected with the balanced profile. + /// - Throws: Resource/configuration failure or cancellation. + public func value(limits: JSONLimits = .balanced) throws -> JSON { try valueWithReport(limits: limits).result } + /// Parses afresh and reports owned stored-byte admission, copying and parsing. + /// - Parameter limits: Tightening ceilings; stored data never selects limits. + /// - Throws: Resource/configuration failure or cancellation. + public func valueWithReport(limits: JSONLimits = .balanced) throws -> JSONOperationReport { + var meter = try JSONWorkMeter(limits: JSONLimits.balanced.intersection(limits), domain: .parsing) + try Self.admit(utf8, meter: &meter) + let value = try Self.parse(utf8, meter: &meter) + try meter.release(UInt64(utf8.count)); try meter.checkpoint() + return .init(result: value, statistics: meter.statistics()) + } + private static func admit(_ bytes: Data, meter: inout JSONWorkMeter) throws { + try meter.require(.inputBytes, UInt64(bytes.count)) + for _ in bytes { try meter.work(); try meter.retain(1) } + } + private static func parse(_ bytes: Data, meter: inout JSONWorkMeter) throws -> JSON { + var input: [UInt8] = [] + for byte in bytes { try meter.work(); try meter.retain(1); input.append(byte) } + let result = try JSONParser().parse(input, meter: &meter) + try meter.release(UInt64(input.count)); return result + } + private static func envelopeCapacity(_ length: UInt64, meter: inout JSONWorkMeter) throws { + try meter.require(.inputBytes, length) + let rounded = try meter.adding(.outputBytes, length, 2) / 3 + let (base64, overflow) = rounded.multipliedReportingOverflow(by: 4) + if overflow { _ = try meter.adding(.outputBytes, UInt64.max, 1) } + try meter.require(.stringBytes, base64) + // Fixed primitive fields in their shortest compact JSON spelling. + let overhead = UInt64(#"{"format":"swift-json-value","version":1,"utf8Base64":""}"#.utf8.count) + _ = try meter.adding(.outputBytes, base64, overhead) + let text = try meter.adding(.aggregateTextBytes, base64, 39) + _ = try meter.adding(.aggregateTextBytes, meter.used(.aggregateTextBytes), text) + } + private func encoded(meter: inout JSONWorkMeter) throws -> (String, UInt64) { + let before = meter.used(.retainedBytes) + do { + try Self.admit(utf8, meter: &meter) + try Self.envelopeCapacity(UInt64(utf8.count), meter: &meter) + let (text, count) = try JSONBase64.encode(utf8, meter: &meter) + try meter.charge(.aggregateTextBytes, count) + try meter.release(UInt64(utf8.count)) + return (text, count) + } catch { try meter.release(meter.used(.retainedBytes) - before); throw error } + } + /// Encodes Version 1 primitive fields; it never invokes foreign numeric/key coding for the inner JSON. + /// - Parameter encoder: The enclosing coder, whose callbacks are outside owned work caps. + /// - Throws: Coder errors, resource/configuration failure or cancellation. + public func encode(to encoder: any Encoder) throws { + let native = encoder as? JSONNativeEncoder + if let native { + try native.state.meter.restrict(to: .balanced) + let (text, reservation) = try encoded(meter: &native.state.meter) + do { try encodeFields(text, to: encoder); try native.state.check() } + catch { try native.state.meter.release(reservation); throw error } + try native.state.meter.release(reservation) + } else { + var local = try JSONWorkMeter(limits: .balanced, domain: .coding) + let (text, reservation) = try encoded(meter: &local) + do { try encodeFields(text, to: encoder); try local.checkpoint() } + catch { try local.release(reservation); throw error } + try local.release(reservation) + } + } + private func encodeFields(_ text: String, to encoder: any Encoder) throws { + // Do not keep an inout meter borrow across arbitrary Encoder callbacks. + var container = encoder.container(keyedBy: Keys.self) + try container.encode("swift-json-value", forKey: .format) + try container.encode(1, forKey: .version) + try container.encode(text, forKey: .utf8Base64) + } + private static func validated(_ marker: String, version: Int, text: String, meter: inout JSONWorkMeter) throws -> Data { + let before = meter.used(.retainedBytes) + do { + try JSONAdmission.text(marker, meter: &meter, owned: true) + try JSONAdmission.text(text, meter: &meter, owned: true) + guard marker == "swift-json-value", version == 1 else { throw JSONFailure(.syntax, code: .invalidSyntax) } + var count: UInt64 = 0 + for _ in text.utf8 { try meter.work(); count = try meter.adding(.stringBytes, count, 1) } + guard count % 4 == 0 else { throw JSONFailure(.syntax, code: .invalidSyntax) } + // Maximum decoded size is checked before Base64's first output byte. + let (maximum, overflow) = (count / 4).multipliedReportingOverflow(by: 3) + if overflow { _ = try meter.adding(.inputBytes, UInt64.max, 1) } + // Padding may reduce the inclusive decoded capacity by up to two. + var padding: UInt64 = 0 + for byte in text.utf8.suffix(2).reversed() { try meter.work(); if byte == 61 { padding += 1 } else { break } } + guard maximum >= padding else { throw JSONFailure(.syntax, code: .invalidSyntax) } + try envelopeCapacity(maximum - padding, meter: &meter) + let bytes = try JSONBase64.decode(text, meter: &meter) + let beforeDOM = meter.used(.retainedBytes) + _ = try parse(bytes, meter: &meter) // Certifies trailing syntax/whitespace/EOF. + try meter.release(meter.used(.retainedBytes) - beforeDOM) + try meter.release(meter.used(.retainedBytes) - before - UInt64(bytes.count)) + try meter.checkpoint(); return bytes + } catch { try meter.release(meter.used(.retainedBytes) - before); throw error } + } + /// Validates Version 1, strict Base64 and the complete inner document before storing only its bytes. + /// - Parameter decoder: The enclosing coder supplying fixed primitive fields. + /// - Throws: Ordinary Codable field errors or bounded corruption; operational errors retain their types. + public init(from decoder: any Decoder) throws { + let native = decoder as? JSONNativeDecoder + if let native { try native.state.meter.restrict(to: .balanced) } + let container = try decoder.container(keyedBy: Keys.self) + let marker = try container.decode(String.self, forKey: .format) + let version = try container.decode(Int.self, forKey: .version) + let text = try container.decode(String.self, forKey: .utf8Base64) + do { + if let native { utf8 = try Self.validated(marker, version: version, text: text, meter: &native.state.meter) } + else { + var local = try JSONWorkMeter(limits: .balanced, domain: .coding) + utf8 = try Self.validated(marker, version: version, text: text, meter: &local) + } + } catch let failure as JSONFailure where failure.category == .syntax { + throw DecodingError.dataCorrupted(.init(codingPath: decoder.codingPath, debugDescription: "Invalid Swift-JSON Version 1 payload", underlyingError: failure)) + } + } +} diff --git a/Sources/JSON/Coding/JSONCodingStrategies.swift b/Sources/JSON/Coding/JSONCodingStrategies.swift index d65478c..17a4fd0 100644 --- a/Sources/JSON/Coding/JSONCodingStrategies.swift +++ b/Sources/JSON/Coding/JSONCodingStrategies.swift @@ -43,10 +43,11 @@ public struct JSONCodingOptions: Sendable { } package final class JSONCodingState { package var meter: JSONWorkMeter - package let options: JSONCodingOptions + package private(set) var options: JSONCodingOptions package var failure: (any Error)? package private(set) var bridgedUserInfo: [CodingUserInfoKey: Any] = [:] private var prepared = false + private var preparedBytes: UInt64 = 0 package init(options: JSONCodingOptions, meter: consuming JSONWorkMeter) { self.options = options; self.meter = meter } @@ -55,13 +56,27 @@ package final class JSONCodingState { if !prepared { try meter.require(.objectMembers, UInt64(options.userInfo.count)) for (key, value) in options.userInfo { + let before = meter.used(.retainedBytes) try JSONAdmission.text(key.rawValue, key: true, meter: &meter, owned: true) try meter.work(); try meter.retain(JSONLogicalWeights.member) bridgedUserInfo[key] = value + preparedBytes += meter.used(.retainedBytes) - before } prepared = true } } + package func withOptions(_ requested: JSONCodingOptions, _ body: () throws -> T) throws -> T { + try meter.restrict(to: requested.limits) + let oldOptions = options, oldInfo = bridgedUserInfo, oldPrepared = prepared, oldBytes = preparedBytes + options = requested; bridgedUserInfo = [:]; prepared = false; preparedBytes = 0 + func restore() throws { + try meter.release(preparedBytes) + options = oldOptions; bridgedUserInfo = oldInfo; prepared = oldPrepared; preparedBytes = oldBytes + } + do { + try check(); let result = try body(); try check(); try restore(); return result + } catch { try restore(); throw error } + } package func latch(_ action: () throws -> Void) { do { try action() } catch { if failure == nil { failure = error } } } package func path(_ path: [any CodingKey], appending key: any CodingKey) throws -> [any CodingKey] { try check(); try meter.require(.maxDepth, UInt64(path.count) + 1) diff --git a/Sources/JSON/Coding/JSONEncodingContainers.swift b/Sources/JSON/Coding/JSONEncodingContainers.swift index 608d1b7..0659879 100644 --- a/Sources/JSON/Coding/JSONEncodingContainers.swift +++ b/Sources/JSON/Coding/JSONEncodingContainers.swift @@ -5,21 +5,9 @@ struct JSONKeyedEncodingContainer: KeyedEncodingContainerProtoco var codingPath: [any CodingKey] { encoder.codingPath } private func child(_ key: any CodingKey) throws -> JSONNativeEncoder { try encoder.state.check() - guard case .keyed(var members) = encoder.box.storage else { throw JSONFailure(.configuration, code: .invalidOperation) } + guard case .keyed(let storage) = encoder.box.storage else { throw JSONFailure(.configuration, code: .invalidOperation) } let child = try encoder.child(key) - var found: Int? - for index in members.indices { - try encoder.state.meter.work() - if try JSONComparison.textEquals(members[index].0.string, key.stringValue, meter: &encoder.state.meter) { found = index; break } - } - for _ in members { try encoder.state.meter.work() } - if let found { members[found] = (try JSONKey(key.stringValue, meter: &encoder.state.meter), child.box) } - else { - try encoder.state.meter.require(.objectMembers, UInt64(members.count) + 1) - try encoder.state.meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot) - members.append((try JSONKey(key.stringValue, meter: &encoder.state.meter), child.box)) - } - encoder.box.storage = .keyed(members) + try storage.set(JSONKey(key.stringValue, meter: &encoder.state.meter), box: child.box, meter: &encoder.state.meter) return child } mutating func encodeNil(forKey key: Key) throws { try child(key).set(.null) } diff --git a/Sources/JSON/Coding/JSONValueConveniences.swift b/Sources/JSON/Coding/JSONValueConveniences.swift new file mode 100644 index 0000000..56b66ab --- /dev/null +++ b/Sources/JSON/Coding/JSONValueConveniences.swift @@ -0,0 +1,28 @@ +// SPDX-License-Identifier: Apache-2.0 +extension JSON { + /// Encodes a Swift model directly through the existing native value coder. + /// No Foundation serialization intermediate or persistence envelope is introduced. + /// - Parameters: + /// - value: The Encodable model; caller callback errors propagate. + /// - coding: Native Date/Data strategies, Sendable context and finite limits. + /// - Throws: Native/caller coding errors, resource/configuration failure or cancellation. + public init(encoding value: T, coding: JSONCodingOptions = .init()) throws { + self = try JSONValueEncoder(options: coding).encode(value) + } + /// Decodes a Swift model with the existing native strategies and exact integers. + /// - Parameters: + /// - type: The target Decodable model. + /// - coding: Native strategies, Sendable context and finite limits. + /// - Throws: Native/caller coding errors, resource/configuration failure or cancellation. + public func decode(_ type: T.Type, coding: JSONCodingOptions = .init()) throws -> T { + try JSONValueDecoder(options: coding).decode(type, from: self) + } + /// Decodes a Swift model and reports successful owned native coding work. + /// - Parameters: + /// - type: The target Decodable model. + /// - coding: Native strategies, Sendable context and finite limits. + /// - Throws: Native/caller coding errors, resource/configuration failure or cancellation. + public func decodeWithReport(_ type: T.Type, coding: JSONCodingOptions = .init()) throws -> JSONOperationReport { + try JSONValueDecoder(options: coding).decodeWithReport(type, from: self) + } +} diff --git a/Sources/JSON/Coding/JSONValueDecoder.swift b/Sources/JSON/Coding/JSONValueDecoder.swift index cabbe2f..323ff74 100644 --- a/Sources/JSON/Coding/JSONValueDecoder.swift +++ b/Sources/JSON/Coding/JSONValueDecoder.swift @@ -87,7 +87,7 @@ package final class JSONNativeDecoder: Decoder { guard length <= 128 else { throw DecodingError.dataCorrupted(.init(codingPath: codingPath,debugDescription: "ISO-8601 text exceeds fixed format capacity")) } } try state.meter.checkpoint() - let strategy = Date.ISO8601FormatStyle().timeZone(separator: .colon).time(includingFractionalSeconds: true) + let strategy = Date.ISO8601FormatStyle().year().month().day().timeZone(separator: .colon).time(includingFractionalSeconds: true) guard let date = try? strategy.parse(string) else { throw DecodingError.dataCorrupted(.init(codingPath: codingPath, debugDescription: "Invalid ISO-8601 date")) } try state.meter.checkpoint(); return date as! T } diff --git a/Sources/JSON/Coding/JSONValueEncoder.swift b/Sources/JSON/Coding/JSONValueEncoder.swift index 2a24b01..c215974 100644 --- a/Sources/JSON/Coding/JSONValueEncoder.swift +++ b/Sources/JSON/Coding/JSONValueEncoder.swift @@ -45,8 +45,36 @@ public struct JSONValueEncoder: Sendable { return result } } +/// Exclusive mutable keyed storage avoids extracting/copying the whole member +/// array for every field. This object belongs to one native coding invocation. +package final class JSONKeyedEncodingStorage { + package private(set) var members: [(JSONKey, JSONEncodingBox)] = [] + private var index = JSONExactKeyIndex() + private var capacity = 0 + package func set(_ key: JSONKey, box: JSONEncodingBox, meter: inout JSONWorkMeter) throws { + if let position = try index.find(key, meter: &meter) { + try meter.work(); members[position] = (key, box); return + } + try meter.require(.objectMembers, UInt64(members.count) + 1) + if members.count == capacity { + let (doubled, overflow) = capacity.multipliedReportingOverflow(by: 2) + guard !overflow else { throw JSONFailure(.resource, code: .counterOverflow) } + let next = min(Int(meter.limits[.objectMembers]), max(1, doubled)) + let (bytes, byteOverflow) = UInt64(next).multipliedReportingOverflow(by: JSONLogicalWeights.member) + guard !byteOverflow else { throw JSONFailure(.resource, code: .counterOverflow) } + try meter.retain(bytes) + do { try meter.work(UInt64(members.count) + 1) } + catch { try meter.release(bytes); throw error } + members.reserveCapacity(next) + try meter.release(UInt64(capacity) * JSONLogicalWeights.member) + capacity = next + } + guard try index.insert(key, position: members.count, meter: &meter) else { throw JSONFailure(.syntax, code: .duplicateKey) } + members.append((key, box)) + } +} package final class JSONEncodingBox { - package enum Storage { case unset, value(JSON), keyed([(JSONKey, JSONEncodingBox)]), unkeyed([JSONEncodingBox]) } + package enum Storage { case unset, value(JSON), keyed(JSONKeyedEncodingStorage), unkeyed([JSONEncodingBox]) } package var storage: Storage = .unset package func materialize(state: JSONCodingState) throws -> JSON { enum Step { case node(JSONEncodingBox), object([JSONKey]), array(Int) } @@ -58,9 +86,11 @@ package final class JSONEncodingBox { switch node.storage { case .unset: throw EncodingError.invalidValue("unset", .init(codingPath: [], debugDescription: "No value was encoded")) case .value(let value): results.append(value) - case .keyed(let members): - pending.append(.object(members.map { $0.0 })) - for (_, box) in members.reversed() { try state.meter.work(); try state.meter.retain(16); pending.append(.node(box)) } + case .keyed(let storage): + var keys: [JSONKey] = [] + for (key, _) in storage.members { try state.meter.work(); try state.meter.retain(JSONLogicalWeights.arraySlot); keys.append(key) } + pending.append(.object(keys)) + for (_, box) in storage.members.reversed() { try state.meter.work(); try state.meter.retain(16); pending.append(.node(box)) } case .unkeyed(let boxes): pending.append(.array(boxes.count)) for box in boxes.reversed() { try state.meter.work(); try state.meter.retain(16); pending.append(.node(box)) } @@ -68,9 +98,9 @@ package final class JSONEncodingBox { case .object(let keys): var members: [JSONMember] = [] for (key, value) in zip(keys, results.suffix(keys.count)) { - try state.meter.work(); try state.meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot); members.append(JSONMember(key, value: value)) + try state.meter.work(); try state.meter.retain(JSONLogicalWeights.member); members.append(JSONMember(key, value: value)) } - results.removeLast(keys.count); results.append(.object(JSONObject(admitted: members))) + results.removeLast(keys.count); results.append(.object(try JSONObject.indexed(admitted: members, meter: &state.meter))) case .array(let count): var values: [JSON] = [] for value in results.suffix(count) { try state.meter.work(); try state.meter.retain(JSONLogicalWeights.arraySlot); values.append(value) } @@ -91,7 +121,7 @@ package final class JSONNativeEncoder: Encoder { state.latch { try state.check() switch box.storage { - case .unset: box.storage = .keyed([]) + case .unset: box.storage = .keyed(JSONKeyedEncodingStorage()) case .keyed: break default: throw EncodingError.invalidValue("container", .init(codingPath: codingPath, debugDescription: "Conflicting containers")) } @@ -129,7 +159,7 @@ package final class JSONNativeEncoder: Encoder { throw EncodingError.invalidValue(date,.init(codingPath: codingPath,debugDescription: "ISO-8601 year must be 0000...9999")) } try state.meter.retain(128); try state.meter.checkpoint() - let text = date.ISO8601Format(.iso8601.timeZone(separator: .colon).time(includingFractionalSeconds: true)) + let text = date.ISO8601Format(.iso8601.year().month().day().timeZone(separator: .colon).time(includingFractionalSeconds: true)) try state.meter.checkpoint(); try set(.string(text)); try state.meter.release(128) } } else { try value.encode(to: self) } diff --git a/Sources/JSON/Construction/JSONObjectAccumulator.swift b/Sources/JSON/Construction/JSONObjectAccumulator.swift index dd1b568..b7df0c4 100644 --- a/Sources/JSON/Construction/JSONObjectAccumulator.swift +++ b/Sources/JSON/Construction/JSONObjectAccumulator.swift @@ -4,6 +4,7 @@ /// Duplicate decoded keys fail; growth and payload work share the builder budget. public struct JSONObjectAccumulator: ~Copyable { private var members: [JSONMember] = [] + private var index = JSONExactKeyIndex() private var meter: JSONWorkMeter /// Creates an exclusive finite builder and admits its outer object frame. /// @@ -27,17 +28,15 @@ public struct JSONObjectAccumulator: ~Copyable { public mutating func append(_ key: String, value: JSON) throws { try meter.require(.objectMembers, UInt64(members.count) + 1) try JSONAdmission.text(key, key: true, meter: &meter, owned: true) - for member in members { - if try JSONComparison.textEquals(member.key.string, key, meter: &meter) { - throw JSONFailure(.syntax, code: .duplicateKey) - } - } + let admittedKey = try JSONKey(key, meter: &meter) + guard try index.find(admittedKey, meter: &meter) == nil else { throw JSONFailure(.syntax, code: .duplicateKey) } try JSONAdmission.value(value, meter: &meter, owned: true, depth: 1) - try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot) - members.append(JSONMember(try JSONKey(key, meter: &meter), value: value)) + try meter.retain(JSONLogicalWeights.member) + guard try index.insert(admittedKey, position: members.count, meter: &meter) else { throw JSONFailure(.syntax, code: .duplicateKey) } + members.append(JSONMember(admittedKey, value: value)) } /// Consumes the builder and transfers its already admitted result. - public consuming func finish() -> JSONObject { JSONObject(admitted: members) } + public consuming func finish() -> JSONObject { JSONObject(admitted: members, index: index) } } extension JSONObject { diff --git a/Sources/JSON/Diagnostics/JSONDiagnostic.swift b/Sources/JSON/Diagnostics/JSONDiagnostic.swift index 4385b7d..38a5acf 100644 --- a/Sources/JSON/Diagnostics/JSONDiagnostic.swift +++ b/Sources/JSON/Diagnostics/JSONDiagnostic.swift @@ -22,6 +22,7 @@ public struct JSONDiagnostic: Sendable, Equatable { /// - error: The error to classify without rendering its arbitrary description. public init(_ error: any Error) { if let error = error as? JSONFailure { kind = .failure; failure = error } + else if let error = error as? JSONAccessFailure { kind = .failure; failure = error.cause } else if error is CancellationError { kind = .cancellation; failure = nil } else { kind = .callerError; failure = nil } } diff --git a/Sources/JSON/Document/JSONSourceMap.swift b/Sources/JSON/Document/JSONSourceMap.swift index 2c1d783..3df9aac 100644 --- a/Sources/JSON/Document/JSONSourceMap.swift +++ b/Sources/JSON/Document/JSONSourceMap.swift @@ -39,6 +39,9 @@ public struct JSONSourceMap: Sendable { /// - limits: Finite inclusive capacities and work quotas for this call. public func span(at path: [JSONLocationComponent], kind: Entry.Kind = .value, limits: JSONLimits = .balanced) throws -> JSONSourceSpan? { var meter = try JSONWorkMeter(limits: limits, domain: .core) + return try span(at: path, kind: kind, meter: &meter) + } + package func span(at path: [JSONLocationComponent], kind: Entry.Kind = .value, meter: inout JSONWorkMeter) throws -> JSONSourceSpan? { for entry in entries { try meter.work() if entry.kind != kind || entry.path.count != path.count { continue } diff --git a/Sources/JSON/Documentation.docc/ExactPersistence.md b/Sources/JSON/Documentation.docc/ExactPersistence.md new file mode 100644 index 0000000..a62ccec --- /dev/null +++ b/Sources/JSON/Documentation.docc/ExactPersistence.md @@ -0,0 +1,40 @@ +# Exact persistence + +Transport exact JSON through a foreign Codable implementation using +``JSONCodablePayload``. The fixed Version 1 envelope carries the ASCII marker +`swift-json-value`, integer version `1` and strict standard Base64 in `utf8Base64`. +The inner strict UTF-8 document keeps decoded scalar sequences, member and array +order, exact decimal meaning and admitted numeric lexemes. Its compact writer +does not preserve source whitespace, escape spelling, labels or source maps. + +Construction/opening use finite balanced limits; explicit limits can tighten +them. Automatic Codable also uses balanced limits. Native nesting shares the +enclosing cumulative meter. Foreign coder allocations and callbacks remain caller +work. Logical bytes include overlapping owned buffers and temporary DOMs, and +are distinct from process RSS. A decoded wrapper stores only validated UTF-8; +``JSONCodablePayload/value(limits:)`` performs a new complete parse. + +Required missing/wrong field types use ordinary Codable errors. An unsupported +marker/version, malformed Base64 or invalid inner document produces bounded +`DecodingError.dataCorrupted`. Resource/configuration errors and cancellation +retain their types. Corruption never becomes a null value. + +SwiftData belongs in an application or example target importing SwiftData. +Use `@Attribute(.codable)` for the wrapper, with an independent application schema +version. An optional attribute of `nil` differs from a present wrapper preserving +JSON null. Query and sort ordinary model fields; this example establishes no +queryability of the embedded JSON. Persist/save, release the model/context/container +and recreate the container at the same disk URL to test reopening. + +Package version, payload format version, application data schema and SwiftData +model schema are separate. Version 1 has no heuristic legacy detection or silent +upgrade. Applications migrating bare JSON/Data must explicitly validate the +conversion and retain the original data according to their own migration policy. + +## Topics + +### Explicit wrapper + +- ``JSONCodablePayload`` +- ``JSONCodablePayload/preservingWithReport(_:limits:)`` +- ``JSONCodablePayload/valueWithReport(limits:)`` diff --git a/Sources/JSON/Documentation.docc/JSON.md b/Sources/JSON/Documentation.docc/JSON.md index 20ae563..bac4608 100644 --- a/Sources/JSON/Documentation.docc/JSON.md +++ b/Sources/JSON/Documentation.docc/JSON.md @@ -54,6 +54,7 @@ belongs exclusively to each invocation. ### Guides - +- ### Exact model - ``JSON/JSON`` @@ -83,3 +84,9 @@ belongs exclusively to each invocation. - ``JSONResourceMetric`` - ``JSONOperationStatistics`` - ``JSONOperationReport`` + +### Scoped reads + +- +- ``JSONReadCursor`` +- ``JSONAccessFailure`` diff --git a/Sources/JSON/Documentation.docc/ScopedReads.md b/Sources/JSON/Documentation.docc/ScopedReads.md new file mode 100644 index 0000000..e0783e6 --- /dev/null +++ b/Sources/JSON/Documentation.docc/ScopedReads.md @@ -0,0 +1,39 @@ +# Navigate and decode within one read scope + +Group related field reads under one finite cumulative budget. + +```swift +import JSON + +let response = try JSONParser().parse( + #"{"users":[{"name":"Ada","age":9007199254740993,"nickname":null}]}"#) +let report = try response.readWithReport { root in + let user = try root["users"][0] + let name = try user["name"].requiredString() + let age = try user["age"].requiredInteger(as: UInt64.self) + return (name, age, try user.stringPresence(of: "nickname")) +} +``` + +``JSONReadCursor`` brackets require a present member or valid array element. An +integer bracket never means a numeric object key. Keys are literal decoded scalar +sequences: slash, tilde, empty and dotted keys need no special bracket syntax. +Use parent presence to distinguish absence, explicit null and a typed non-null +value. Wrong kinds and incompatible models throw; they do not select a default. + +Required integer conversion preserves values above 2^53 and checks the target range. +For floating point, choose the fidelity explicitly: decimal 0.1 rejects +`requiredDouble(policy: .exact)` and permits finite `.rounded` conversion. +Native model projection uses `decode(_:coding:)`; native Float/Double models keep +their existing finite-rounding semantics. + +Root admission runs once. Owned path/copy/search/conversion/model work shares one +ledger, including prior work when coding limits tighten it. The report's core domain +and numeric counters describe owned operations, not caller instructions or RSS. +Catching resource failure or cancellation inside the callback cannot produce success. + +Cursors are synchronous and non-Sendable. Escaped handles become unusable, and the +scope releases its root and source references. Return ordinary values or models from +the callback. Later direct collection operations follow their existing contracts. +``JSONAccessFailure`` describes ordinary failures with a complete bounded typed path; +``JSONDocument`` reads can add coherent source locations. Replaced roots have none. diff --git a/Sources/JSON/Errors/JSONAccessFailure.swift b/Sources/JSON/Errors/JSONAccessFailure.swift new file mode 100644 index 0000000..b2ceed8 --- /dev/null +++ b/Sources/JSON/Errors/JSONAccessFailure.swift @@ -0,0 +1,13 @@ +// SPDX-License-Identifier: Apache-2.0 +/// An ordinary read failure with a complete bounded typed path and no JSON payload. +/// +/// Resource failures, limit configuration errors and cancellation keep their own +/// types. A mapped read can attach the selected value or missing member's parent +/// location to the cause. Replaced/unmapped documents never invent a location. +public struct JSONAccessFailure: Error, Sendable { + /// The stable navigation, kind or conversion failure, with an optional source location. + public let cause: JSONFailure + /// The exact attempted key/index path, including the failing hop. + public let path: [JSONLocationComponent] + package init(cause: JSONFailure, path: [JSONLocationComponent]) { self.cause = cause; self.path = path } +} diff --git a/Sources/JSON/Limits/JSONWorkMeter.swift b/Sources/JSON/Limits/JSONWorkMeter.swift index 2366971..79b6332 100644 --- a/Sources/JSON/Limits/JSONWorkMeter.swift +++ b/Sources/JSON/Limits/JSONWorkMeter.swift @@ -1,7 +1,13 @@ // SPDX-License-Identifier: Apache-2.0 /// Noncopyable so a nested helper cannot accidentally fork an invocation's budget. +private final class JSONMeterHealth { + var failure: (any Error)? + var observed = Array(repeating: UInt64(0), count: JSONResourceMetric.allCases.count) +} package struct JSONWorkMeter: ~Copyable { - package let limits: JSONLimits + package private(set) var limits: JSONLimits + private let health = JSONMeterHealth() + private let trackAcceptedCapacities: Bool package let domain: JSONOperationDomain private var counters: [UInt64] private var retired: [UInt64] @@ -13,29 +19,52 @@ package struct JSONWorkMeter: ~Copyable { /// Observer execution itself is trusted test code, outside owned work quotas. package var checkpointObserver: (@Sendable (UInt64) -> Void)? package mutating func schemaEvaluationPhase(_ active: Bool) { compilingEvaluation = active } - package init(limits: JSONLimits, domain: JSONOperationDomain) throws { + package init(limits: JSONLimits, domain: JSONOperationDomain, trackAcceptedCapacities: Bool = false) throws { try Task.checkCancellation() try limits.validate() self.limits = limits; self.domain = domain + self.trackAcceptedCapacities = trackAcceptedCapacities || domain == .coding counters = Array(repeating: 0, count: JSONResourceMetric.allCases.count) retired = Array(repeating: 0, count: JSONResourceMetric.allCases.count) } package func checkpoint() throws { - checkpointObserver?(used(.workUnits)) - try Task.checkCancellation() + if let failure = health.failure { throw failure } + do { + checkpointObserver?(used(.workUnits)) + try Task.checkCancellation() + } catch { health.failure = error; throw error } } package func used(_ metric: JSONResourceMetric) -> UInt64 { counters[metric.index] } package func require(_ metric: JSONResourceMetric, _ attempted: UInt64) throws { - guard attempted <= limits[metric] else { throw violation(metric, attempted) } + if let failure = health.failure { throw failure } + guard attempted <= limits[metric] else { + let failure = violation(metric, attempted); health.failure = failure; throw failure + } + // Fixed-policy operations never tighten after admission. Coding and read + // scopes retain the capacity history needed by restrict. Cumulative work + // already lives in counters; live ownership must use current, not peak. + if trackAcceptedCapacities { + switch metric { + case .workUnits, .numericWorkUnits, .compileWorkUnits, .evaluationWorkUnits, + .patchWorkUnits, .totalWorkUnits, .regexCompileWorkUnits, + .regexSearchWorkUnits, .aggregateRegexWorkUnits, .retainedBytes, + .regexRetainedBytes, .annotationBytes, .compiledPlanBytes: break + default: health.observed[metric.index] = max(health.observed[metric.index], attempted) + } + } } private func violation(_ metric: JSONResourceMetric, _ attempted: UInt64?, overflow: Bool = false) -> JSONFailure { - JSONFailure(.resource, code: overflow ? .counterOverflow : .limitExceeded, + let failure = JSONFailure(.resource, code: overflow ? .counterOverflow : .limitExceeded, resource: .init(domain: domain, metric: metric, limit: limits[metric], attemptedUsage: attempted, overflow: overflow)) + if health.failure == nil { health.failure = failure } + return failure } package func adding(_ metric: JSONResourceMetric, _ current: UInt64, _ amount: UInt64) throws -> UInt64 { let (next, overflow) = current.addingReportingOverflow(amount) - guard !overflow else { throw violation(metric, nil, overflow: true) } + guard !overflow else { + let failure = violation(metric, nil, overflow: true); health.failure = failure; throw failure + } try require(metric, next) return next } @@ -83,6 +112,19 @@ package struct JSONWorkMeter: ~Copyable { if let plan { counters[JSONResourceMetric.compiledPlanBytes.index] = plan } counters[JSONResourceMetric.retainedBytes.index] = retained; peak = max(peak,retained) } + /// Tightening keeps the same ledger. Earlier accepted capacities and consumed + /// work must fit the new ceiling; failures stay terminal even if caught. + package mutating func restrict(to requested: JSONLimits) throws { + try checkpoint() + guard trackAcceptedCapacities else { throw JSONFailure(.configuration, code: .invalidOperation) } + let restricted = try limits.intersection(requested) + limits = restricted + for metric in JSONResourceMetric.allCases where metric != .cancellationQuantum { + let live = metric == .retainedBytes || metric == .regexRetainedBytes || metric == .annotationBytes || metric == .compiledPlanBytes + try require(metric, live ? used(metric) : max(used(metric), health.observed[metric.index])) + } + sinceCancellationCheck = 0 + } /// Only live ownership is refunded; work and lifetime admission remain monotonic. package mutating func release(_ bytes: UInt64) throws { guard bytes <= used(.retainedBytes) else { throw JSONFailure(.configuration, code: .invalidLimits) } diff --git a/Sources/JSON/Number/JSONNumberConversions.swift b/Sources/JSON/Number/JSONNumberConversions.swift index 0990214..46e6a3e 100644 --- a/Sources/JSON/Number/JSONNumberConversions.swift +++ b/Sources/JSON/Number/JSONNumberConversions.swift @@ -147,6 +147,9 @@ extension JSONNumber { /// - limits: Finite inclusive capacities and work quotas for this call. public func doubleValue(policy: JSONNumberConversionPolicy = .exact, limits: JSONLimits = .balanced) throws -> Double { var meter = try JSONWorkMeter(limits: limits, domain: .core) + return try doubleValue(policy: policy, meter: &meter) + } + package func doubleValue(policy: JSONNumberConversionPolicy, meter: inout JSONWorkMeter) throws -> Double { let text = try binaryConversionText(meter: &meter) try meter.checkpoint() guard let value = Double(text), value.isFinite else { throw JSONFailure(.conversion, code: .inexactConversion) } @@ -170,6 +173,9 @@ extension JSONNumber { /// - limits: Finite inclusive capacities and work quotas for this call. public func decimalValue(policy: JSONNumberConversionPolicy = .exact, limits: JSONLimits = .balanced) throws -> Decimal { var meter = try JSONWorkMeter(limits: limits, domain: .core) + return try decimalValue(policy: policy, meter: &meter) + } + package func decimalValue(policy: JSONNumberConversionPolicy, meter: inout JSONWorkMeter) throws -> Decimal { let c = components guard c.digits.count <= 39, c.exponent >= -128, c.exponent <= 165 else { throw JSONFailure(.conversion, code: .inexactConversion) diff --git a/Sources/JSON/Pointer/JSONEditing.swift b/Sources/JSON/Pointer/JSONEditing.swift index 168220d..e17e366 100644 --- a/Sources/JSON/Pointer/JSONEditing.swift +++ b/Sources/JSON/Pointer/JSONEditing.swift @@ -36,17 +36,17 @@ package enum JSONEditing { switch change { case .remove: continue case .add(let value), .replace(let value): - try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot); members.append(JSONMember(member.key, value: value)); continue + try meter.retain(JSONLogicalWeights.member); members.append(JSONMember(member.key, value: value)); continue } } - try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot); members.append(member) + try meter.retain(JSONLogicalWeights.member); members.append(member) } if !found { guard case .add(let value) = change else { throw JSONFailure(.pointer, code: .missingTarget) } try meter.require(.objectMembers, UInt64(members.count) + 1) - try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot); members.append(JSONMember(token, value: value)) + try meter.retain(JSONLogicalWeights.member); members.append(JSONMember(token, value: value)) } - return .object(JSONObject(admitted: members)) + return .object(try JSONObject.indexed(admitted: members, meter: &meter)) case .array(let array): let add: Bool if case .add = change { add = true } else { add = false } diff --git a/Sources/JSON/Reading/JSONReadCursor.swift b/Sources/JSON/Reading/JSONReadCursor.swift new file mode 100644 index 0000000..b494f2f --- /dev/null +++ b/Sources/JSON/Reading/JSONReadCursor.swift @@ -0,0 +1,291 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation + +/// Synchronous typed access to one admitted immutable root under a shared budget. +/// +/// A cursor is deliberately not Sendable. Escaped handles become unusable when +/// their read scope exits; they retain no selected value or source snapshot. +/// Ordinary failures carry JSONAccessFailure's complete typed path. Resource +/// failure and cancellation are terminal even if caught inside the callback. +public struct JSONReadCursor { + private let scope: JSONReadScope + private let selection: JSONReadPath + private var path: [JSONLocationComponent] { selection.components } + package init(scope: JSONReadScope, selection: JSONReadPath) { self.scope = scope; self.selection = selection } + /// Requires an exactly matching object member; keys are literal decoded scalars. + /// - Parameter key: The member name; empty, slash, tilde and dotted keys are literal. + /// - Throws: Bounded access failure, resource/configuration failure or cancellation. + public subscript(_ key: String) -> Self { + get throws { + try scope.run(path) { state in + try JSONAdmission.text(key, key: true, meter: &state.meter) + return try child(.key(JSONKey(key, meter: &state.meter)), state: state) + } + } + } + /// Requires an exactly matching object member using an existing scalar key. + /// - Parameter key: The exact member key, without canonical normalization. + /// - Throws: Bounded access failure, resource/configuration failure or cancellation. + public subscript(_ key: JSONKey) -> Self { + get throws { + try scope.run(path) { state in + try JSONAdmission.text(key.string, key: true, meter: &state.meter) + return try child(.key(key), state: state) + } + } + } + /// Requires an array and a nonnegative in-range element; numeric keys stay keys. + /// - Parameter index: The zero-based element index, without negative indexing. + /// - Throws: Bounded access failure, resource/configuration failure or cancellation. + public subscript(_ index: Int) -> Self { + get throws { try scope.run(path) { try child(.index(index), state: $0) } } + } + private func child(_ part: JSONLocationComponent, state: JSONCodingState) throws -> Self { + let attempted = try scope.append(part, to: path, state: state) + return try scope.run(attempted.components) { state in + _ = try scope.select(attempted.components, state: state) + return Self(scope: scope, selection: attempted) + } + } + private func selected(_ body: (JSON, JSONCodingState) throws -> T) throws -> T { + try scope.run(path) { state in try body(scope.select(path, state: state), state) } + } + /// Returns the selected value, including JSON null, without coercion. + /// Subsequent caller collection operations retain their ordinary accounting contracts. + /// - Throws: Bounded access failure, resource/configuration failure or cancellation. + public func requiredValue() throws -> JSON { try selected { value, state in try state.meter.work(); return value } } + /// Requires a JSON string and admits its returned text under the active meter. + /// - Throws: Wrong-kind access failure, resource/configuration failure or cancellation. + public func requiredString() throws -> String { + try selected { value, state in + guard case .string(let text) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + let before = state.meter.used(.retainedBytes) + try JSONAdmission.text(text, meter: &state.meter, owned: true) + try state.meter.release(state.meter.used(.retainedBytes) - before) + return text + } + } + /// Requires a JSON Boolean; numbers and strings are not coerced. + /// - Throws: Wrong-kind access failure, resource/configuration failure or cancellation. + public func requiredBoolean() throws -> Bool { + try selected { value, _ in + guard case .boolean(let result) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) }; return result + } + } + /// Requires an exact JSON number, retaining its admitted original lexeme. + /// - Throws: Wrong-kind access failure, resource/configuration failure or cancellation. + public func requiredNumber() throws -> JSONNumber { + try selected { value, _ in + guard case .number(let result) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) }; return result + } + } + /// Requires an ordered exact-key object; later direct lookup is caller work. + /// - Throws: Wrong-kind access failure, resource/configuration failure or cancellation. + public func requiredObject() throws -> JSONObject { + try selected { value, _ in + guard case .object(let result) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) }; return result + } + } + /// Requires an array; later direct collection traversal is caller work. + /// - Throws: Wrong-kind access failure, resource/configuration failure or cancellation. + public func requiredArray() throws -> [JSON] { + try selected { value, _ in + guard case .array(let result) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) }; return result + } + } + /// Requires an integral exact number within the target's signed/unsigned range. + /// - Parameter type: The requested fixed-width integer type. + /// - Throws: Kind/conversion access failure, resource/configuration failure or cancellation. + public func requiredInteger(as type: T.Type) throws -> T { + try selected { value, state in + guard case .number(let number) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + return try number.integer(as: type, meter: &state.meter) + } + } + /// Projects an exact number to a finite Double with an explicit fidelity choice. + /// Decimal 0.1 rejects `.exact`; `.rounded` permits finite approximation. + /// - Parameter policy: Mathematical equality or explicitly permitted finite rounding. + /// - Throws: Kind/conversion access failure, resource/configuration failure or cancellation. + public func requiredDouble(policy: JSONNumberConversionPolicy) throws -> Double { + try selected { value, state in + guard case .number(let number) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + let before = state.meter.used(.retainedBytes) + do { + let result = try number.doubleValue(policy: policy, meter: &state.meter) + try state.meter.release(state.meter.used(.retainedBytes) - before); return result + } catch { try state.meter.release(state.meter.used(.retainedBytes) - before); throw error } + } + } + /// Projects an exact number to finite Decimal with an explicit fidelity choice. + /// - Parameter policy: Mathematical equality or explicitly permitted finite rounding. + /// - Throws: Kind/conversion access failure, resource/configuration failure or cancellation. + public func requiredDecimal(policy: JSONNumberConversionPolicy) throws -> Decimal { + try selected { value, state in + guard case .number(let number) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + let before = state.meter.used(.retainedBytes) + do { + let result = try number.decimalValue(policy: policy, meter: &state.meter) + try state.meter.release(state.meter.used(.retainedBytes) - before); return result + } catch { try state.meter.release(state.meter.used(.retainedBytes) - before); throw error } + } + } + private func field(_ key: JSONKey, _ body: (JSON, JSONCodingState) throws -> T) throws -> JSONPresence { + try scope.run(path) { state in + try JSONAdmission.text(key.string, key: true, meter: &state.meter) + let attempted = try scope.append(.key(key), to: path, state: state) + return try scope.run(attempted.components) { state in + guard case .object(let object) = try scope.select(path, state: state) else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + guard let value = try object.lookup(key.string, meter: &state.meter) else { return .absent } + if value.isNull { return .null } + return .value(try body(value, state)) + } + } + } + private func admitted(_ key: String, _ body: (JSONKey) throws -> JSONPresence) throws -> JSONPresence { + try scope.run(path) { state in + try JSONAdmission.text(key, key: true, meter: &state.meter) + return try body(JSONKey(key, meter: &state.meter)) + } + } + /// Distinguishes absence, explicit null and an existing member's exact value. + /// - Parameter key: The exact decoded-scalar member key. + /// - Throws: Kind/access failure, resource/configuration failure or cancellation. + public func presence(of key: JSONKey) throws -> JSONPresence { try field(key) { value, state in try state.meter.work(); return value } } + /// Distinguishes absence, explicit null and an existing member's exact value. + /// - Parameter key: The literal decoded-scalar member name. + /// - Throws: Kind/access failure, resource/configuration failure or cancellation. + public func presence(of key: String) throws -> JSONPresence { try admitted(key) { try presence(of: $0) } } + + /// Distinguishes absence and null while requiring a non-null JSON string. + /// - Parameter key: The exact decoded-scalar member key. + /// - Throws: Kind/access failure, resource/configuration failure or cancellation. + public func stringPresence(of key: JSONKey) throws -> JSONPresence { + try field(key) { value, state in + guard case .string(let result) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + let before = state.meter.used(.retainedBytes) + try JSONAdmission.text(result, meter: &state.meter, owned: true) + try state.meter.release(state.meter.used(.retainedBytes) - before) + return result + } + } + /// Distinguishes absence and null while requiring a non-null JSON string. + /// - Parameter key: The literal decoded-scalar member name. + /// - Throws: Kind/access failure, resource/configuration failure or cancellation. + public func stringPresence(of key: String) throws -> JSONPresence { try admitted(key) { try stringPresence(of: $0) } } + + /// Distinguishes absence and null while requiring a non-null JSON boolean. + /// - Parameter key: The exact decoded-scalar member key. + /// - Throws: Kind/access failure, resource/configuration failure or cancellation. + public func booleanPresence(of key: JSONKey) throws -> JSONPresence { + try field(key) { value, state in + guard case .boolean(let result) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + try state.meter.work() + return result + } + } + /// Distinguishes absence and null while requiring a non-null JSON boolean. + /// - Parameter key: The literal decoded-scalar member name. + /// - Throws: Kind/access failure, resource/configuration failure or cancellation. + public func booleanPresence(of key: String) throws -> JSONPresence { try admitted(key) { try booleanPresence(of: $0) } } + + /// Distinguishes absence and null while requiring a non-null JSON number. + /// - Parameter key: The exact decoded-scalar member key. + /// - Throws: Kind/access failure, resource/configuration failure or cancellation. + public func numberPresence(of key: JSONKey) throws -> JSONPresence { + try field(key) { value, state in + guard case .number(let result) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + try state.meter.work() + return result + } + } + /// Distinguishes absence and null while requiring a non-null JSON number. + /// - Parameter key: The literal decoded-scalar member name. + /// - Throws: Kind/access failure, resource/configuration failure or cancellation. + public func numberPresence(of key: String) throws -> JSONPresence { try admitted(key) { try numberPresence(of: $0) } } + private func project(_ type: T.Type, value: JSON, coding: JSONCodingOptions, state: JSONCodingState) throws -> T { + let before = state.meter.used(.retainedBytes) + do { + let result = try state.withOptions(coding) { try JSONNativeDecoder(value: value, state: state, codingPath: []).decode(type) } + try state.meter.release(state.meter.used(.retainedBytes) - before) + return result + } catch { try state.meter.release(state.meter.used(.retainedBytes) - before); throw error } + } + /// Projects the selected value through the existing native decoder and ledger. + /// Coding ceilings permanently tighten this scope against prior usage. Caller + /// Decodable computation remains caller work; its errors propagate. + /// - Parameters: + /// - type: The target Decodable model. + /// - coding: Native strategies/context and intersected finite limits. + /// - Throws: Native/caller decoding errors, resource/configuration failure or cancellation. + public func decode(_ type: T.Type, coding: JSONCodingOptions = .init()) throws -> T { + try selected { value, state in + try project(type, value: value, coding: coding, state: state) + } + } + /// Decodes an existing non-null member, without calling the model for absence/null. + /// - Parameters: + /// - key: The exact decoded-scalar member key. + /// - type: The target Decodable model. + /// - coding: Native strategies/context and intersected finite limits. + /// - Throws: Native/caller decoding errors, resource/configuration failure or cancellation. + public func presence(of key: JSONKey, as type: T.Type, coding: JSONCodingOptions = .init()) throws -> JSONPresence { + try field(key) { value, state in + try project(type, value: value, coding: coding, state: state) + } + } + /// Decodes an existing non-null member, without calling the model for absence/null. + /// - Parameters: + /// - key: The literal decoded-scalar member name. + /// - type: The target Decodable model. + /// - coding: Native strategies/context and intersected finite limits. + /// - Throws: Native/caller decoding errors, resource/configuration failure or cancellation. + public func presence(of key: String, as type: T.Type, coding: JSONCodingOptions = .init()) throws -> JSONPresence { + try scope.run(path) { state in + try JSONAdmission.text(key, key: true, meter: &state.meter) + return try presence(of: JSONKey(key, meter: &state.meter), as: type, coding: coding) + } + } +} + +extension JSON { + /// Runs related typed reads with one root admission and cumulative finite budget. + /// The synchronous callback receives a non-Sendable, scope-bound cursor. + /// - Parameters: + /// - limits: Inclusive root, path, work, numeric and logical retention ceilings. + /// - body: Caller computation; returned payloads retain ordinary value semantics. + /// - Throws: Access/caller errors, resource/configuration failure or cancellation. + public func read(limits: JSONLimits = .balanced, _ body: (JSONReadCursor) throws -> Result) throws -> Result { + try readWithReport(limits: limits, body).result + } + /// Runs related reads and reports all successful owned work in the core domain. + /// Catching terminal failure in the callback cannot produce a successful report. + /// - Parameters: + /// - limits: Inclusive root, path, work, numeric and logical retention ceilings. + /// - body: Synchronous caller computation, outside owned instruction/RSS claims. + /// - Throws: Access/caller errors, resource/configuration failure or cancellation. + public func readWithReport(limits: JSONLimits = .balanced, _ body: (JSONReadCursor) throws -> Result) throws -> JSONOperationReport { + try JSONDocument(root: self).readWithReport(limits: limits, body) + } +} +extension JSONDocument { + /// Runs related typed reads with optional coherent mapped source diagnostics. + /// - Parameters: + /// - limits: Inclusive root, source, path and cumulative work/retention ceilings. + /// - body: Synchronous caller computation using a scope-bound non-Sendable cursor. + /// - Throws: Access/caller errors, resource/configuration failure or cancellation. + public func read(limits: JSONLimits = .balanced, _ body: (JSONReadCursor) throws -> Result) throws -> Result { + try readWithReport(limits: limits, body).result + } + /// Reports successful reads while preserving this document's source coherence. + /// Scope exit releases library root/source ownership and invalidates escaped handles. + /// - Parameters: + /// - limits: Inclusive root, source, path and cumulative work/retention ceilings. + /// - body: Synchronous caller computation, outside owned instruction/RSS claims. + /// - Throws: Access/caller errors, resource/configuration failure or cancellation. + public func readWithReport(limits: JSONLimits = .balanced, _ body: (JSONReadCursor) throws -> Result) throws -> JSONOperationReport { + let scope = try JSONReadScope(self, limits: limits) + defer { scope.invalidate() } + let root = JSONReadCursor(scope: scope, selection: JSONReadPath(scope: scope, components: [], bytes: 0)) + return try scope.complete(body(root)) + } +} diff --git a/Sources/JSON/Reading/JSONReadScope.swift b/Sources/JSON/Reading/JSONReadScope.swift new file mode 100644 index 0000000..1c9dbef --- /dev/null +++ b/Sources/JSON/Reading/JSONReadScope.swift @@ -0,0 +1,155 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation + +/// Guarded synchronous ownership; deliberately not Sendable. An escaped handle +/// stores only a bounded path and this invalidated scope, never a selected DOM. +package final class JSONReadScope { + private let owner = ObjectIdentifier(Thread.current) + package var document: JSONDocument? + package var state: JSONCodingState? + private var rootBytes: UInt64 = 0 + private var pathBytes: UInt64 = 0 + private var retiredPathBytes: UInt64 = 0 + package init(_ document: JSONDocument, limits: JSONLimits) throws { + let state = JSONCodingState(options: .init(limits: limits), meter: try JSONWorkMeter(limits: limits, domain: .core, trackAcceptedCapacities: true)) + try JSONAdmission.value(document.root, meter: &state.meter, owned: true) + if let source = document.source { try JSONAdmission.text(source, meter: &state.meter, owned: true) } + if let bytes = document.sourceBytes { + try state.meter.require(.inputBytes, UInt64(bytes.count)) + for _ in bytes { try state.meter.work(); try state.meter.retain(1) } + } + if let map = document.sourceMap { + for entry in map.entries { + try state.meter.work(); try state.meter.charge(.sourceMapEntries) + try state.meter.retain(JSONLogicalWeights.sourceMapEntry) + for part in entry.path { + try state.meter.work(); try state.meter.retain(JSONLogicalWeights.frame) + if case .key(let key) = part { try JSONAdmission.text(key.string, key: true, meter: &state.meter, owned: true) } + } + } + } + self.document = document; self.state = state + rootBytes = state.meter.used(.retainedBytes) + } + package func retirePath(_ bytes: UInt64) { if ObjectIdentifier(Thread.current) == owner, state != nil { retiredPathBytes += bytes } } + package func check() throws -> JSONCodingState { + guard ObjectIdentifier(Thread.current) == owner else { throw JSONFailure(.configuration, code: .invalidOperation) } + guard document != nil, let state else { throw JSONFailure(.configuration, code: .invalidOperation) } + if retiredPathBytes != 0 { + try state.meter.release(retiredPathBytes); pathBytes -= retiredPathBytes; retiredPathBytes = 0 + } + try state.check(); return state + } + package func invalidate() { document = nil; state = nil } + package func complete(_ result: T) throws -> JSONOperationReport { + let state = try check() + // Paths/root/source cease to be scope-owned. Any escaped path is caller + // storage and cannot access this invalidated root or ledger. + try state.meter.release(rootBytes + pathBytes) + let statistics = state.meter.statistics() + invalidate() + return .init(result: result, statistics: statistics) + } + package func run(_ path: [JSONLocationComponent], _ body: (JSONCodingState) throws -> T) throws -> T { + do { return try body(check()) } + catch is CancellationError { + let error = CancellationError(); if state?.failure == nil { state?.failure = error }; throw error + } + catch let failure as JSONFailure where failure.category == .resource { + if state?.failure == nil { state?.failure = failure }; throw failure + } + catch let failure as JSONFailure where failure.category == .typeMismatch || failure.category == .missingValue || failure.category == .conversion || failure.code == .invalidIndex { + guard let state else { throw failure } + let location = try location(for: path, missing: failure.category == .missingValue, state: state) + throw JSONAccessFailure(cause: .init(failure.category, code: failure.code, location: location ?? failure.location, resource: failure.resource), path: path) + } + } + private func location(for path: [JSONLocationComponent], missing: Bool, state: JSONCodingState) throws -> JSONSourceLocation? { + guard let document, let map = document.sourceMap else { return nil } + var selected: [JSONLocationComponent] = [] + for part in path.prefix(missing ? max(0, path.count - 1) : path.count) { + try state.meter.work(); try state.meter.retain(JSONLogicalWeights.frame); selected.append(part) + } + var span: JSONSourceSpan? + do { + span = try map.span(at: selected, meter: &state.meter) + if span == nil, !missing, !selected.isEmpty { + // A wrong-container or negative-index hop has no child span. + // Borrow the already bounded prefix; do not invent a location. + selected.removeLast(); try state.meter.release(JSONLogicalWeights.frame) + span = try map.span(at: selected, meter: &state.meter) + } + } + catch { try state.meter.release(UInt64(selected.count) * JSONLogicalWeights.frame); throw error } + try state.meter.release(UInt64(selected.count) * JSONLogicalWeights.frame) + guard let span else { return nil } + guard let bytes = document.sourceBytes else { return .init(byteOffset: span.start) } + var line: UInt64 = 1, column: UInt64 = 1, offset: UInt64 = 0 + for byte in bytes { + if offset == span.start { break } + try state.meter.work(); offset += 1 + if byte == 10 { line += 1; column = 1 } else { column += 1 } + } + return .init(byteOffset: span.start, line: line, column: column) + } + package func select(_ path: [JSONLocationComponent], state: JSONCodingState) throws -> JSON { + guard let document else { throw JSONFailure(.configuration, code: .invalidOperation) } + var value = document.root + for part in path { + try state.meter.work() + switch part { + case .key(let key): + guard case .object(let object) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + guard let child = try object.lookup(key.string, meter: &state.meter) else { throw JSONFailure(.missingValue, code: .missingMember) } + value = child + case .index(let index): + guard case .array(let array) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + guard index >= 0 else { throw JSONFailure(.configuration, code: .invalidIndex) } + guard index < array.count else { throw JSONFailure(.missingValue, code: .missingTarget) } + value = array[index] + } + } + return value + } + package func append(_ part: JSONLocationComponent, to path: [JSONLocationComponent], state: JSONCodingState) throws -> JSONReadPath { + try state.meter.require(.pointerTokens, UInt64(path.count) + 1) + var pointerBytes: UInt64 = 0, owned: UInt64 = 0 + for position in 0...path.count { + let component = position == path.count ? part : path[position] + pointerBytes = try state.meter.adding(.pointerBytes, pointerBytes, 1) + let text: String + switch component { + case .key(let key): text = key.string + case .index(let index): + // Preflight the bounded formatting bridge before it allocates. + try state.meter.retain(21); try state.meter.checkpoint() + text = String(index) + try state.meter.work(UInt64(text.utf8.count)); try state.meter.release(21) + } + try state.meter.work(); owned = try state.meter.adding(.retainedBytes, owned, JSONLogicalWeights.indexSlot) + for byte in text.utf8 { + try state.meter.work() + pointerBytes = try state.meter.adding(.pointerBytes, pointerBytes, byte == 47 || byte == 126 ? 2 : 1) + if case .key = component { owned = try state.meter.adding(.retainedBytes, owned, 1) } + } + } + try state.meter.require(.diagnosticBytes, pointerBytes) + try state.meter.retain(owned) + var result: [JSONLocationComponent] = [] + do { + for component in path { try state.meter.work(); result.append(component) } + try state.meter.work(); result.append(part) + } catch { try state.meter.release(owned); throw error } + pathBytes += owned + return JSONReadPath(scope: self, components: result, bytes: owned) + } +} +package final class JSONReadPath { + package let scope: JSONReadScope + package let components: [JSONLocationComponent] + private let bytes: UInt64 + package init(scope: JSONReadScope, components: [JSONLocationComponent], bytes: UInt64) { + self.scope = scope; self.components = components; self.bytes = bytes + } + deinit { scope.retirePath(bytes) } +} diff --git a/Sources/JSON/Value/JSONComparison.swift b/Sources/JSON/Value/JSONComparison.swift index 394086c..22694cc 100644 --- a/Sources/JSON/Value/JSONComparison.swift +++ b/Sources/JSON/Value/JSONComparison.swift @@ -43,10 +43,11 @@ package enum JSONAdmission { case .object(let object): let containerDepth = try meter.adding(.maxDepth, parentDepth, 1) try meter.require(.objectMembers, UInt64(object.count)) + if owned { try meter.retain(object.exactIndex.retainedBytes) } for member in object.reversed() { try text(member.key.string, key: true, meter: &meter, owned: owned) try meter.work(); try meter.retain(JSONLogicalWeights.frame) - if owned { try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot) } + if owned { try meter.retain(JSONLogicalWeights.member) } stack.append((member.value, containerDepth)) } } diff --git a/Sources/JSON/Value/JSONExactKeyIndex.swift b/Sources/JSON/Value/JSONExactKeyIndex.swift new file mode 100644 index 0000000..ab77da6 --- /dev/null +++ b/Sources/JSON/Value/JSONExactKeyIndex.swift @@ -0,0 +1,111 @@ +// SPDX-License-Identifier: Apache-2.0 +/// Invocation-owned while building; immutable after publication. Fingerprints +/// only select candidates. Every hit confirms the complete decoded UTF-8 key. +package struct JSONExactKeyIndex: Sendable { + private struct Entry: Sendable { let key: JSONKey; let position: Int } + private var linear: [Entry] = [] + private var table: [Entry?] = [] + private var capacity = 0 + private var count = 0 + private let fingerprintMask: UInt64 + package init(fingerprintMask: UInt64 = .max) { self.fingerprintMask = fingerprintMask } + package var retainedBytes: UInt64 { UInt64(capacity) * JSONLogicalWeights.indexSlot } + private func same(_ a: JSONKey, _ b: JSONKey, meter: inout JSONWorkMeter) throws -> Bool { + try meter.work() + guard a.fingerprint & fingerprintMask == b.fingerprint & fingerprintMask else { return false } + return try JSONComparison.textEquals(a.string, b.string, meter: &meter) + } + private func slot(_ key: JSONKey, in slots: [Entry?], meter: inout JSONWorkMeter) throws -> (Int, Int?) { + var index = Int(key.fingerprint & fingerprintMask & UInt64(slots.count - 1)) + for _ in slots.indices { + try meter.work() + guard let entry = slots[index] else { return (index, nil) } + if try same(entry.key, key, meter: &meter) { return (index, entry.position) } + index = (index + 1) & (slots.count - 1) + } + throw JSONFailure(.configuration, code: .invalidOperation) + } + package func find(_ key: JSONKey, meter: inout JSONWorkMeter) throws -> Int? { + if !table.isEmpty { return try slot(key, in: table, meter: &meter).1 } + for entry in linear { + if try same(entry.key, key, meter: &meter) { return entry.position } + } + return nil + } + package func find(_ key: JSONKey) -> Int? { + if table.isEmpty { return linear.first { $0.key == key }?.position } + var index = Int(key.fingerprint & fingerprintMask & UInt64(table.count - 1)) + for _ in table.indices { + guard let entry = table[index] else { return nil } + if entry.key == key { return entry.position } + index = (index + 1) & (table.count - 1) + } + return nil + } + private func bytes(_ slots: Int, meter: inout JSONWorkMeter) throws -> UInt64 { + let (bytes, overflow) = UInt64(slots).multipliedReportingOverflow(by: JSONLogicalWeights.indexSlot) + guard !overflow else { throw JSONFailure(.resource, code: .counterOverflow) } + try meter.require(.retainedBytes, bytes) + return bytes + } + /// No mutation occurs until all throwing probes, growth/copies and charges + /// complete. A rejected append therefore preserves valid prior metadata. + package mutating func insert(_ key: JSONKey, position: Int, meter: inout JSONWorkMeter) throws -> Bool { + if try find(key, meter: &meter) != nil { return false } + try meter.require(.objectMembers, UInt64(count) + 1) + let entry = Entry(key: key, position: position) + if count < 31 { + if count == capacity { + let next = min(Int(meter.limits[.objectMembers]), max(1, capacity * 2)) + let nextBytes = try bytes(next, meter: &meter) + try meter.retain(nextBytes) + // reserveCapacity can copy the old buffer; charge its traversal + // while both old and replacement capacities are live. + do { try meter.work(UInt64(count) + 1) } + catch { try meter.release(nextBytes); throw error } + linear.reserveCapacity(next) + try meter.release(retainedBytes) + capacity = next + } else { try meter.work() } + linear.append(entry); count += 1 + return true + } + if table.isEmpty || count >= table.count / 2 { + let (next, overflow) = max(32, table.count).multipliedReportingOverflow(by: 2) + guard !overflow else { throw JSONFailure(.resource, code: .counterOverflow) } + let nextBytes = try bytes(next, meter: &meter) + try meter.retain(nextBytes) + var replacement: [Entry?] = [] + replacement.reserveCapacity(next) + do { + for _ in 0.. Bool { + try meter.work() + guard self.fingerprint == fingerprint else { return false } + return try JSONComparison.textEquals(string, spelling, meter: &meter) + } /// Constructs a trusted literal value without a resource budget; bounded operations admit it later. /// /// - Parameters: diff --git a/Sources/JSON/Value/JSONObject.swift b/Sources/JSON/Value/JSONObject.swift index ff13b29..818ba93 100644 --- a/Sources/JSON/Value/JSONObject.swift +++ b/Sources/JSON/Value/JSONObject.swift @@ -2,7 +2,7 @@ /// An ordered collection of exact-key members with order-independent equality. /// /// Replacing a member preserves its position; removing and re-inserting appends it. -/// Lookup is linear in the scalar implementation. Use bounded operations for untrusted work. +/// Private exact-key metadata accelerates wide objects. Use bounded operations for untrusted work. public struct JSONObject: Sendable, Hashable, RandomAccessCollection { /// The integer position of a member in insertion order. public typealias Index = Int @@ -10,6 +10,7 @@ public struct JSONObject: Sendable, Hashable, RandomAccessCollection { public typealias Element = JSONMember /// Members in deterministic insertion order. public let members: [JSONMember] + package let exactIndex: JSONExactKeyIndex /// Admits ordered members and rejects identical decoded scalar keys. /// /// - Throws: Duplicate-key, value/growth/work failure or cancellation. @@ -21,26 +22,35 @@ public struct JSONObject: Sendable, Hashable, RandomAccessCollection { var meter = try JSONWorkMeter(limits: limits, domain: .core) self = try Self.checked(members: members, meter: &meter) } - package init(admitted members: [JSONMember]) { - self.members = members + package init(admitted members: [JSONMember], index: JSONExactKeyIndex) { + self.members = members; exactIndex = index + } + package init(empty: Void) { members = []; exactIndex = .init() } + package static func indexed(admitted members: [JSONMember], meter: inout JSONWorkMeter) throws -> Self { + var index = JSONExactKeyIndex() + for position in members.indices { + guard try index.insert(members[position].key, position: position, meter: &meter) else { + throw JSONFailure(.syntax, code: .duplicateKey) + } + } + return Self(admitted: members, index: index) } package static func checked(members: [JSONMember], meter: inout JSONWorkMeter) throws -> Self { try meter.require(.objectMembers, UInt64(members.count)) var output: [JSONMember] = [] + var index = JSONExactKeyIndex() for member in members { try JSONAdmission.text(member.key.string, key: true, meter: &meter, owned: true) - for previous in output { - if try JSONComparison.textEquals(previous.key.string, member.key.string, meter: &meter) { - throw JSONFailure(.syntax, code: .duplicateKey) - } + guard try index.insert(member.key, position: output.count, meter: &meter) else { + throw JSONFailure(.syntax, code: .duplicateKey) } try JSONAdmission.value(member.value, meter: &meter, owned: true, depth: 1) - try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot) + try meter.retain(JSONLogicalWeights.member) output.append(member) } try meter.charge(.nodes); try meter.retain(JSONLogicalWeights.node) try meter.require(.maxDepth, 1) - return Self(admitted: output) + return Self(admitted: output, index: index) } /// The first member index; equal to endIndex for an empty object. public var startIndex: Int { members.startIndex } @@ -52,15 +62,15 @@ public struct JSONObject: Sendable, Hashable, RandomAccessCollection { /// - Parameters: /// - position: A valid insertion-order member index. public subscript(position: Int) -> JSONMember { members[position] } - /// Looks up an exact decoded-scalar key with unmetered linear traversal. + /// Looks up an exact decoded-scalar key using private unmetered metadata. /// /// Returns nil for absence and JSON.null for a present null. Use bounded JSON /// member access when untrusted traversal requires work accounting. /// /// - Parameters: /// - key: The exact decoded-scalar member key, without normalization. - public subscript(key: JSONKey) -> JSON? { members.first { $0.key == key }?.value } - /// Looks up an exact decoded-scalar key with unmetered linear traversal. + public subscript(key: JSONKey) -> JSON? { exactIndex.find(key).map { members[$0].value } } + /// Looks up an exact decoded-scalar key using private unmetered metadata. /// /// Returns nil for absence and JSON.null for a present null. Use bounded JSON /// member access when untrusted traversal requires work accounting. @@ -82,11 +92,16 @@ public struct JSONObject: Sendable, Hashable, RandomAccessCollection { /// - hasher: The caller's randomized Swift hasher. public func hash(into hasher: inout Hasher) { JSON.object(self).hash(into: &hasher) } package func lookup(_ key: String, meter: inout JSONWorkMeter) throws -> JSON? { - for member in members { + // Preserve the qualified scalar tiny/early-hit path before hashing. + for member in members.prefix(members.count < 32 ? members.count : 8) { try meter.work() - if try JSONComparison.textEquals(member.key.string, key, meter: &meter) { return member.value } + if try JSONComparison.textEquals(member.key.string, key, meter: &meter) { + return member.value + } } - return nil + guard members.count >= 32 else { return nil } + let query = try JSONKey(key, meter: &meter) + return try exactIndex.find(query, meter: &meter).map { members[$0].value } } /// Returns an admitted object with a member added or replaced in place. /// diff --git a/Sources/JSONPatch/JSONMergePatch.swift b/Sources/JSONPatch/JSONMergePatch.swift index 4aea5c9..5c4b683 100644 --- a/Sources/JSONPatch/JSONMergePatch.swift +++ b/Sources/JSONPatch/JSONMergePatch.swift @@ -45,7 +45,7 @@ public struct JSONMergePatch: Sendable { return .init(result: patch, statistics: meter.statistics()) } try meter.retain(JSONLogicalWeights.frame) - var stack = [Frame(target: value.objectValue == nil ? .object(JSONObject(admitted: [])) : value, patch: root, returnKey: nil)] + var stack = [Frame(target: value.objectValue == nil ? .object(JSONObject(empty: ())) : value, patch: root, returnKey: nil)] while !stack.isEmpty { // Retire replaced immutable targets and operation-local scratch. The active frames // still own their targets; re-admit those snapshots without refunding charged work. @@ -78,7 +78,7 @@ public struct JSONMergePatch: Sendable { } } else if case .object(let nested) = member.value { try meter.require(.maxDepth, UInt64(stack.count) + 1); try meter.retain(JSONLogicalWeights.frame) - let target: JSON = existing?.objectValue == nil ? .object(JSONObject(admitted: [])) : existing! + let target: JSON = existing?.objectValue == nil ? .object(JSONObject(empty: ())) : existing! stack.append(Frame(target: target, patch: nested, returnKey: member.key)) } else { try meter.resetValueBounds(); try JSONAdmission.value(member.value, meter: &meter, owned: true) diff --git a/Sources/JSONSchema/Document/JSONSchemaBuiltins.swift b/Sources/JSONSchema/Document/JSONSchemaBuiltins.swift index 3b1153a..e35e698 100644 --- a/Sources/JSONSchema/Document/JSONSchemaBuiltins.swift +++ b/Sources/JSONSchema/Document/JSONSchemaBuiltins.swift @@ -32,7 +32,7 @@ extension JSONSchemaRegistry { for (path, identifier, size, expectedHash) in files { try meter.checkpoint(); try meter.charge(.schemaDocuments); try meter.resetValueBounds() try meter.require(.inputBytes, size); try meter.charge(.schemaBytes, size); try meter.retain(size) - let url = root.appendingPathComponent("Resources/Draft2020-12").appendingPathComponent(path) + let url = root.appendingPathComponent("Draft2020-12").appendingPathComponent(path) let handle: FileHandle do { handle = try FileHandle(forReadingFrom: url) } catch { throw JSONSchemaCompilationFailure(.metaSchema) } diff --git a/Sources/JSONSchema/Inspection/JSONSchemaNodeView.swift b/Sources/JSONSchema/Inspection/JSONSchemaNodeView.swift index c3ca178..20e68b0 100644 --- a/Sources/JSONSchema/Inspection/JSONSchemaNodeView.swift +++ b/Sources/JSONSchema/Inspection/JSONSchemaNodeView.swift @@ -50,21 +50,21 @@ public struct JSONSchemaNodeView: Sendable { try meter.work(); try meter.retain(JSONLogicalWeights.arraySlot); keywords.append(member.key) for name in annotationNames { if try JSONComparison.textEquals(member.key.string, name, meter: &meter) { - try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot); annotations.append(member); break + try meter.retain(JSONLogicalWeights.member); annotations.append(member); break } } if try JSONSchemaPlan.Keyword.classify(member.key,meter: &meter) == nil { - try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot); annotations.append(member) + try meter.retain(JSONLogicalWeights.member); annotations.append(member) } for name in constraintNames { if try JSONComparison.textEquals(member.key.string, name, meter: &meter) { - try meter.retain(JSONLogicalWeights.member + JSONLogicalWeights.indexSlot); constraints.append(member); break + try meter.retain(JSONLogicalWeights.member); constraints.append(member); break } } } } return .init(raw: raw, documentPointer: pointer, retrievalURI: retrievalURI, declaredKeywords: keywords, - declaredAnnotations: .init(values: JSONObject(admitted: annotations)), declaredConstraints: JSONObject(admitted: constraints), resourceURI: nil, resourcePointer: nil, referenceTargets: nil) + declaredAnnotations: .init(values: try JSONObject.indexed(admitted: annotations, meter: &meter)), declaredConstraints: try JSONObject.indexed(admitted: constraints, meter: &meter), resourceURI: nil, resourcePointer: nil, referenceTargets: nil) } } extension JSONSchemaDocument { diff --git a/Sources/JSONSchema/Output/JSONValidationOutputAdapter.swift b/Sources/JSONSchema/Output/JSONValidationOutputAdapter.swift index d2c7aea..4f0b6f8 100644 --- a/Sources/JSONSchema/Output/JSONValidationOutputAdapter.swift +++ b/Sources/JSONSchema/Output/JSONValidationOutputAdapter.swift @@ -46,12 +46,12 @@ private enum JSONValidationOutputAdapter { try meter.work(); try meter.require(.objectMembers,UInt64(members.count)+1) for _ in key.utf8 { try meter.work(); try meter.charge(.diagnosticBytes) } try JSONAdmission.text(key,key: true,meter: &meter,owned: true) - try meter.retain(JSONLogicalWeights.member+JSONLogicalWeights.indexSlot) + try meter.retain(JSONLogicalWeights.member) members.append(.init(try JSONKey(key,meter: &meter),value: value)) } static func object(_ members: [JSONMember],depth: UInt64,meter: inout JSONWorkMeter) throws -> JSON { try meter.require(.maxDepth,depth); try meter.charge(.nodes); try meter.retain(JSONLogicalWeights.node) - return .object(JSONObject(admitted: members)) + return .object(try JSONObject.indexed(admitted: members, meter: &meter)) } static func array(_ values: [JSON],meter: inout JSONWorkMeter) throws -> JSON { try meter.require(.maxDepth,2); try meter.charge(.nodes); try meter.retain(JSONLogicalWeights.node); return .array(values) diff --git a/Tests/JSONPatchTests/AcceptanceWaveTests.swift b/Tests/JSONPatchTests/AcceptanceWaveTests.swift new file mode 100644 index 0000000..3bf3ad8 --- /dev/null +++ b/Tests/JSONPatchTests/AcceptanceWaveTests.swift @@ -0,0 +1,38 @@ +// SPDX-License-Identifier: Apache-2.0 +import Testing +import JSON +import JSONPatch + +@Test func qaWidePatchEditsPreserveExactKeyOrderThroughIndexTransitions() throws { + // The two scalar-distinct keys make width 30 an actual 32-member object. + // Moving a member removes it before adding, exercising 32 -> 31 -> 32. + for width in [29, 30, 31, 32, 33, 61, 62, 63, 64, 65] { + let members = (0.. 0) + } + // Invalid evaluation must not mutate the reused immutable plan. + #expect(try JSONSchemaValidator().validate(value, against: plan).isValid) + } +} + +@Test func qaRepeatedPatternSearchesShareAggregateQuotaAndRecoverOnNewInvocation() throws { + let schema = try JSONParser().parse(#"{"patternProperties":{"^k[0-9]+$":true}}"#) + let plan = try JSONSchemaCompiler().compile(JSONSchemaDocument(schema)) + let instance = JSON.object(try JSONObject(members: (0..<64).map { .init("k\($0)", value: .null) })) + let report = try JSONSchemaValidator().validateWithReport(instance, against: plan) + #expect(report.result.isValid && report.statistics[.aggregateRegexWorkUnits] > 0) + var limits = JSONLimits.balanced + limits[.aggregateRegexWorkUnits] = report.statistics[.aggregateRegexWorkUnits] + #expect(try JSONSchemaValidator(options: .init(limits: limits)).validate(instance, against: plan).isValid) + limits[.aggregateRegexWorkUnits] -= 1 + do { _ = try JSONSchemaValidator(options: .init(limits: limits)).validate(instance, against: plan); Issue.record("Exhaustion became a verdict") } + catch let failure as JSONFailure { #expect(failure.category == .resource && failure.resource?.metric == .aggregateRegexWorkUnits) } + #expect(try JSONSchemaValidator().validate(instance, against: plan).isValid) +} diff --git a/Tests/JSONSchemaTests/IndexedObjectTests.swift b/Tests/JSONSchemaTests/IndexedObjectTests.swift new file mode 100644 index 0000000..984eb94 --- /dev/null +++ b/Tests/JSONSchemaTests/IndexedObjectTests.swift @@ -0,0 +1,16 @@ +// SPDX-License-Identifier: Apache-2.0 +import Testing +import JSON +import JSONSchema + +@Test func wideIndexedInstancesPreserveRequiredDependenciesAndExactKeys() throws { + let members = (0..<128).map { JSONMember("field-\($0)", value: .null) } + [JSONMember("é", value: 1), JSONMember("e\u{301}", value: 2)] + let value = JSON.object(try JSONObject(members: members)) + let schema = try JSONParser().parse(#"{"type":"object","required":["field-127","é","e\u0301"],"properties":{"field-127":{"type":"null"},"é":{"const":1},"e\u0301":{"const":2}},"dependentRequired":{"field-126":["field-127"]}}"#) + let plan = try JSONSchemaCompiler().compile(JSONSchemaDocument(schema)) + #expect(try JSONSchemaValidator().validate(value, against: plan).isValid) + let missing = JSON.object(try value.objectValue!.removing("field-127")) + #expect(try !JSONSchemaValidator().validate(missing, against: plan).isValid) + let changed = JSON.object(try value.objectValue!.setting(2, for: "é")) + #expect(try !JSONSchemaValidator().validate(changed, against: plan).isValid) +} diff --git a/Tests/JSONStreamingTests/AcceptanceWaveTests.swift b/Tests/JSONStreamingTests/AcceptanceWaveTests.swift new file mode 100644 index 0000000..e231a37 --- /dev/null +++ b/Tests/JSONStreamingTests/AcceptanceWaveTests.swift @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import Testing +import JSON +import JSONStreaming + +@Test func qaWideStreamEverySplitPreservesPayloadLexemesAndScalarKeys() async throws { + let fields = (0..<33).map { "\"k\($0)\":\($0)" }.joined(separator: ",") + let text = "{\(fields),\"é\":9007199254740993,\"é\":-0.00e+2,\"🚀\":0.1}" + let expected = Data(text.utf8) + for split in 0...text.utf8.count { + var iterator = JSONValueSequence(Chunks(text + " \r\n", split: split)).makeAsyncIterator() + let value = try #require(await iterator.next()) + let restored = try JSONCodablePayload(preserving: value).value() + #expect(try JSONWriter(options: .init(numberStyle: .preserved)).write(restored) == expected) + #expect(try restored.read { try $0["é"].requiredInteger(as: UInt64.self) } == 9_007_199_254_740_993) + #expect(try await iterator.next() == nil) + } +} + +@Test func qaWideDuplicateFailureHasStableLocationAtEverySplit() async throws { + let fields = (0..<33).map { "\"k\($0)\":null" }.joined(separator: ",") + let prefix = "[{\(fields)," + let text = prefix + "\"k0\" ???}]" + for split in 0...text.utf8.count { + var iterator = JSONTopLevelArraySequence(Chunks(text, split: split)).makeAsyncIterator() + do { + while let event = try await iterator.next() { + if case .documentCompleted = event { Issue.record("Malformed document completed") } + } + Issue.record("Duplicate accepted") + } catch let failure as JSONFailure { + #expect(failure.code == .duplicateKey) + #expect(failure.location?.byteOffset == UInt64(prefix.utf8.count)) + } + #expect(try await iterator.next() == nil) + } +} + +private actor QACancellingSink { + var accepted: [UInt8] = [] + func accept(_ chunk: JSONByteChunk) { + accepted.append(contentsOf: chunk.bytes) + if accepted.count == 3 { withUnsafeCurrentTask { $0?.cancel() } } + } +} +@Test func qaSinkCancellationAfterAcceptanceKeepsPrefixAndStopsHandoffs() async throws { + let value = try JSONParser().parse(#"{"é":[9007199254740993,0.1,null]}"#) + let expected = Array(try JSONWriter().write(value)) + let task = Task { + let sink = QACancellingSink() + do { + try await JSONAsyncWriter(chunkSize: 1).write(value) { chunk in + await sink.accept(chunk) + } + Issue.record("Cancelled writer completed") + } catch is CancellationError { } + #expect(await sink.accepted == Array(expected.prefix(3))) + } + try await task.value +} diff --git a/Tests/JSONTests/Release1_1/AcceptanceWaveTests.swift b/Tests/JSONTests/Release1_1/AcceptanceWaveTests.swift new file mode 100644 index 0000000..af6284a --- /dev/null +++ b/Tests/JSONTests/Release1_1/AcceptanceWaveTests.swift @@ -0,0 +1,269 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import Testing +@testable import JSON + +private func qaPreserved(_ value: JSON) throws -> Data { + try JSONWriter(options: .init(numberStyle: .preserved)).write(value) +} + +// Fixed seed; the complete compact token is the independent fidelity oracle. +private func qaDocuments() -> [String] { + var seed: UInt64 = 0x534A_1100_A11C_E001 + return (0..<192).map { index in + seed = seed &* 6_364_136_223_846_793_005 &+ 1_442_695_040_888_963_407 + let exact = "9007199254740993\(seed % 1000)" + let fields = (0..<(32 + Int(seed % 5))).map { "\"k\($0)\":\($0)" }.joined(separator: ",") + switch index % 6 { + case 0: return "null" + case 1: return index % 12 == 1 ? "true" : "false" + case 2: return index % 12 == 2 ? "-0.00e+2" : exact + case 3: return "\"é-é-🚀-\(seed % 1000)\"" + case 4: return "[\(exact),0.1,-0.00e+2,null,[],{},\"é\"]" + default: return "{\(fields),\"é\":\(exact),\"é\":0.1,\"/~/\":null,\"\":false}" + } + } +} + +@Test func qaGeneratedExactRoundTripsAcrossNativeAndForeignPayloads() throws { + let documents = qaDocuments() + #expect(documents.count == 192) + for (index, compact) in documents.enumerated() { + let expected = Data(compact.utf8) + let root = try JSONParser().parse(" \r\n" + compact + "\t ") + #expect(try qaPreserved(root) == expected, "seed case \(index)") + let projected = try root.read { try $0.decode(JSON.self) } + #expect(try qaPreserved(projected) == expected) + let payload = try JSONCodablePayload(preserving: root) + let native = try JSONValueEncoder().encode(payload) + let nativeRestored = try native.decode(JSONCodablePayload.self) + let jsonRestored = try JSONDecoder().decode(JSONCodablePayload.self, from: JSONEncoder().encode(payload)) + let plistRestored = try PropertyListDecoder().decode(JSONCodablePayload.self, from: PropertyListEncoder().encode(payload)) + for restored in [nativeRestored, jsonRestored, plistRestored] { + #expect(try qaPreserved(restored.value()) == expected, "seed case \(index)") + } + } +} + +@Test func qaIndexThresholdEditsAgreeWithOrderedScalarOracle() throws { + // Four special keys follow the generated fields. Include actual member + // counts 7/8/9, 31/32/33, 63/64/65 and 127/128/129, not just field widths. + for width in [0, 1, 3, 4, 5, 7, 8, 15, 16, 27, 28, 29, 31, 32, 33, 59, 60, 61, 63, 64, 65, 123, 124, 125, 127, 128, 257] { + let keys = (0.. startingWork, "failed work must not be refunded") + } +} +@Test func qaCaughtMalformedDataReleasesScratchWithoutRefundingWork() throws { + for malformed in ["AB==", "AAB=", "AAAA!", "AAAAAA==x"] { + _ = try JSON.string(malformed).decode(QARecoveringData.self) + } +} + +@Test func qaReaderTighteningPersistsAfterOrdinaryProjectionFailure() throws { + let root = try JSONParser().parse(#"{"history":"0123456789","n":1,"bad":"wrong"}"#) + let admission = try root.readWithReport { _ in 0 } + var coding = JSONCodingOptions() + coding.limits[.workUnits] = admission.statistics[.workUnits] + 1000 + #expect(throws: JSONFailure.self) { + try root.read { cursor in + do { _ = try cursor["bad"].decode(Int.self, coding: coding) } catch is DecodingError { } + for _ in 0..<1000 { _ = try cursor["n"].requiredInteger(as: Int.self) } + } + } + coding = .init(); coding.limits[.stringBytes] = 9 + #expect(throws: JSONFailure.self) { + try root.read { cursor in + do { _ = try cursor["n"].decode(Int.self, coding: coding) } catch {} + return 0 // Restriction checks admitted history, including unselected text. + } + } +} + +private enum QACallerError: Error { case expected } +private struct QAThrowingModel: Decodable { + init(from decoder: any Decoder) throws { + _ = try decoder.singleValueContainer().decode(Double.self) + throw QACallerError.expected + } +} +@Test func qaCaughtCallerProjectionReleasesOwnedTemporaries() throws { + let number = try JSONParser().parse("0.125") + func attempt(_ cursor: JSONReadCursor) throws { + do { _ = try cursor.decode(QAThrowingModel.self); Issue.record("Caller error disappeared") } + catch QACallerError.expected { } + } + let one = try number.readWithReport { cursor in + try attempt(cursor) + #expect(try cursor.requiredDouble(policy: .exact) == 0.125) + } + var limits = JSONLimits.balanced; limits[.retainedBytes] = one.statistics.peakLogicalRetainedBytes + let repeated = try number.readWithReport(limits: limits) { cursor in + for _ in 0..<100 { try attempt(cursor) } + #expect(try cursor.requiredDouble(policy: .exact) == 0.125) + } + #expect(repeated.statistics[.retainedBytes] == 0) + #expect(repeated.statistics[.workUnits] > one.statistics[.workUnits]) +} + +@Test func qaBase64CleanupKeepsResourceAndCancellationTerminal() async throws { + let canonical = Data(repeating: 255, count: 256).base64EncodedString() + var limits = JSONLimits.balanced; limits[.retainedBytes] = 49 + var meter = try JSONWorkMeter(limits: limits, domain: .coding) + try meter.retain(17) // An enclosing owner's reservation must survive. + do { _ = try JSONBase64.decode(canonical, meter: &meter); Issue.record("Expected scratch refusal") } + catch let failure as JSONFailure { #expect(failure.resource?.metric == .retainedBytes) } + #expect(meter.used(.retainedBytes) == 17 && meter.used(.workUnits) > 0) + #expect(throws: JSONFailure.self) { try meter.checkpoint() } + try meter.release(17) + let cancelled = Task { + var options = JSONLimits.balanced; options[.cancellationQuantum] = 1 + var local = try JSONWorkMeter(limits: options, domain: .coding) + try local.retain(17) + local.checkpointObserver = { work in + if work >= 20 { withUnsafeCurrentTask { $0?.cancel() } } + } + do { _ = try JSONBase64.decode(canonical, meter: &local); Issue.record("Cancellation disappeared") } + catch is CancellationError { } + #expect(local.used(.retainedBytes) == 17 && local.used(.workUnits) >= 20) + #expect(throws: CancellationError.self) { try local.checkpoint() } + try local.release(17) + } + try await cancelled.value +} + +@Test func qaPayloadCreateAndOpenInclusiveWorkRetentionMatrix() throws { + for compact in ["null", "0.1", "-0.00e+2", "[]", "{}", #"{"é":9007199254740993,"é":[null,false]}"#] { + let value = try JSONParser().parse(compact) + let made = try JSONCodablePayload.preservingWithReport(value) + let opened = try made.result.valueWithReport() + for (operation, statistics) in [(0, made.statistics), (1, opened.statistics)] { + for metric in [JSONResourceMetric.workUnits, .retainedBytes] { + let boundary = metric == .retainedBytes ? statistics.peakLogicalRetainedBytes : statistics[metric] + var limits = JSONLimits.balanced; limits[metric] = boundary + if operation == 0 { _ = try JSONCodablePayload(preserving: value, limits: limits) } + else { #expect(try qaPreserved(made.result.value(limits: limits)) == Data(compact.utf8)) } + limits[metric] = boundary - 1 + do { + if operation == 0 { _ = try JSONCodablePayload(preserving: value, limits: limits) } + else { _ = try made.result.value(limits: limits) } + Issue.record("Missing inclusive quota refusal") + } catch let failure as JSONFailure { + #expect(failure.category == .resource && failure.resource?.metric == metric) + } + } + } + } +} + +@Test func qaMappedScalarPathsUseUTF8ByteCoordinates() throws { + let text = "{\n \"é\":{\"/~/\":[null,\"wrong\"]},\n \"é\":false\n}" + let document = try JSONParser().parseDocument(Data(text.utf8), source: "qa") + let prefix = "{\n \"é\":{\"/~/\":[null," + do { _ = try document.read { try $0["é"]["/~/"][1].requiredInteger(as: Int.self) }; Issue.record("Missing mapped failure") } + catch let error as JSONAccessFailure { + #expect(error.path == [.key("é"), .key("/~/"), .index(1)]) + #expect(error.cause.location?.byteOffset == UInt64(prefix.utf8.count)) + #expect(error.cause.location?.line == 2) + #expect(error.cause.location?.column == UInt64(prefix.split(separator: "\n").last!.utf8.count + 1)) + } + #expect(try document.read { try $0["e\u{301}"].requiredBoolean() } == false) +} + +@Test func qaSharedPayloadAndRootHaveIndependentConcurrentReadScopes() async throws { + let root = try JSONParser().parse(#"{"é":9007199254740993,"é":0.1,"a":[null,false]}"#) + let payload = try JSONCodablePayload(preserving: root) + let expected = try qaPreserved(root) + try await withThrowingTaskGroup(of: Void.self) { group in + for _ in 0..<16 { + group.addTask { + for _ in 0..<20 { + #expect(try qaPreserved(payload.value()) == expected) + let report = try root.readWithReport { cursor in + #expect(try cursor["é"].requiredInteger(as: UInt64.self) == 9_007_199_254_740_993) + return try cursor["e\u{301}"].requiredDouble(policy: .rounded) + } + #expect(report.result == 0.1 && report.statistics[.retainedBytes] == 0) + } + } + } + try await group.waitForAll() + } +} diff --git a/Tests/JSONTests/Release1_1/IndexPrototype.swift b/Tests/JSONTests/Release1_1/IndexPrototype.swift new file mode 100644 index 0000000..252c4f0 --- /dev/null +++ b/Tests/JSONTests/Release1_1/IndexPrototype.swift @@ -0,0 +1,188 @@ +// SPDX-License-Identifier: Apache-2.0 +// Controlled index candidates for qualification. No production publication path. +@testable import JSON + +struct ProbeKey { + let bytes: [UInt8] + let fingerprint: UInt64 + let position: Int + init(_ text: String, position: Int, collision: Bool = false, meter: inout JSONWorkMeter) throws { + var bytes: [UInt8] = [], fingerprint: UInt64 = 14695981039346656037 + for byte in text.utf8 { + try meter.work(); try meter.require(.keyBytes, UInt64(bytes.count) + 1) + try meter.retain(1); bytes.append(byte) + fingerprint = (fingerprint ^ UInt64(byte)) &* 1099511628211 + } + self.bytes = bytes; self.fingerprint = collision ? 0 : fingerprint; self.position = position + } + func compare(_ other: Self, meter: inout JSONWorkMeter) throws -> Int { + try meter.work() + if fingerprint != other.fingerprint { return fingerprint < other.fingerprint ? -1 : 1 } + return try compareBytes(other, meter: &meter) + } + func compareBytes(_ other: Self, meter: inout JSONWorkMeter) throws -> Int { + for index in 0.. Int? { + for previous in keys { + try meter.work() + if try previous.compareBytes(key, meter: &meter) == 0 { return previous.position } + } + return nil + } +} + +protocol ProbeIndex { + init() + mutating func insert(_ key: ProbeKey, meter: inout JSONWorkMeter) throws + func find(_ key: ProbeKey, meter: inout JSONWorkMeter) throws -> Int? +} + +struct ProbeLinearIndex: ProbeIndex { + private var keys: [ProbeKey] = [] + init() {} + mutating func insert(_ key: ProbeKey, meter: inout JSONWorkMeter) throws { + guard try find(key, meter: &meter) == nil else { throw JSONFailure(.syntax, code: .duplicateKey) } + try meter.require(.objectMembers, UInt64(keys.count) + 1) + try meter.retain(JSONLogicalWeights.indexSlot); keys.append(key) + } + func find(_ key: ProbeKey, meter: inout JSONWorkMeter) throws -> Int? { + for previous in keys { + try meter.work() + if try previous.compare(key, meter: &meter) == 0 { return previous.position } + } + return nil + } +} + +struct ProbeAVLIndex: ProbeIndex { + private struct Node { let key: ProbeKey; var left = -1; var right = -1; var height = 1 } + private var nodes: [Node] = [] + private var root = -1 + private var slots = 0 + init() {} + private func height(_ index: Int) -> Int { index < 0 ? 0 : nodes[index].height } + private mutating func update(_ index: Int) { nodes[index].height = 1 + max(height(nodes[index].left), height(nodes[index].right)) } + private mutating func rotateRight(_ index: Int, meter: inout JSONWorkMeter) throws -> Int { + try meter.work(4) + let child = nodes[index].left + nodes[index].left = nodes[child].right; nodes[child].right = index + update(index); update(child); return child + } + private mutating func rotateLeft(_ index: Int, meter: inout JSONWorkMeter) throws -> Int { + try meter.work(4) + let child = nodes[index].right + nodes[index].right = nodes[child].left; nodes[child].left = index + update(index); update(child); return child + } + private mutating func append(_ key: ProbeKey, meter: inout JSONWorkMeter) throws -> Int { + try meter.require(.objectMembers, UInt64(nodes.count) + 1) + if nodes.count == slots { + let (doubled, overflow) = slots.multipliedReportingOverflow(by: 2) + guard !overflow else { throw JSONFailure(.resource, code: .counterOverflow) } + let next = min(Int(meter.limits[.objectMembers]), max(1, doubled)) + let (bytes, byteOverflow) = UInt64(next).multipliedReportingOverflow(by: JSONLogicalWeights.indexSlot) + guard !byteOverflow else { throw JSONFailure(.resource, code: .counterOverflow) } + // Admit the new allocation while the old capacity still coexists. + try meter.retain(bytes); try meter.work(UInt64(nodes.count)); nodes.reserveCapacity(next) + try meter.release(UInt64(slots) * JSONLogicalWeights.indexSlot); slots = next + } + try meter.work(); nodes.append(Node(key: key)); return nodes.count - 1 + } + private mutating func insert(_ key: ProbeKey, at index: Int, meter: inout JSONWorkMeter) throws -> Int { + if index < 0 { return try append(key, meter: &meter) } + try meter.retain(JSONLogicalWeights.frame) + let comparison = try key.compare(nodes[index].key, meter: &meter) + if comparison == 0 { throw JSONFailure(.syntax, code: .duplicateKey) } + if comparison < 0 { nodes[index].left = try insert(key, at: nodes[index].left, meter: &meter) } + else { nodes[index].right = try insert(key, at: nodes[index].right, meter: &meter) } + try meter.work(); update(index) + let balance = height(nodes[index].left) - height(nodes[index].right) + var result = index + if balance > 1 { + let left = nodes[index].left + if height(nodes[left].left) < height(nodes[left].right) { nodes[index].left = try rotateLeft(left, meter: &meter) } + result = try rotateRight(index, meter: &meter) + } else if balance < -1 { + let right = nodes[index].right + if height(nodes[right].right) < height(nodes[right].left) { nodes[index].right = try rotateRight(right, meter: &meter) } + result = try rotateLeft(index, meter: &meter) + } + try meter.release(JSONLogicalWeights.frame) + return result + } + mutating func insert(_ key: ProbeKey, meter: inout JSONWorkMeter) throws { root = try insert(key, at: root, meter: &meter) } + func find(_ key: ProbeKey, meter: inout JSONWorkMeter) throws -> Int? { + var index = root + while index >= 0 { + try meter.work() + let comparison = try key.compare(nodes[index].key, meter: &meter) + if comparison == 0 { return nodes[index].key.position } + index = comparison < 0 ? nodes[index].left : nodes[index].right + } + return nil + } +} + +struct ProbeHashIndex: ProbeIndex { + private var slots: [ProbeKey?] = [] + private var count = 0 + init() {} + private func position(_ key: ProbeKey, in table: [ProbeKey?], meter: inout JSONWorkMeter) throws -> (Int, Bool) { + var position = Int(key.fingerprint & UInt64(table.count - 1)) + for _ in table.indices { + try meter.work() + guard let previous = table[position] else { return (position, false) } + if try previous.compare(key, meter: &meter) == 0 { return (position, true) } + position = (position + 1) & (table.count - 1) + } + throw JSONFailure(.configuration, code: .invalidOperation) + } + private mutating func grow(meter: inout JSONWorkMeter) throws { + let (capacity, overflow) = max(1, slots.count).multipliedReportingOverflow(by: 2) + guard !overflow else { throw JSONFailure(.resource, code: .counterOverflow) } + let (bytes, byteOverflow) = UInt64(capacity).multipliedReportingOverflow(by: JSONLogicalWeights.indexSlot) + guard !byteOverflow else { throw JSONFailure(.resource, code: .counterOverflow) } + try meter.retain(bytes) + var replacement: [ProbeKey?] = [] + replacement.reserveCapacity(capacity) + for _ in 0..= slots.count / 2 { try grow(meter: &meter) } + let (index, found) = try position(key, in: slots, meter: &meter) + guard !found else { throw JSONFailure(.syntax, code: .duplicateKey) } + slots[index] = key; count += 1 + } + func find(_ key: ProbeKey, meter: inout JSONWorkMeter) throws -> Int? { + guard !slots.isEmpty else { return nil } + let (index, found) = try position(key, in: slots, meter: &meter) + return found ? slots[index]?.position : nil + } +} diff --git a/Tests/JSONTests/Release1_1/IndexPublicationTests.swift b/Tests/JSONTests/Release1_1/IndexPublicationTests.swift new file mode 100644 index 0000000..70f1c2b --- /dev/null +++ b/Tests/JSONTests/Release1_1/IndexPublicationTests.swift @@ -0,0 +1,106 @@ +// SPDX-License-Identifier: Apache-2.0 +import Testing +@testable import JSON + +private struct WideFields: Encodable { + let keys: [String] + func encode(to encoder: any Encoder) throws { + var container = encoder.container(keyedBy: JSONCodingKey.self) + for (position, key) in keys.enumerated() { try container.encode(position, forKey: JSONCodingKey(key)) } + // Replacing keeps the first key at its insertion position. + try container.encode(0, forKey: JSONCodingKey(keys[0])) + } +} + +@Test func indexedPublicationCoversBuildersParserNativeCodingAndEditing() throws { + let keys = (0..<129).map { "field-\($0)" } + ["\u{e9}", "e\u{301}", "", "/", "~"] + let members = keys.enumerated().map { JSONMember($0.element, value: .number(JSONNumber(integer: $0.offset))) } + let checked = try JSONObject(members: members) + let built = try JSONObject.build { builder in + for member in members { try builder.append(member.key.string, value: member.value) } + } + let parsed = try JSONParser().parse(JSONWriter().write(.object(checked))).objectValue! + let coded = try JSONValueEncoder().encode(WideFields(keys: keys)).objectValue! + let edited = try checked.setting(.boolean(true), for: "replacement").removing("replacement") + for object in [checked, built, parsed, coded, edited] { + #expect(object.members.map(\.key) == checked.members.map(\.key)) + for member in members { + #expect(object[member.key] == member.value) + #expect(try JSON.object(object).requiredMember(member.key.string) == member.value) + } + #expect(object["absent"] == nil) + } +} + +@Test func controlledIndexCollisionsAndRefusedRehashPreservePriorMetadata() throws { + var index = JSONExactKeyIndex(fingerprintMask: 0) + var meter = try JSONWorkMeter(limits: .offline, domain: .core) + for position in 0..<257 { + #expect(try index.insert(JSONKey("collision-\(position)"), position: position, meter: &meter)) + } + for position in 0..<257 { #expect(try index.find(JSONKey("collision-\(position)"), meter: &meter) == position) } + #expect(try !index.insert(JSONKey("collision-1"), position: 999, meter: &meter)) + #expect(try index.find(JSONKey("absent"), meter: &meter) == nil) + var bounded = JSONExactKeyIndex() + var limits = JSONLimits.balanced; limits[.retainedBytes] = 32 * 48 + var limited = try JSONWorkMeter(limits: limits, domain: .core) + for position in 0..<31 { #expect(try bounded.insert(JSONKey("key-\(position)"), position: position, meter: &limited)) } + #expect(throws: JSONFailure.self) { try bounded.insert(JSONKey("key-31"), position: 31, meter: &limited) } + for position in 0..<31 { #expect(bounded.find(JSONKey("key-\(position)")) == position) } + #expect(bounded.find(JSONKey("key-31")) == nil) +} + +@Test func indexedDuplicateRecognitionRetainsSecondKeyLocationAndNonthrowingFinish() throws { + let fields = (0..<64).map { "\"k\($0)\":null" }.joined(separator: ",") + let prefix = "{" + fields + "," + // The value/colon after the duplicate is malformed; duplicate still wins. + let malformed = prefix + "\"k0\" ???" + do { + _ = try JSONParser().parse(malformed) + Issue.record("Duplicate was not recognized") + } catch let error as JSONFailure { + #expect(error.code == .duplicateKey) + #expect(error.location?.byteOffset == UInt64(prefix.utf8.count)) + } + var builder = try JSONObjectAccumulator() + for position in 0..<64 { try builder.append("k\(position)", value: .null) } + #expect(throws: JSONFailure.self) { try builder.append("k0", value: .boolean(true)) } + let completed = builder.finish() + #expect(completed.count == 64) + #expect(completed["k0"] == .null) +} + +@Test func indexedGrowthAndCollisionScansCheckCancellation() async throws { + let task = Task.detached { () throws -> Bool in + var meter = try JSONWorkMeter(limits: .offline, domain: .core) + var index = JSONExactKeyIndex(fingerprintMask: 0) + for position in 0..<64 { _ = try index.insert(JSONKey("key-\(position)"), position: position, meter: &meter) } + let starting = meter.statistics()[.workUnits] + meter.checkpointObserver = { work in + if work > starting { withUnsafeCurrentTask { $0?.cancel() } } + } + do { + for position in 64..<1000 { _ = try index.insert(JSONKey("key-\(position)"), position: position, meter: &meter) } + return false + } catch is CancellationError { + // Prior keys remain valid after an interrupted owned search/growth. + return index.find(JSONKey("key-0")) == 0 && index.find(JSONKey("key-63")) == 63 + } + } + #expect(try await task.value) +} + +@Test func immutableIndexedObjectCanBeSharedAcrossTasks() async throws { + let members = (0..<256).map { JSONMember("key-\($0)", value: .number(JSONNumber(integer: $0))) } + let value = JSON.object(try JSONObject(members: members)) + try await withThrowingTaskGroup(of: Void.self) { group in + for position in 0..<32 { + group.addTask { + for _ in 0..<10 { + let selected = try value.requiredMember("key-\(position * 7)"); #expect(selected == .number(JSONNumber(integer: position * 7))) + } + } + } + try await group.waitForAll() + } +} diff --git a/Tests/JSONTests/Release1_1/LookupIncrementTests.swift b/Tests/JSONTests/Release1_1/LookupIncrementTests.swift new file mode 100644 index 0000000..dcfe860 --- /dev/null +++ b/Tests/JSONTests/Release1_1/LookupIncrementTests.swift @@ -0,0 +1,61 @@ +// SPDX-License-Identifier: Apache-2.0 +import Testing +@testable import JSON + +@Test func wideLookupPreservesExactKeysOrderAndCollisionConfirmation() throws { + let prefix = String(repeating: "p", count: 64) + let padding = (0..<32).map { JSONMember(prefix + String($0), value: .boolean(false)) } + let special = ["", "0", "/", "~", "a.b", "\u{e9}", "e\u{301}"] + let members = padding + special.map { JSONMember($0, value: .string($0)) } + [JSONMember("nullable", value: .null)] + let object = try JSONObject(members: members) + let value = JSON.object(object) + for key in special { + let selected = try value.requiredMember(key) + #expect(selected.stringValue?.utf8.elementsEqual(key.utf8) == true) + #expect(try JSONPointer(tokens: [key]).resolve(in: value) == selected) + } + #expect(try value.requiredMember("nullable") == .null) + #expect(try value.presence(of: "absent") == .absent) + #expect(try value.presence(of: "nullable") == .null) + #expect(object.members.map(\.key) == members.map(\.key)) + #expect(try JSONComparison.equals(value, .object(JSONObject(members: members.reversed())))) + // Force a fingerprint hit at the production candidate comparator. Distinct + // decoded scalar spellings must still fail full UTF-8 confirmation. + let composed = JSONKey("\u{e9}") + var meter = try JSONWorkMeter(limits: .balanced, domain: .core) + #expect(try !composed.matches("e\u{301}", fingerprint: composed.fingerprint, meter: &meter)) + #expect(try composed.matches("\u{e9}", fingerprint: composed.fingerprint, meter: &meter)) +} + +@Test func wideLookupEnforcesInclusiveCumulativeWorkAndInnerCancellation() async throws { + let prefix = String(repeating: "p", count: 128) + let members = (0..<256).map { JSONMember(prefix + String($0), value: .boolean(true)) } + let object = try JSONObject(members: members, limits: .offline) + let key = members.last!.key.string + let value = JSON.object(object) + var meter = try JSONWorkMeter(limits: .offline, domain: .core) + try JSONAdmission.text(key, key: true, meter: &meter) + #expect(try object.lookup(key, meter: &meter) == .boolean(true)) + let inclusive = meter.statistics()[.workUnits] + var limits = JSONLimits.offline; limits[.workUnits] = inclusive + #expect(try value.requiredMember(key, limits: limits) == .boolean(true)) + limits[.workUnits] = inclusive - 1 + do { + _ = try value.requiredMember(key, limits: limits) + Issue.record("Lookup ignored its inclusive work ceiling") + } catch let failure as JSONFailure { #expect(failure.resource?.metric == .workUnits) } + var shared = try JSONWorkMeter(limits: limits, domain: .core) + #expect(try object.lookup(key, meter: &shared) == .boolean(true)) + #expect(throws: JSONFailure.self) { try object.lookup(key, meter: &shared) } + let task = Task.detached { () throws -> UInt64 in + var meter = try JSONWorkMeter(limits: .offline, domain: .core) + meter.checkpointObserver = { work in + if work >= 1400 { withUnsafeCurrentTask { $0?.cancel() } } + } + do { + for _ in 0..<1000 { _ = try object.lookup(key, meter: &meter) } + return 0 + } catch is CancellationError { return meter.statistics()[.workUnits] } + } + #expect(try await task.value >= 1400) +} diff --git a/Tests/JSONTests/Release1_1/MilestoneTests.swift b/Tests/JSONTests/Release1_1/MilestoneTests.swift new file mode 100644 index 0000000..c098d63 --- /dev/null +++ b/Tests/JSONTests/Release1_1/MilestoneTests.swift @@ -0,0 +1,159 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import Testing +@testable import JSON + +@Test func readerProbeSharesNavigationConversionAndProjectionMeter() throws { + struct Model: Decodable { let count: UInt64 } + let value = try JSONParser().parse(#"{"users":[{"name":"Ada","empty":null,"model":{"count":9007199254740993}}]}"#) + let report = try probeReadWithReport(value) { root in + let user = try root["users"][0] + #expect(try user[JSONKey("name")].requiredString() == "Ada") + guard case .absent = try user.presence(of: "missing"), + case .null = try user.presence(of: "empty"), + case .value(let present) = try user.presence(of: "name") else { + Issue.record("Presence conflated missing, null or value"); return UInt64(0) + } + #expect(try present.requiredString() == "Ada") + let number = try user["model"]["count"].requiredInteger(as: UInt64.self) + let before = try #require(user.scope.state).meter.statistics()[.workUnits] + let identity = ObjectIdentifier(try #require(user.scope.state)) + let model = try user["model"].decode(Model.self) + #expect(ObjectIdentifier(try #require(user.scope.state)) == identity) + #expect(try #require(user.scope.state).meter.statistics()[.workUnits] > before) + #expect(model.count == number && number == 9_007_199_254_740_993) + return number + } + #expect(report.statistics.domain == .core) + #expect(report.statistics[.numericWorkUnits] > 0) + // Same scope that succeeds once cannot reset its meter for each repeated read. + var limits = JSONLimits.balanced; limits[.workUnits] = report.statistics[.workUnits] + #expect(throws: JSONFailure.self) { + try probeReadWithReport(value, limits: limits) { root in + for _ in 0..<10 { _ = try root["users"][0]["model"].decode(Model.self) } + } + } +} + +@Test func readerProbeInvalidatesEscapesAndReleasesRootAndSource() throws { + let document = try JSONParser().parseDocument(Data(#"{"name":"Ada"}"#.utf8), source: "probe.json") + let scope = try ProbeReadScope(document, limits: .balanced) + weak let state = scope.state + let cursor = ProbeReadCursor(scope: scope, path: []) + #expect(try cursor["name"].requiredString() == "Ada") + scope.invalidate() + #expect(scope.document == nil && scope.state == nil && state == nil) + #expect(throws: JSONFailure.self) { try cursor["name"].requiredString() } + let escaped = try probeReadWithReport(document.root) { try $0["name"] }.result + #expect(escaped.scope.document == nil && escaped.scope.state == nil) + #expect(throws: JSONFailure.self) { try escaped.requiredString() } + var failedEscape: ProbeReadCursor? + enum CallerError: Error { case failed } + #expect(throws: CallerError.self) { + try probeReadWithReport(document.root) { root in + failedEscape = root + throw CallerError.failed + } + } + #expect(try #require(failedEscape).scope.document == nil) +} + +@Test func readerProbeLatchesCaughtResourceFailures() throws { + let value = try JSONParser().parse(#"{"name":"Ada"}"#) + var limits = JSONLimits.balanced; limits[.pointerTokens] = 0 + #expect(throws: JSONFailure.self) { + try probeReadWithReport(value, limits: limits) { root in + do { _ = try root["name"] } catch let failure as JSONFailure { + #expect(failure.resource?.metric == .pointerTokens) + } + return "caught" + } + } + let report = try probeReadWithReport(value) { root in + do { _ = try root["missing"] } catch let failure as ProbeAccessFailure { + #expect(failure.path == [.key("missing")]) + #expect(failure.cause.code == .missingMember) + } + return try root["name"].requiredString() + } + #expect(report.result == "Ada") +} + +@Test func readerProbeLatchesCaughtInnerCancellation() async throws { + let value = try JSONParser().parse(#"{"name":"Ada"}"#) + let task = Task.detached { () throws -> Bool in + do { + _ = try probeReadWithReport(value) { root in + let state = try #require(root.scope.state) + let before = state.meter.statistics()[.workUnits] + state.meter.checkpointObserver = { work in + if work > before { withUnsafeCurrentTask { $0?.cancel() } } + } + do { _ = try root["name"].requiredString() } catch is CancellationError {} + return true + } + return false + } catch is CancellationError { return true } + } + #expect(try await task.value) +} + +private func indexIdentity(_ type: I.Type) throws { + var meter = try JSONWorkMeter(limits: .offline, domain: .core) + var index = I() + let strings = ["", "0", "/", "~", "a.b", "\u{e9}", "e\u{301}"] + (0..<512).map { "prefix-\($0)" } + for (position, text) in strings.enumerated() { + try index.insert(ProbeKey(text, position: position, collision: true, meter: &meter), meter: &meter) + } + for (position, text) in strings.enumerated() { + #expect(try index.find(ProbeKey(text, position: -1, collision: true, meter: &meter), meter: &meter) == position) + } + #expect(try index.find(ProbeKey("absent", position: -1, collision: true, meter: &meter), meter: &meter) == nil) + #expect(throws: JSONFailure.self) { + try index.insert(ProbeKey("e\u{301}", position: -1, collision: true, meter: &meter), meter: &meter) + } +} +@Test func indexProbesConfirmExactIdentityDespiteSyntheticCollisions() throws { + try indexIdentity(ProbeScalarIndex.self); try indexIdentity(ProbeLinearIndex.self) + try indexIdentity(ProbeAVLIndex.self); try indexIdentity(ProbeHashIndex.self) +} + +@Test func indexProbesEnforceGrowthWorkAndInnerCancellation() async throws { + for budget: JSONResourceMetric in [.workUnits, .retainedBytes] { + var limits = JSONLimits.balanced; limits[budget] = 1 + var meter = try JSONWorkMeter(limits: limits, domain: .core) + var index = ProbeHashIndex() + #expect(throws: JSONFailure.self) { + try index.insert(ProbeKey("", position: 0, meter: &meter), meter: &meter) + } + } + let task = Task.detached { () throws -> UInt64 in + var meter = try JSONWorkMeter(limits: .offline, domain: .core) + var index = ProbeAVLIndex() + for position in 0..<512 { + try index.insert(ProbeKey("key\(position)", position: position, meter: &meter), meter: &meter) + } + let query = try ProbeKey("key511", position: -1, meter: &meter) + let before = meter.statistics()[.workUnits] + meter.checkpointObserver = { work in + if work > before { withUnsafeCurrentTask { $0?.cancel() } } + } + do { + for _ in 0..<10000 { _ = try index.find(query, meter: &meter) } + return 0 + } catch is CancellationError { return meter.statistics()[.workUnits] } + } + #expect(try await task.value > 0) +} + +@Test func duplicateRecognitionPrecedesValueAndClosingSyntax() throws { + // Second decoded key is complete, but neither its colon nor value has arrived. + let input = Array(#"{"a":0,"\u0061""#.utf8) + do { + _ = try JSONParser().parse(input) + Issue.record("Duplicate input unexpectedly parsed") + } catch let failure as JSONFailure { + #expect(failure.code == .duplicateKey) + #expect(failure.location?.byteOffset == 7) + } +} diff --git a/Tests/JSONTests/Release1_1/PayloadTests.swift b/Tests/JSONTests/Release1_1/PayloadTests.swift new file mode 100644 index 0000000..476c272 --- /dev/null +++ b/Tests/JSONTests/Release1_1/PayloadTests.swift @@ -0,0 +1,153 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import CryptoKit +import Testing +@testable import JSON + +private struct PayloadFixtures: Decodable { + struct Fixture: Decodable { + struct Envelope: Codable { let format: String; let version: Int; let utf8Base64: String } + let id: String; let inner_utf8_sha256: String; let inner_json: String; let envelope: Envelope + } + let cases: [Fixture] +} +private func payloadFixtures() throws -> PayloadFixtures { + var root = URL(fileURLWithPath: #filePath) + for _ in 0..<4 { root.deleteLastPathComponent() } + return try JSONDecoder().decode(PayloadFixtures.self, from: Data(contentsOf: root.appendingPathComponent("Tools/Release1_1Probes/Fixtures/version1.json"))) +} +private func preserved(_ value: JSON) throws -> Data { try JSONWriter(options: .init(numberStyle: .preserved)).write(value) } +private struct PayloadPair: Codable { let a: JSONCodablePayload; let b: JSONCodablePayload } + +@Test func payloadVersionOneGoldenFixturesPreserveScalarsOrderAndLexemes() throws { + let fixtures = try payloadFixtures().cases + #expect(fixtures.count == 8) + for fixture in fixtures { + let bytes = Data(fixture.inner_json.utf8) + #expect(SHA256.hash(data: bytes).map { String(format: "%02x", $0) }.joined() == fixture.inner_utf8_sha256) + let root = try JSONParser().parse(bytes) + let payload = try JSONCodablePayload(preserving: root) + let encoded = try JSONEncoder().encode(payload) + let fields = try JSONDecoder().decode(PayloadFixtures.Fixture.Envelope.self, from: encoded) + #expect(fields.format == "swift-json-value" && fields.version == 1) + #expect(fields.utf8Base64 == fixture.envelope.utf8Base64) + let decoded = try JSONDecoder().decode(JSONCodablePayload.self, from: JSONEncoder().encode(fixture.envelope)) + let opened = try decoded.value() + #expect(try preserved(opened) == bytes) + #expect(try preserved(JSONDecoder().decode(JSONCodablePayload.self, from: encoded).value()) == bytes) + } + for text in ["[]", "{}"] { + let payload = try JSONCodablePayload(preserving: JSONParser().parse(text)) + #expect(try preserved(JSONDecoder().decode(JSONCodablePayload.self, from: JSONEncoder().encode(payload)).value()) == Data(text.utf8)) + } +} +@Test func payloadForeignPropertyListAndNestedNativeCodingRemainExact() throws { + let root = try JSONParser().parse(#"{"b":-0.00e+2,"é":9007199254740993,"é":0.1,"":null}"#) + let payload = try JSONCodablePayload(preserving: root) + let pair = PayloadPair(a: payload, b: payload) + let plist = try PropertyListEncoder().encode(pair) + let reread = try PropertyListDecoder().decode(PayloadPair.self, from: plist) + #expect(try preserved(reread.a.value()) == preserved(root)) + #expect(try preserved(reread.b.value()) == preserved(root)) + let native = try JSONValueEncoder().encode(pair) + let decoded = try native.decode(PayloadPair.self) + #expect(try preserved(decoded.a.value()) == preserved(root)) + #expect(try preserved(decoded.b.value()) == preserved(root)) +} +@Test func payloadWrongFieldsAndUnsupportedVersionsUseCodableErrors() throws { + for text in [#"{}"#, #"{"format":null,"version":1,"utf8Base64":"bnVsbA=="}"#, + #"{"format":3,"version":1,"utf8Base64":"bnVsbA=="}"#, + #"{"format":"swift-json-value","version":"1","utf8Base64":"bnVsbA=="}"#, + #"{"format":"other","version":1,"utf8Base64":"bnVsbA=="}"#, + #"{"format":"swift-json-value","version":2,"utf8Base64":"bnVsbA=="}"#] { + #expect(throws: DecodingError.self) { try JSONDecoder().decode(JSONCodablePayload.self, from: Data(text.utf8)) } + } + let extra = #"{"format":"swift-json-value","version":1,"utf8Base64":"bnVsbA==","metadata":true}"# + #expect(try JSONDecoder().decode(JSONCodablePayload.self, from: Data(extra.utf8)).value().isNull) +} +@Test func payloadRejectsMalformedBase64AndCompleteInnerGrammar() throws { + var texts = ["", "A", "bnVsbA=", "bnVsbA===", "bnVs bA==", "bnVsbB==", "AB==", "AAB=", "____", "AA==AA=="] + let invalidBytes: [[UInt8]] = [[0xFF], Array("null true".utf8), Array("01".utf8), Array(#"{"é":0,"\u00e9":1}"#.utf8), Array(#""\uD800""#.utf8), Array("[1,]".utf8)] + texts += invalidBytes.map { Data($0).base64EncodedString() } // Fixture creation only. + for text in texts { + let fields = PayloadFixtures.Fixture.Envelope(format: "swift-json-value", version: 1, utf8Base64: text) + #expect(throws: DecodingError.self) { try JSONDecoder().decode(JSONCodablePayload.self, from: JSONEncoder().encode(fields)) } + } + // The shared native Data codec also requires canonical unused padding bits. + for text in ["AB==", "AAB="] { #expect(throws: DecodingError.self) { try JSONValueDecoder().decode(Data.self, from: .string(text)) } } + let spaced = PayloadFixtures.Fixture.Envelope(format: "swift-json-value", version: 1, utf8Base64: Data(" null \n".utf8).base64EncodedString()) + #expect(try JSONDecoder().decode(JSONCodablePayload.self, from: JSONEncoder().encode(spaced)).value().isNull) +} +@Test func payloadInclusiveOwnedBudgetsAndEnvelopeExpansionAreChecked() throws { + let root = try JSONParser().parse("0.1") + let made = try JSONCodablePayload.preservingWithReport(root) + let work = made.statistics[.workUnits] + var limits = JSONLimits.balanced; limits[.workUnits] = work + #expect(try JSONCodablePayload.preservingWithReport(root, limits: limits).statistics[.workUnits] == work) + limits[.workUnits] = work - 1 + #expect(throws: JSONFailure.self) { try JSONCodablePayload(preserving: root, limits: limits) } + limits = .balanced; limits[.stringBytes] = 3 // Inner 0.1 fits; MC4x does not. + #expect(throws: JSONFailure.self) { try JSONCodablePayload(preserving: root, limits: limits) } + limits = .balanced; limits[.outputBytes] = 4 // Inner fits; fixed envelope does not. + #expect(throws: JSONFailure.self) { try JSONCodablePayload(preserving: root, limits: limits) } + let opened = try made.result.valueWithReport() + limits = .balanced; limits[.retainedBytes] = opened.statistics.peakLogicalRetainedBytes + #expect(try preserved(made.result.value(limits: limits)) == preserved(root)) + limits[.retainedBytes] -= 1 + #expect(throws: JSONFailure.self) { try made.result.value(limits: limits) } + #expect(try made.result.valueWithReport(limits: .offline).statistics[.workUnits] == opened.statistics[.workUnits]) +} +@Test func payloadNativeCompositionUsesOneCumulativeMeter() throws { + let payload = try JSONCodablePayload(preserving: JSONParser().parse("[0.1,9007199254740993,null]")) + let single = try JSONValueEncoder().encodeWithReport(payload) + let pair = PayloadPair(a: payload, b: payload) + let both = try JSONValueEncoder().encodeWithReport(pair) + #expect(both.statistics[.workUnits] > single.statistics[.workUnits]) + var limits = JSONLimits.balanced; limits[.workUnits] = single.statistics[.workUnits] + #expect(throws: JSONFailure.self) { try JSONValueEncoder(options: .init(limits: limits)).encode(pair) } + let opened = try JSONValueDecoder().decodeWithReport(JSONCodablePayload.self, from: single.result) + limits = .balanced; limits[.workUnits] = opened.statistics[.workUnits] - 1 + #expect(throws: JSONFailure.self) { try JSONValueDecoder(options: .init(limits: limits)).decode(JSONCodablePayload.self, from: single.result) } + #expect(opened.statistics[.nodes] > 3) +} +private struct CatchingPayload: Decodable { + init(from decoder: any Decoder) throws { + do { _ = try JSONCodablePayload(from: decoder) } catch { /* Deliberate caller misuse. */ } + } +} +private struct InterruptingPayload: Encodable { + let payload: JSONCodablePayload + func encode(to encoder: any Encoder) throws { + let native = try #require(encoder as? JSONNativeEncoder) + let threshold = native.state.meter.used(.workUnits) + 100 + native.state.meter.checkpointObserver = { work in + if work >= threshold { withUnsafeCurrentTask { $0?.cancel() } } + } + try payload.encode(to: encoder) + } +} +@Test func payloadCaughtOperationalFailureStaysTerminalAndCorruptionDoesNot() throws { + let payload = try JSONCodablePayload(preserving: JSONParser().parse("[0.1,9007199254740993,null]")) + let envelope = try JSONValueEncoder().encode(payload) + let report = try JSONValueDecoder().decodeWithReport(JSONCodablePayload.self, from: envelope) + var limits = JSONLimits.balanced; limits[.workUnits] = report.statistics[.workUnits] - 1 + #expect(throws: JSONFailure.self) { try JSONValueDecoder(options: .init(limits: limits)).decode(CatchingPayload.self, from: envelope) } + let corrupt = try JSONParser().parse(#"{"format":"swift-json-value","version":2,"utf8Base64":"bnVsbA=="}"#) + _ = try corrupt.decode(CatchingPayload.self) // Corruption can be handled explicitly. + var meter = try JSONWorkMeter(limits: limits, domain: .coding) + try meter.work(limits[.workUnits]) + #expect(throws: JSONFailure.self) { try meter.work() } + let terminal = meter.statistics() + #expect(throws: JSONFailure.self) { try meter.checkpoint() } + #expect(meter.statistics()[.workUnits] == terminal[.workUnits]) +} +@Test func payloadCancellationDoesNotReturnAValidatedResult() async throws { + let payload = try JSONCodablePayload(preserving: .string(String(repeating: "é", count: 1024))) + let task = Task { + withUnsafeCurrentTask { $0?.cancel() } + return try payload.valueWithReport() + } + await #expect(throws: CancellationError.self) { try await task.value } + let inOwnedLoop = Task { try JSONValueEncoder().encode(InterruptingPayload(payload: payload)) } + await #expect(throws: CancellationError.self) { try await inOwnedLoop.value } +} diff --git a/Tests/JSONTests/Release1_1/ReadAccessTests.swift b/Tests/JSONTests/Release1_1/ReadAccessTests.swift new file mode 100644 index 0000000..777e934 --- /dev/null +++ b/Tests/JSONTests/Release1_1/ReadAccessTests.swift @@ -0,0 +1,255 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import Testing +@testable import JSON + +private struct ReadModel: Codable, Equatable { let count: UInt64; let name: String; let bytes: Data } +private struct MeterWitness: Decodable { + let identity: ObjectIdentifier + let work: UInt64 + init(from decoder: any Decoder) throws { + let native = try #require(decoder as? JSONNativeDecoder) + identity = ObjectIdentifier(native.state); work = native.state.meter.statistics()[.workUnits] + } +} +private struct UserInfoWitness: Decodable { + let marker: String + init(from decoder: any Decoder) throws { + marker = try #require(decoder.userInfo[CodingUserInfoKey(rawValue: "marker")!] as? String) + } +} +private enum ReadCallerFailure: Error { case expected } +private struct RefusingDecode: Decodable { init(from decoder: any Decoder) throws { throw ReadCallerFailure.expected } } +private struct CatchingOwnedFailure: Decodable { + init(from decoder: any Decoder) throws { + let c = try decoder.singleValueContainer() + do { _ = try c.decode(UInt64.self) } catch {} + } +} +private func expectAccess(_ path: [JSONLocationComponent], _ code: JSONFailure.Code, _ body: () throws -> Void) { + do { try body(); Issue.record("Expected a typed access failure") } + catch let error as JSONAccessFailure { #expect(error.path == path); #expect(error.cause.code == code) } + catch { Issue.record("Unexpected error: \(type(of: error))") } +} + +@Test func readAccessChainedBracketsAndRequiredKinds() throws { + let value = try JSONParser().parse(#"{"users":[{"name":"Ada","age":9007199254740993}],"0":false,"empty":"","zero":0,"false":false,"null":null}"#) + try value.read { root in + let user = try root[JSONKey("users")][0] + #expect(try user["name"].requiredString() == "Ada") + #expect(try user["age"].requiredInteger(as: UInt64.self) == 9_007_199_254_740_993) + #expect(try root["0"].requiredBoolean() == false) + #expect(try root["users"].requiredArray().count == 1) + #expect(try user.requiredObject().count == 2) + #expect(try root["null"].requiredValue() == .null) + #expect(try root.stringPresence(of: "empty") == .value("")) + #expect(try root.numberPresence(of: "zero") == .value(JSONNumber(integer: 0))) + #expect(try root.booleanPresence(of: "false") == .value(false)) + #expect(try root.presence(of: JSONKey("missing")) == .absent) + #expect(try root.stringPresence(of: JSONKey("null")) == .null) + } +} + +@Test func readAccessExactKeysAndFirstFailingHop() throws { + let keys = ["", "0", "/", "~", "a.b", "\u{e9}", "e\u{301}", "🚀"] + let value = JSON.object(try JSONObject(members: keys.map { .init($0, value: .string($0)) })) + try value.read { root in + for key in keys { let text = try root[key].requiredString(); #expect(text.utf8.elementsEqual(key.utf8)) } + expectAccess([.key("missing")], .missingMember) { _ = try root["missing"]["later"] } + expectAccess([.index(0)], .wrongKind) { _ = try root[0] } + expectAccess([.key("0"), .key("later")], .wrongKind) { _ = try root["0"]["later"] } + expectAccess([.key("0")], .wrongKind) { _ = try root["0"].requiredBoolean() } + } + try JSON.array([.null]).read { root in + expectAccess([.index(-1)], .invalidIndex) { _ = try root[-1] } + expectAccess([.index(1)], .missingTarget) { _ = try root[1] } + expectAccess([.key("0")], .wrongKind) { _ = try root["0"] } + } +} + +@Test func readAccessExactNumericPoliciesAndBounds() throws { + for text in ["9007199254740993", "18446744073709551615"] { + let value = try JSONParser().parse(text) + #expect(try value.read { try $0.requiredInteger(as: UInt64.self) } == UInt64(text)) + } + #expect(try JSONParser().parse("-9223372036854775808").read { try $0.requiredInteger(as: Int64.self) } == Int64.min) + for text in ["1.5", "18446744073709551616", "-1"] { + let value = try JSONParser().parse(text) + expectAccess([], .inexactConversion) { _ = try value.read { try $0.requiredInteger(as: UInt64.self) } } + } + let tenth = try JSONParser().parse("0.1") + expectAccess([], .inexactConversion) { _ = try tenth.read { try $0.requiredDouble(policy: .exact) } } + #expect(try tenth.read { try $0.requiredDouble(policy: .rounded) } == 0.1) + #expect(try tenth.read { try $0.requiredDecimal(policy: .exact) } == Decimal(string: "0.1")) + let huge = try JSONParser().parse("1e100000") + expectAccess([], .inexactConversion) { _ = try huge.read { try $0.requiredDouble(policy: .rounded) } } +} + +@Test func readAccessMappedDiagnosticsAndReplacedRootCoherence() throws { + let bytes = Data("{\n \"user\": {\"age\": \"wrong\"}\n}".utf8) + let document = try JSONParser().parseDocument(bytes, source: "fixture") + do { + _ = try document.read { try $0["user"]["age"].requiredInteger(as: Int.self) } + Issue.record("Expected mapped kind failure") + } catch let error as JSONAccessFailure { + #expect(error.path == [.key("user"), .key("age")]) + #expect(error.cause.location?.byteOffset == 20) + #expect(error.cause.location?.line == 2) + #expect(error.cause.location?.column == 19) + #expect(JSONDiagnostic(error).failure == error.cause) + } + do { _ = try document.read { try $0["user"]["age"]["child"].requiredValue() } } + catch let error as JSONAccessFailure { #expect(error.cause.location?.byteOffset == 20) } + do { _ = try document.read { try $0["user"]["missing"].requiredValue() } } + catch let error as JSONAccessFailure { #expect(error.cause.location?.byteOffset == 12) } + do { _ = try document.replacingRoot(document.root).read { try $0["user"]["age"].requiredBoolean() } } + catch let error as JSONAccessFailure { #expect(error.cause.location == nil) } +} + +@Test func readAccessSharedProjectionLimitsAndTerminalAccounting() throws { + var history = JSONCodingOptions(); history.limits[.stringBytes] = 6 + #expect(throws: JSONFailure.self) { try JSON.string("history").read { try $0.decode(String.self, coding: history) } } + var huge = JSONLimits.balanced; huge[.inputBytes] = UInt64(Int.max) - 1 + var lifetime = try JSONWorkMeter(limits: huge, domain: .core) + for _ in 0..<2 { try lifetime.charge(.inputBytes, huge[.inputBytes]); try lifetime.resetValueBounds() } + #expect(throws: JSONFailure.self) { try lifetime.charge(.inputBytes, huge[.inputBytes]) } + let terminal = lifetime.statistics()[.inputBytes] + #expect(throws: JSONFailure.self) { try lifetime.checkpoint() } + #expect(lifetime.statistics()[.inputBytes] == terminal) + let model = ReadModel(count: .max, name: "e\u{301}", bytes: Data([0, 1, 255])) + let value = try JSON(encoding: model) + #expect(try value.decode(ReadModel.self) == model) + #expect(try value.read { try $0.decode(ReadModel.self) } == model) + let baseline = try value.readWithReport { _ in 0 } + let decoded = try value.readWithReport { try $0.decode(ReadModel.self) } + #expect(decoded.statistics.domain == .core) + #expect(decoded.statistics[.workUnits] > baseline.statistics[.workUnits]) + #expect(decoded.statistics[.numericWorkUnits] > baseline.statistics[.numericWorkUnits]) + try value.read { root in + let first = try root.decode(MeterWitness.self) + let second = try root["count"].decode(MeterWitness.self) + #expect(first.identity == second.identity && second.work > first.work) + let info = JSONCodingOptions(userInfo: [CodingUserInfoKey(rawValue: "marker")!: "qualified"]) + #expect(try root.decode(UserInfoWitness.self, coding: info).marker == "qualified") + } + var coding = JSONCodingOptions(); coding.limits[.workUnits] = baseline.statistics[.workUnits] - 1 + #expect(throws: JSONFailure.self) { + try value.read { root in + do { _ = try root.decode(ReadModel.self, coding: coding) } catch {} + return 0 + } + } + var limits = JSONLimits.balanced; limits[.workUnits] = decoded.statistics[.workUnits] + #expect(try value.read(limits: limits) { try $0.decode(ReadModel.self) } == model) + #expect(throws: JSONFailure.self) { + try value.read(limits: limits) { root in + _ = try root.decode(ReadModel.self) + return try root.decode(ReadModel.self) + } + } + #expect(throws: ReadCallerFailure.self) { try value.read { try $0.decode(RefusingDecode.self) } } + let text = try JSONParser().parse("18446744073709551615") + let admitted = try text.readWithReport { _ in 0 } + var tight = JSONCodingOptions(); tight.limits[.workUnits] = admitted.statistics[.workUnits] + 2 + #expect(throws: JSONFailure.self) { try text.read { try $0.decode(CatchingOwnedFailure.self, coding: tight) } } + #expect(throws: JSONFailure.self) { try text.decode(CatchingOwnedFailure.self, coding: tight) } +} + +@Test func readAccessPresenceAvoidsCallerDecodeAndPreservesNativeFidelity() throws { + let value = try JSONParser().parse(#"{"null":null,"wrong":"no","good":9007199254740993,"é":1,"e\u0301":2}"#) + try value.read { root in + switch try root.presence(of: "missing", as: RefusingDecode.self) { case .absent: break; default: Issue.record("Missing decoded") } + switch try root.presence(of: JSONKey("null"), as: RefusingDecode.self) { case .null: break; default: Issue.record("Null decoded") } + #expect(try root.presence(of: "good", as: UInt64.self) == .value(9_007_199_254_740_993)) + #expect(throws: DecodingError.self) { try root.presence(of: "wrong", as: UInt64.self) } + #expect(throws: ReadCallerFailure.self) { try root.presence(of: "good", as: RefusingDecode.self) } + #expect(try root.decode(JSON.self) == value) + #expect(throws: DecodingError.self) { try root.decode([String: JSON].self) } + } + #expect(try value.decode(JSON.self) == value) + #expect(try JSONParser().parse("9007199254740993").decode(JSONNumber.self) == JSONNumber(parsing: "9007199254740993")) + #expect(try value.decode(JSONObject.self).count == 5) +} + +@Test func readAccessEscapedHandlesInvalidateAndReleaseRootAndSource() throws { + let document = try JSONParser().parseDocument(Data(#"{"child":{"name":"value"}}"#.utf8)) + let escaped = try document.read { try $0["child"] } + #expect(throws: JSONFailure.self) { try escaped.requiredValue() } + let scope = try JSONReadScope(document, limits: .balanced) + weak let state = scope.state + let cursor = JSONReadCursor(scope: scope, selection: JSONReadPath(scope: scope, components: [], bytes: 0)) + _ = try scope.complete(try cursor["child"]) + #expect(scope.document == nil && scope.state == nil && state == nil) + #expect(throws: JSONFailure.self) { try cursor.requiredObject() } +} + +@Test func readAccessPathCopiesReleaseLiveRetentionAndFailCompletePaths() throws { + let value = try JSONParser().parse(#"{"child":{"name":"value"}}"#) + func attempt(_ root: JSONReadCursor) throws { + do { + _ = try root["child"]["name"].requiredBoolean() + Issue.record("Expected ordinary kind failure") + } catch let error as JSONAccessFailure { #expect(error.path == [.key("child"), .key("name")]) } + } + let one = try value.readWithReport { root in try attempt(root) } + var limits = JSONLimits.balanced; limits[.retainedBytes] = one.statistics.peakLogicalRetainedBytes + let repeated = try value.readWithReport(limits: limits) { root in + for _ in 0..<100 { try attempt(root) } + } + #expect(repeated.statistics.peakLogicalRetainedBytes == one.statistics.peakLogicalRetainedBytes) + #expect(repeated.statistics[.retainedBytes] == 0) + #expect(repeated.statistics[.workUnits] > one.statistics[.workUnits]) + let oneText = try value.readWithReport { try $0["child"]["name"].requiredString() } + limits = .balanced; limits[.retainedBytes] = oneText.statistics.peakLogicalRetainedBytes + let repeatedText = try value.readWithReport(limits: limits) { root in + for _ in 0..<100 { _ = try root["child"]["name"].requiredString() } + } + #expect(repeatedText.statistics[.retainedBytes] == 0) + #expect(repeatedText.statistics.peakLogicalRetainedBytes == oneText.statistics.peakLogicalRetainedBytes) + limits = .balanced; limits[.pointerBytes] = 5 + #expect(throws: JSONFailure.self) { try value.read(limits: limits) { try $0["child"] } } + limits = .balanced; limits[.diagnosticBytes] = 5 + #expect(throws: JSONFailure.self) { try value.read(limits: limits) { try $0["child"] } } + +} + +@Test func readAccessCaughtCancellationCannotCertifyCompletion() async throws { + let task = Task.detached { () throws -> Bool in + let value = try JSONParser().parse(#"{"name":"value"}"#) + do { + _ = try value.read { root in + withUnsafeCurrentTask { $0?.cancel() } + do { _ = try root["name"].requiredString() } catch is CancellationError {} + return 0 + } + return false + } catch is CancellationError { return true } + } + #expect(try await task.value) +} + +@Test func readAccessBridgeStrategiesAndForeignNaturalCodingRemainDistinct() throws { + let value = ReadModel(count: .max, name: "sensor", bytes: Data([1, 2, 255])) + for strategy in [JSONCodingOptions.DataStrategy.base64, .deferredToData] { + let coding = JSONCodingOptions(dataStrategy: strategy) + let encoded = try JSON(encoding: value, coding: coding) + #expect(try encoded == JSONValueEncoder(options: coding).encode(value)) + #expect(try encoded.decode(ReadModel.self, coding: coding) == value) + } + for strategy in [JSONCodingOptions.DateStrategy.deferredToDate, .secondsSince1970, .iso8601] { + let coding = JSONCodingOptions(dateStrategy: strategy) + let date = Date(timeIntervalSince1970: 1_600_000_000.25) + let encoded = try JSON(encoding: date, coding: coding) + let decoded = try encoded.decode(Date.self, coding: coding) + let reference = try JSONValueDecoder(options: coding).decode(Date.self, from: encoded) + #expect(decoded == reference) + #expect(decoded == date) + } + for text in ["-9223372036854775808", "18446744073709551615", "0.1"] { + let number = try JSONNumber(parsing: text) + let bytes = try Foundation.JSONEncoder().encode(number) + #expect(try Foundation.JSONDecoder().decode(JSONNumber.self, from: bytes) == number) + } + let huge = try JSONNumber(parsing: "1e100000") + #expect(throws: EncodingError.self) { try Foundation.JSONEncoder().encode(huge) } +} diff --git a/Tests/JSONTests/Release1_1/ReaderPrototype.swift b/Tests/JSONTests/Release1_1/ReaderPrototype.swift new file mode 100644 index 0000000..ad391a5 --- /dev/null +++ b/Tests/JSONTests/Release1_1/ReaderPrototype.swift @@ -0,0 +1,179 @@ +// SPDX-License-Identifier: Apache-2.0 +// Qualification prototype, compiled into tests or ReaderProbe, never into JSON. +@testable import JSON + +public struct ProbeAccessFailure: Error, Sendable { + public let cause: JSONFailure + public let path: [JSONLocationComponent] +} + +// Deliberately not Sendable: the synchronous invocation owns mutable state. +final class ProbeReadScope { + var document: JSONDocument? + var state: JSONCodingState? + private var terminal: (any Error)? + + init(_ document: JSONDocument, limits: JSONLimits) throws { + let state = JSONCodingState(options: .init(limits: limits), + meter: try JSONWorkMeter(limits: limits, domain: .core)) + try JSONAdmission.value(document.root, meter: &state.meter, owned: true) + self.document = document + self.state = state + } + + func invalidate() { document = nil; state = nil } + + func check() throws -> JSONCodingState { + if let terminal { throw terminal } + guard document != nil, let state else { + throw JSONFailure(.configuration, code: .invalidOperation) + } + try state.check() + return state + } + + func run(_ path: [JSONLocationComponent], _ body: (JSONCodingState) throws -> T) throws -> T { + do { return try body(check()) } + catch { + if error is CancellationError || (error as? JSONFailure)?.category == .resource { + if terminal == nil { terminal = error } + throw error + } + if let failure = error as? JSONFailure, + failure.category != .configuration || failure.code == .invalidIndex { + throw ProbeAccessFailure(cause: failure, path: path) + } + throw error + } + } + + func select(_ path: [JSONLocationComponent], state: JSONCodingState) throws -> JSON { + guard let document else { throw JSONFailure(.configuration, code: .invalidOperation) } + var value = document.root + for part in path { + try state.meter.work() + switch part { + case .key(let key): + guard case .object(let object) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + guard let child = try object.lookup(key.string, meter: &state.meter) else { + throw JSONFailure(.missingValue, code: .missingMember) + } + value = child + case .index(let index): + guard case .array(let array) = value else { throw JSONFailure(.typeMismatch, code: .wrongKind) } + guard index >= 0 else { throw JSONFailure(.configuration, code: .invalidIndex) } + guard index < array.count else { throw JSONFailure(.missingValue, code: .missingTarget) } + value = array[index] + } + } + return value + } + + func append(_ part: JSONLocationComponent, to path: [JSONLocationComponent], state: JSONCodingState) throws -> [JSONLocationComponent] { + try state.meter.require(.pointerTokens, UInt64(path.count) + 1) + var bytes: UInt64 = 0 + // Charge equivalent Pointer text, complete typed path copies and owned slots. + for position in 0...path.count { + let component = position == path.count ? part : path[position] + bytes = try state.meter.adding(.pointerBytes, bytes, 1) + let text: String + switch component { + case .key(let key): text = key.string + case .index(let index): text = String(index) + } + try state.meter.work(); try state.meter.retain(JSONLogicalWeights.indexSlot) + for byte in text.utf8 { + try state.meter.work() + bytes = try state.meter.adding(.pointerBytes, bytes, byte == 47 || byte == 126 ? 2 : 1) + try state.meter.retain(1) + } + } + try state.meter.require(.diagnosticBytes, bytes) + var result: [JSONLocationComponent] = [] + for component in path { try state.meter.work(); result.append(component) } + result.append(part) + return result + } +} + +/// Experimental shape only. Handles own typed paths, never selected DOM payloads. +public struct ProbeReadCursor { + let scope: ProbeReadScope + let path: [JSONLocationComponent] + + public subscript(_ key: String) -> Self { + get throws { + try scope.run(path) { state in + try JSONAdmission.text(key, key: true, meter: &state.meter) + return try child(.key(JSONKey(key, meter: &state.meter)), state: state) + } + } + } + public subscript(_ key: JSONKey) -> Self { + get throws { + try scope.run(path) { state in + try JSONAdmission.text(key.string, key: true, meter: &state.meter) + return try child(.key(key), state: state) + } + } + } + public subscript(_ index: Int) -> Self { + get throws { try scope.run(path) { try child(.index(index), state: $0) } } + } + public func presence(of key: JSONKey) throws -> JSONPresence { + try scope.run(path) { state in + try JSONAdmission.text(key.string, key: true, meter: &state.meter) + let attempted = try scope.append(.key(key), to: path, state: state) + return try scope.run(attempted) { state in + guard case .object(let object) = try scope.select(path, state: state) else { + throw JSONFailure(.typeMismatch, code: .wrongKind) + } + guard let value = try object.lookup(key.string, meter: &state.meter) else { return .absent } + if case .null = value { return .null } + return .value(Self(scope: scope, path: attempted)) + } + } + } + private func child(_ part: JSONLocationComponent, state: JSONCodingState) throws -> Self { + let attempted = try scope.append(part, to: path, state: state) + // Failure includes the attempted hop. No cursor stores the selected value. + return try scope.run(attempted) { state in + _ = try scope.select(attempted, state: state) + return Self(scope: scope, path: attempted) + } + } + public func requiredString() throws -> String { + try scope.run(path) { state in + guard case .string(let text) = try scope.select(path, state: state) else { + throw JSONFailure(.typeMismatch, code: .wrongKind) + } + try JSONAdmission.text(text, meter: &state.meter, owned: true) + return text + } + } + public func requiredInteger(as type: T.Type) throws -> T { + try scope.run(path) { state in + guard case .number(let number) = try scope.select(path, state: state) else { + throw JSONFailure(.typeMismatch, code: .wrongKind) + } + return try number.integer(as: type, meter: &state.meter) + } + } + public func decode(_ type: T.Type) throws -> T { + try scope.run(path) { state in + let value = try scope.select(path, state: state) + return try JSONNativeDecoder(value: value, state: state, codingPath: []).decode(type) + } + } +} + +public func probeReadWithReport(_ value: JSON, limits: JSONLimits = .balanced, + _ body: (ProbeReadCursor) throws -> Result) throws -> JSONOperationReport { + let scope = try ProbeReadScope(JSONDocument(root: value), limits: limits) + defer { scope.invalidate() } + let result = try body(ProbeReadCursor(scope: scope, path: [])) + // Catching a resource or cancellation failure inside body cannot certify success. + return try scope.run([]) { state in + .init(result: result, statistics: state.meter.statistics()) + } +} diff --git a/Tools/MobileQualification/Package.swift b/Tools/MobileQualification/Package.swift new file mode 100644 index 0000000..126610d --- /dev/null +++ b/Tools/MobileQualification/Package.swift @@ -0,0 +1,14 @@ +// swift-tools-version: 6.4 +// SPDX-License-Identifier: Apache-2.0 +import PackageDescription +let package = Package(name: "MobileQualification", platforms: [.macOS("27.0"), .iOS("27.0")], + products: [.executable(name: "MobileQualification", targets: ["Runtime"])], + dependencies: [.package(name: "Swift-JSON", path: "../..")], + targets: [ + .target(name: "ProbeCases", dependencies: [ + .product(name: "JSON", package: "Swift-JSON"), .product(name: "JSONSchema", package: "Swift-JSON"), + .product(name: "JSONPatch", package: "Swift-JSON"), .product(name: "JSONStreaming", package: "Swift-JSON")], + resources: [.copy("Fixtures/version1.json")]), + .executableTarget(name: "Runtime", dependencies: ["ProbeCases"]), + .testTarget(name: "ProbeTests", dependencies: ["ProbeCases"]) + ], swiftLanguageModes: [.v6]) diff --git a/Tools/MobileQualification/README.md b/Tools/MobileQualification/README.md new file mode 100644 index 0000000..7185ac6 --- /dev/null +++ b/Tools/MobileQualification/README.md @@ -0,0 +1,32 @@ +# Public Apple runtime and SwiftData example + +This separate package imports all four public products and SwiftData. It adds no +SwiftData dependency to the root package. Native host requirements remain Swift +6.4/language 6/macOS and SDK 27; the mobile minima are iOS/iPadOS 27. + +```sh +swift test --package-path Tools/MobileQualification +swift run -c release --package-path Tools/MobileQualification MobileQualification +python3 Scripts/qualify_mobile_runtime.py --output .build/ci/mobile-runtime/receipt.json +``` + +Three Swift Testing functions check exact public APIs/fixtures, stream/sink +completion/cancellation and disk persistence. Eight frozen Version 1 fixtures +preserve scalar bytes, exact decimals beyond UInt64/Decimal, order and lexemes. +The model uses explicit nonoptional/optional `@Attribute(.codable)` payloads and +independent applicationSchemaVersion. Query/sort ordinary name fields; embedded +JSON predicates and cloud synchronization are not claimed. + +Each disk phase runs in a separate exited process: save, reopen/check/update, +reopen/check updated data. All use one disk URL. This ensures prior models, +contexts, containers and SDK-retained references are gone. A first simulator +release probe found SDK container retention beyond lexical scope; its weak-reference +failure remains evidence. Mac test-bundle layouts locate the actual built runtime +within five ancestor checks. The simulator controller creates/deletes only its own +isolated iPhone/iPad devices and records exact outcomes and source hashes. + +SDK compilation, simulator runtime and physical-device performance are distinct +claims. The [1.1 qualification policy](../../Documentation/Architecture/Release1.1/QualificationPolicy.json) +accepts iPad SDK/simulator behavior and disk persistence for this release. +Physical iPad checks are deferred and nonblocking; physical iPhone checks remain +required. No physical iPad performance, memory or energy claim follows. diff --git a/Tools/MobileQualification/Sources/ProbeCases/DiskPersistence.swift b/Tools/MobileQualification/Sources/ProbeCases/DiskPersistence.swift new file mode 100644 index 0000000..866de89 --- /dev/null +++ b/Tools/MobileQualification/Sources/ProbeCases/DiskPersistence.swift @@ -0,0 +1,93 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import SwiftData +import JSON + +@Model final class ConfigurationSnapshot { + var name: String + var applicationSchemaVersion: Int + @Attribute(.codable) var payload: JSONCodablePayload + @Attribute(.codable) var optionalPayload: JSONCodablePayload? + init(name: String, payload: JSONCodablePayload, optional: JSONCodablePayload?) { + self.name = name; applicationSchemaVersion = 1 + self.payload = payload; optionalPayload = optional + } +} +@MainActor private func container(_ url: URL) throws -> ModelContainer { + try ModelContainer(for: ConfigurationSnapshot.self, configurations: ModelConfiguration(url: url)) +} +@MainActor private func save(_ url: URL, fixtures: FrozenFixtures) throws { + try autoreleasepool { + let store = try container(url) + let context = ModelContext(store); context.autosaveEnabled = false + for item in fixtures.cases { + let payload = try JSONCodablePayload(preserving: JSONParser().parse(item.inner_json)) + context.insert(ConfigurationSnapshot(name: item.id, payload: payload, + optional: item.id == "null" ? try JSONCodablePayload(preserving: .null) : nil)) + } + try context.save() + } +} +@MainActor private func reopenAndUpdate(_ url: URL, fixtures: FrozenFixtures) throws { + try autoreleasepool { + let store = try container(url) + let context = ModelContext(store); context.autosaveEnabled = false + let rows = try context.fetch(FetchDescriptor(sortBy: [SortDescriptor(\.name)])) + try verify(rows.count == 8 && rows.map(\.name) == fixtures.cases.map(\.id).sorted(), "ordinary field sorting") + for row in rows { + let fixture = fixtures.cases.first { $0.id == row.name }! + try verify(exactBytes(row.payload.value()) == Data(fixture.inner_json.utf8), "disk fidelity including scalar/order/lexeme") + try verify(row.applicationSchemaVersion == 1, "application schema version") + if row.name == "null" { try verify(row.optionalPayload?.value().isNull == true, "present null attribute") } + else { try verify(row.optionalPayload == nil, "absent optional attribute") } + } + let selection = FetchDescriptor(predicate: #Predicate { $0.name == "decimal" }) + let decimal = try context.fetch(selection) + try verify(decimal.count == 1, "ordinary field predicate") + decimal[0].payload = try JSONCodablePayload(preserving: JSONParser().parse("0.1000e+1")) + decimal[0].applicationSchemaVersion = 2 + decimal[0].optionalPayload = try JSONCodablePayload(preserving: .null) + try context.save() + } +} +// Each phase is executed by a distinct process. Process exit closes the store and +// removes all model/context/container state, including SDK-retained references. +@MainActor public func diskPhase(_ phase: String, url: URL) throws { + let fixtures = try FrozenFixtures.load() + if phase == "save" { try save(url, fixtures: fixtures); return } + try verify(FileManager.default.fileExists(atPath: url.path), "actual disk store exists") + if phase == "reopen-update" { try reopenAndUpdate(url, fixtures: fixtures); return } + try verify(phase == "verify-update", "known disk phase") + try autoreleasepool { + let store = try container(url), context = ModelContext(store) + let rows = try context.fetch(FetchDescriptor(predicate: #Predicate { $0.name == "decimal" })) + try verify(rows.count == 1 && rows[0].applicationSchemaVersion == 2, "updated ordinary field persisted") + try verify(exactBytes(rows[0].payload.value()) == Data("0.1000e+1".utf8), "updated lexeme persisted") + try verify(rows[0].optionalPayload?.value().isNull == true, "updated present null persisted") + } +} +#if os(macOS) +public func diskPersistence() throws { + let directory = FileManager.default.temporaryDirectory.appendingPathComponent("swift-json-disk-" + UUID().uuidString) + try FileManager.default.createDirectory(at: directory, withIntermediateDirectories: true) + defer { try? FileManager.default.removeItem(at: directory) } + let url = directory.appendingPathComponent("snapshots.store") + var products = Bundle.module.bundleURL.deletingLastPathComponent() + for _ in 0..<5 { + if FileManager.default.isExecutableFile(atPath: products.appendingPathComponent("MobileQualification").path) { break } + products.deleteLastPathComponent() + } + let executable = products.appendingPathComponent("MobileQualification") + for phase in ["save", "reopen-update", "verify-update"] { + let process = Process(), pipe = Pipe() + process.executableURL = executable + process.arguments = ["--disk-phase", phase, url.path] + process.standardOutput = pipe; process.standardError = pipe + try process.run() + let output = pipe.fileHandleForReading.readDataToEndOfFile() + process.waitUntilExit() + try verify(process.terminationReason == .exit && process.terminationStatus == 0, "disk child process exit " + phase) + try verify(String(decoding: output, as: UTF8.self).split(separator: "\n").contains(Substring("DISK_PHASE passed phase=" + phase + " rows=8")), "disk child process outcome") + } +} +#endif diff --git a/Tools/MobileQualification/Sources/ProbeCases/Fixtures/version1.json b/Tools/MobileQualification/Sources/ProbeCases/Fixtures/version1.json new file mode 100644 index 0000000..ca0cd0c --- /dev/null +++ b/Tools/MobileQualification/Sources/ProbeCases/Fixtures/version1.json @@ -0,0 +1,86 @@ +{ + "format_version": 1, + "status": "logical-envelope-fixtures; exact persistence codec qualification not yet run", + "cases": [ + { + "id": "null", + "inner_utf8_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "inner_json": "null", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "bnVsbA==" + } + }, + { + "id": "boolean", + "inner_utf8_sha256": "b5bea41b6c623f7c09f1bf24dcae58ebab3c0cdd90ad966bc43a45b44867e12b", + "inner_json": "true", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "dHJ1ZQ==" + } + }, + { + "id": "string", + "inner_utf8_sha256": "06c3994e4804c1a17dfcc7607f72944152a706367c0c322b34fd9a8f1b1a455d", + "inner_json": "\"é é 🚀\"", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "IsOpIGXMgSDwn5qAIg==" + } + }, + { + "id": "decimal", + "inner_utf8_sha256": "14be4b45f18e0d8c67b4f719b5144eee88497e413709d11d85b096d8e2346310", + "inner_json": "0.1", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "MC4x" + } + }, + { + "id": "lexeme", + "inner_utf8_sha256": "59bbf72ab162059f732c213462a666646ff97896f9efd0b2ec53b97dda502472", + "inner_json": "-0.00e+2", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "LTAuMDBlKzI=" + } + }, + { + "id": "large-number", + "inner_utf8_sha256": "decc538c077786966ac863b5532c4027b8587ff40f6e3103379af62b44eae44d", + "inner_json": "123456789012345678901234567890123456789012345678901234567890", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkw" + } + }, + { + "id": "array", + "inner_utf8_sha256": "c742fdf61c92ed1d0081e93b41b69224909adffd4484ae4a023085d827ccb229", + "inner_json": "[1e0,null,\"x\"]", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "WzFlMCxudWxsLCJ4Il0=" + } + }, + { + "id": "ordered-scalars", + "inner_utf8_sha256": "57c900d6ddd444977ea738f05bf6c030f968423d21f1a1410384841af835b048", + "inner_json": "{\"b\":1.00,\"é\":2,\"é\":3,\"a\":-0}", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "eyJiIjoxLjAwLCLDqSI6MiwiZcyBIjozLCJhIjotMH0=" + } + } + ] +} diff --git a/Tools/MobileQualification/Sources/ProbeCases/PublicBehavior.swift b/Tools/MobileQualification/Sources/ProbeCases/PublicBehavior.swift new file mode 100644 index 0000000..b017134 --- /dev/null +++ b/Tools/MobileQualification/Sources/ProbeCases/PublicBehavior.swift @@ -0,0 +1,129 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import CryptoKit +import JSON +import JSONSchema +import JSONPatch +import JSONStreaming + +public enum ProbeFailure: Error { case assertion(String) } +public func verify(_ condition: Bool, _ name: String) throws { if !condition { throw ProbeFailure.assertion(name) } } +public struct FrozenFixtures: Decodable { + public struct Item: Decodable { + public let id: String + public let inner_json: String + public let inner_utf8_sha256: String + } + public let cases: [Item] + public static func load() throws -> Self { + guard let url = Bundle.module.url(forResource: "version1", withExtension: "json") else { throw ProbeFailure.assertion("fixture resource") } + let value = try JSONDecoder().decode(Self.self, from: Data(contentsOf: url)) + try verify(value.cases.count == 8, "eight pinned fixtures") + for item in value.cases { + let hash = SHA256.hash(data: Data(item.inner_json.utf8)).map { String(format: "%02x", $0) }.joined() + try verify(hash == item.inner_utf8_sha256, "fixture SHA-256") + } + return value + } +} +public func exactBytes(_ value: JSON) throws -> Data { try JSONWriter(options: .init(numberStyle: .preserved)).write(value) } +public func publicBehavior() throws { + let value = try JSONParser().parse(#"{"count":9007199254740993,"é":1,"é":2,"items":[0.1,null]}"#) + let report = try value.readWithReport { root in + try verify(root["count"].requiredInteger(as: UInt64.self) == 9007199254740993, "exact integer") + try verify(root["é"].requiredInteger(as: Int.self) == 1 && root["é"].requiredInteger(as: Int.self) == 2, "scalar key identity") + do { _ = try root["items"][0].requiredDouble(policy: .exact); throw ProbeFailure.assertion("exact binary refusal") } + catch let error as JSONAccessFailure { try verify(error.cause.category == .conversion, "conversion classification") } + try verify(root["items"][0].requiredDouble(policy: .rounded) == 0.1, "finite rounded binary bridge") + } + try verify(report.statistics[.workUnits] > 0 && report.statistics[.numericWorkUnits] > 0, "public cumulative counts") + let registry = try JSONSchemaRegistry.draft2020_12() + try verify(registry.documents.count == 9, "packaged pinned schemas with CryptoKit") + let schema = try JSONSchemaDocument(JSONParser().parse(#"{"type":"object","properties":{"count":{"minimum":9007199254740993},"é":{"type":"integer"}},"required":["count","é"],"propertyNames":{"pattern":"^.+$"}}"#)) + let plan = try JSONSchemaCompiler().compile(schema) + try verify(JSONSchemaValidator().validate(value, against: plan).isValid, "offline plan/native regex valid") + try verify(!JSONSchemaValidator().validate(.null, against: plan).isValid, "ordinary invalid verdict") + let patch = try JSONPatch(json: JSONParser().parse(#"[{"op":"test","path":"/count","value":9007199254740993},{"op":"replace","path":"/é","value":3}]"#)) + guard case .document(let changed) = try patch.applying(to: value) else { throw ProbeFailure.assertion("unexpected removed root") } + try verify(changed.read { try $0["é"].requiredInteger(as: Int.self) } == 3 && value.read { try $0["é"].requiredInteger(as: Int.self) } == 1, "staged Patch") + try verify(JSONMergePatch(.null).applying(to: value).isNull, "Merge Patch null") + for fixture in try FrozenFixtures.load().cases { + let payload = try JSONCodablePayload(preserving: JSONParser().parse(fixture.inner_json)) + let reopened = try PropertyListDecoder().decode(JSONCodablePayload.self, from: PropertyListEncoder().encode(payload)) + try verify(exactBytes(reopened.value()) == Data(fixture.inner_json.utf8), "opaque payload fidelity") + } + try acceptanceWave() +} +private struct RecoveringData: Decodable { + let rejected: Int + init(from decoder: any Decoder) throws { + let container = try decoder.singleValueContainer() + var count = 0 + for _ in 0..<100 { + do { _ = try container.decode(Data.self); throw ProbeFailure.assertion("malformed Base64 accepted") } + catch is DecodingError { count += 1 } + } + rejected = count + } +} +private func acceptanceWave() throws { + var limits = JSONLimits.balanced; limits[.retainedBytes] = 32 + let recovered = try JSON.string("AAAA!").decodeWithReport(RecoveringData.self, coding: .init(limits: limits)) + try verify(recovered.result.rejected == 100 && recovered.statistics[.retainedBytes] == 0, "ordinary corruption releases scratch") + try verify(recovered.statistics[.workUnits] > 100 && recovered.statistics.peakLogicalRetainedBytes <= 32, "failed work remains cumulative") + let members = (0..<33).map { JSONMember("k\($0)", value: .number(JSONNumber(integer: $0))) } + [.init("é", value: 1), .init("e\u{301}", value: 2)] + let wide = JSON.object(try JSONObject(members: members)) + let payload = try JSONCodablePayload(preserving: wide) + let reopened = try JSON(encoding: payload).decode(JSONCodablePayload.self).value() + try verify(exactBytes(reopened) == exactBytes(wide), "indexed native payload fidelity") + let raw = try JSONParser().parse(#"{"patternProperties":{"^k[0-9]+$":{"type":"integer"}},"properties":{"é":{"const":1},"e\u0301":{"const":2}},"unevaluatedProperties":false}"#) + let plan = try JSONSchemaCompiler().compile(JSONSchemaDocument(raw)) + try verify(JSONSchemaValidator().validate(reopened, against: plan).isValid, "indexed evaluated locations") + let patch = try JSONPatch(json: JSONParser().parse(#"[{"op":"move","from":"/k32","path":"/new"},{"op":"remove","path":"/k0"}]"#)) + guard case .document(let changed) = try patch.applying(to: reopened) else { throw ProbeFailure.assertion("indexed Patch root removed") } + try changed.read { cursor in + try verify(cursor["new"].requiredInteger(as: Int.self) == 32, "indexed Patch selection") + try verify(cursor.presence(of: "k0") == .absent, "indexed Patch removal") + try verify(cursor["é"].requiredInteger(as: Int.self) == 1 && cursor["e\u{301}"].requiredInteger(as: Int.self) == 2, "indexed scalar keys") + } +} +private struct Chunks: AsyncSequence, Sendable { + typealias Element = JSONByteChunk + let text: String + func makeAsyncIterator() -> Iterator { .init(bytes: Array(text.utf8)) } + struct Iterator: AsyncIteratorProtocol { + let bytes: [UInt8]; var offset = 0 + mutating func next() async -> JSONByteChunk? { + guard offset < bytes.count else { return nil } + defer { offset += 1 }; return .init([bytes[offset]]) + } + } +} +private actor Sink { + var bytes: [UInt8] = [] + func accept(_ chunk: JSONByteChunk) async { await Task.yield(); bytes.append(contentsOf: chunk.bytes) } +} +public func streamAndCancellation() async throws { + var elements = 0, completed = 0 + for try await event in JSONTopLevelArraySequence(Chunks(text: "[0.1,null] \n")) { + switch event { case .element(let index, _): try verify(index == UInt64(elements), "stream order"); elements += 1 + case .documentCompleted(let summary): try verify(summary.records == 2, "certified record count"); completed += 1 } + } + try verify(elements == 2 && completed == 1, "explicit completion") + var bad = JSONTopLevelArraySequence(Chunks(text: "[0]x")).makeAsyncIterator() + do { + while let event = try await bad.next() { if case .documentCompleted = event { throw ProbeFailure.assertion("premature completion") } } + throw ProbeFailure.assertion("trailing grammar accepted") + } catch let error as JSONFailure { try verify(error.category == .syntax, "operational syntax error") } + let end = try await bad.next(); try verify(end == nil, "terminal pull") + let sink = Sink(), value = try JSONParser().parse(#"{"x":[0.1,null,"é"]}"#) + let statistics = try await JSONAsyncWriter(chunkSize: 3).writeWithReport(value) { await sink.accept($0) } + let written = await sink.bytes + try verify(written == Array(JSONWriter().write(value)) && statistics[.outputBytes] == UInt64(written.count), "awaited sink") + let task = Task { + withUnsafeCurrentTask { $0?.cancel() } + return try value.read { _ in true } + } + do { _ = try await task.value; throw ProbeFailure.assertion("cancelled read succeeded") } + catch is CancellationError { } +} diff --git a/Tools/MobileQualification/Sources/Runtime/Runtime.swift b/Tools/MobileQualification/Sources/Runtime/Runtime.swift new file mode 100644 index 0000000..2f9cc80 --- /dev/null +++ b/Tools/MobileQualification/Sources/Runtime/Runtime.swift @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import ProbeCases +@main struct Runtime { + static func main() async throws { + if CommandLine.arguments.count == 4, CommandLine.arguments[1] == "--disk-phase" { + let phase = CommandLine.arguments[2], url = URL(fileURLWithPath: CommandLine.arguments[3]) + try await MainActor.run { try diskPhase(phase, url: url) } + print("DISK_PHASE passed phase=" + phase + " rows=8") + return + } + try publicBehavior(); print("PUBLIC_BEHAVIOR passed fixtures=8") + try await streamAndCancellation(); print("STREAM_CANCELLATION passed") + #if os(macOS) + try diskPersistence(); print("DISK_PERSISTENCE passed containers=3 rows=8") + print("MOBILE_QUALIFICATION passed cases=3") + #else + print("DISK_PERSISTENCE requires three separate phase processes") + print("MOBILE_QUALIFICATION passed cases=2") + #endif + } +} diff --git a/Tools/MobileQualification/Tests/ProbeTests/RuntimeTests.swift b/Tools/MobileQualification/Tests/ProbeTests/RuntimeTests.swift new file mode 100644 index 0000000..f2589ef --- /dev/null +++ b/Tools/MobileQualification/Tests/ProbeTests/RuntimeTests.swift @@ -0,0 +1,6 @@ +// SPDX-License-Identifier: Apache-2.0 +import Testing +import ProbeCases +@Test func publicProductsAndPayloadFixtures() throws { try publicBehavior() } +@Test func awaitedStreamsAndCancellation() async throws { try await streamAndCancellation() } +@Test @MainActor func swiftDataDiskTeardownReopenAndUpdate() throws { try diskPersistence() } diff --git a/Tools/Release1_1Probes/ConstructionBenchmark.swift b/Tools/Release1_1Probes/ConstructionBenchmark.swift new file mode 100644 index 0000000..d6981e1 --- /dev/null +++ b/Tools/Release1_1Probes/ConstructionBenchmark.swift @@ -0,0 +1,103 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import Darwin +import CryptoKit +@testable import JSON + +private final class AllocationObservations { + var snapshots: [[String: UInt64]] = [] + func capture(_ position: Int) { + var info = malloc_statistics_t(); malloc_zone_statistics(nil, &info) + snapshots.append(["position": UInt64(position), "blocksInUse": UInt64(info.blocks_in_use), "bytesInUse": UInt64(info.size_in_use)]) + } +} +private struct Fields: Encodable { + let keys: [String] + var observations: AllocationObservations? = nil + func encode(to encoder: any Encoder) throws { + var c = encoder.container(keyedBy: JSONCodingKey.self) + observations?.capture(0) + for (position, key) in keys.enumerated() { + try c.encodeNil(forKey: JSONCodingKey(key)) + if [1,32,256,1024,keys.count].contains(position + 1) { observations?.capture(position + 1) } + } + } +} +private struct Sample: Encodable { + let operation: String, width: Int, shape: String, sample: Int, status: String + let inputSHA256: String, seconds: Double + let work: UInt64?, logicalPeak: UInt64?, failure: String? + let processPeakRSS: Int64, allocatorBlocksInUse: UInt64, allocatorBytesInUse: UInt64 +} +@main struct ConstructionBenchmark { + static func main() throws { + var samples: [Sample] = [], allocationRows: [[String: String]] = [] + for width in [0,1,4,8,16,32,64,256,1024,10000] { + for shape in ["ascii", "prefix", "unicode"] { + let keys = (0..= 0 || failure != nil { + samples.append(.init(operation: operation, width: width, shape: shape, sample: sample, + status: failure == nil ? "passed" : "refused", inputSHA256: digest, + seconds: Double(duration.seconds) + Double(duration.attoseconds)/1e18, + work: stats?[.workUnits], logicalPeak: stats?.peakLogicalRetainedBytes, failure: failure, + processPeakRSS: Int64(usage.ru_maxrss), allocatorBlocksInUse: UInt64(info.blocks_in_use), allocatorBytesInUse: UInt64(info.size_in_use))) + } + if failure != nil { break } + } + } + if [32,256,1024].contains(width) { + let observer = AllocationObservations() + _ = try JSONValueEncoder(options: .init(limits: .offline)).encode(Fields(keys: keys, observations: observer)) + let data = try JSONEncoder().encode(observer.snapshots) + allocationRows.append(["width": String(width), "shape": shape, "snapshots": String(decoding: data, as: UTF8.self)]) + } + } + } + let encoder = JSONEncoder(); encoder.outputFormatting = [.sortedKeys] + let url = URL(fileURLWithPath: CommandLine.arguments[1]) + try encoder.encode(samples).write(to: url) + try encoder.encode(allocationRows).write(to: url.deletingPathExtension().appendingPathExtension("allocations.json")) + print("CONSTRUCTION_BENCHMARK cells=120 samples=\(samples.count) warmups=3 measured_samples=9 resource_refusals=\(samples.filter { $0.status == "refused" }.count) allocation_cells=\(allocationRows.count)") + } +} diff --git a/Tools/Release1_1Probes/Consumer.swift b/Tools/Release1_1Probes/Consumer.swift new file mode 100644 index 0000000..6c5f327 --- /dev/null +++ b/Tools/Release1_1Probes/Consumer.swift @@ -0,0 +1,22 @@ +// SPDX-License-Identifier: Apache-2.0 +import JSON +import ReaderProbe +struct Model: Decodable { let count: UInt64 } +let value = try JSONParser().parse(#"{"users":[{"name":"Ada","empty":null,"model":{"count":9007199254740993}}]}"#) +let report = try probeReadWithReport(value) { root in + let user = try root["users"][0] + let name = try user[JSONKey("name")].requiredString() + guard case .absent = try user.presence(of: "missing"), + case .null = try user.presence(of: "empty"), + case .value(let present) = try user.presence(of: "name") else { fatalError("Presence mismatch") } + let presentName = try present.requiredString() + precondition(presentName == name) + let count = try user["model"]["count"].requiredInteger(as: UInt64.self) + let model = try user["model"].decode(Model.self) + precondition(model.count == count) + return name +} +precondition(report.result == "Ada") +precondition(report.statistics[.workUnits] > 0) +precondition(report.statistics[.numericWorkUnits] > 0) +print("READER_CONSUMER passed chains=String/JSONKey/Int presence=absent/null/value exact=9007199254740993 domain=\(report.statistics.domain) work=\(report.statistics[.workUnits]) numeric=\(report.statistics[.numericWorkUnits])") diff --git a/Tools/Release1_1Probes/Fixtures/version1.json b/Tools/Release1_1Probes/Fixtures/version1.json new file mode 100644 index 0000000..ca0cd0c --- /dev/null +++ b/Tools/Release1_1Probes/Fixtures/version1.json @@ -0,0 +1,86 @@ +{ + "format_version": 1, + "status": "logical-envelope-fixtures; exact persistence codec qualification not yet run", + "cases": [ + { + "id": "null", + "inner_utf8_sha256": "74234e98afe7498fb5daf1f36ac2d78acc339464f950703b8c019892f982b90b", + "inner_json": "null", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "bnVsbA==" + } + }, + { + "id": "boolean", + "inner_utf8_sha256": "b5bea41b6c623f7c09f1bf24dcae58ebab3c0cdd90ad966bc43a45b44867e12b", + "inner_json": "true", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "dHJ1ZQ==" + } + }, + { + "id": "string", + "inner_utf8_sha256": "06c3994e4804c1a17dfcc7607f72944152a706367c0c322b34fd9a8f1b1a455d", + "inner_json": "\"é é 🚀\"", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "IsOpIGXMgSDwn5qAIg==" + } + }, + { + "id": "decimal", + "inner_utf8_sha256": "14be4b45f18e0d8c67b4f719b5144eee88497e413709d11d85b096d8e2346310", + "inner_json": "0.1", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "MC4x" + } + }, + { + "id": "lexeme", + "inner_utf8_sha256": "59bbf72ab162059f732c213462a666646ff97896f9efd0b2ec53b97dda502472", + "inner_json": "-0.00e+2", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "LTAuMDBlKzI=" + } + }, + { + "id": "large-number", + "inner_utf8_sha256": "decc538c077786966ac863b5532c4027b8587ff40f6e3103379af62b44eae44d", + "inner_json": "123456789012345678901234567890123456789012345678901234567890", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "MTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkwMTIzNDU2Nzg5MDEyMzQ1Njc4OTAxMjM0NTY3ODkw" + } + }, + { + "id": "array", + "inner_utf8_sha256": "c742fdf61c92ed1d0081e93b41b69224909adffd4484ae4a023085d827ccb229", + "inner_json": "[1e0,null,\"x\"]", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "WzFlMCxudWxsLCJ4Il0=" + } + }, + { + "id": "ordered-scalars", + "inner_utf8_sha256": "57c900d6ddd444977ea738f05bf6c030f968423d21f1a1410384841af835b048", + "inner_json": "{\"b\":1.00,\"é\":2,\"é\":3,\"a\":-0}", + "envelope": { + "format": "swift-json-value", + "version": 1, + "utf8Base64": "eyJiIjoxLjAwLCLDqSI6MiwiZcyBIjozLCJhIjotMH0=" + } + } + ] +} diff --git a/Tools/Release1_1Probes/IndexBenchmark.swift b/Tools/Release1_1Probes/IndexBenchmark.swift new file mode 100644 index 0000000..2dfd2eb --- /dev/null +++ b/Tools/Release1_1Probes/IndexBenchmark.swift @@ -0,0 +1,86 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import CryptoKit +@testable import JSON + +@main struct IndexBenchmark { + struct Sample: Codable { + let algorithm: String, shape: String, operation: String, status: String + let width: Int, sample: Int + let seconds: Double, work: UInt64, logicalPeak: UInt64 + let inputSHA256: String + let failure: String? + } + static func seconds(_ duration: Duration) -> Double { + Double(duration.components.seconds) + Double(duration.components.attoseconds) / 1e18 + } + static func keys(_ width: Int, shape: String) -> [String] { + (0..(_ type: I.Type, algorithm: String, width: Int, shape: String) throws -> [Sample] { + let keys = keys(width, shape: shape) + let digest = SHA256.hash(data: try JSONEncoder().encode(keys)).map { String(format: "%02x", $0) }.joined() + let clock = ContinuousClock() + var results: [Sample] = [] + for sample in -3..<9 { + var meter = try JSONWorkMeter(limits: .offline, domain: .core) + var index = I() + let start = clock.now + do { + for (position, text) in keys.enumerated() { + try index.insert(ProbeKey(text, position: position, meter: &meter), meter: &meter) + } + } catch let error as JSONFailure where error.category == .resource { + results.append(.init(algorithm: algorithm, shape: shape, operation: "construction", status: "refused", width: width, sample: sample, + seconds: seconds(start.duration(to: clock.now)), work: meter.statistics()[.workUnits], + logicalPeak: meter.statistics().peakLogicalRetainedBytes, inputSHA256: digest, failure: error.resource?.metric.rawValue)) + // A refusal is recorded once, never converted into a timing advantage. + return results + } + let construction = seconds(start.duration(to: clock.now)) + if sample >= 0 { + results.append(.init(algorithm: algorithm, shape: shape, operation: "construction", status: "passed", width: width, sample: sample, + seconds: construction, work: meter.statistics()[.workUnits], logicalPeak: meter.statistics().peakLogicalRetainedBytes, + inputSHA256: digest, failure: nil)) + } + let before = meter.statistics()[.workUnits] + let lookupStart = clock.now + for _ in 0..<3 { + for position in [0, width / 2, width - 1, width] { + let text = position >= 0 && position < width ? keys[position] : "missing" + let query = try ProbeKey(text, position: -1, meter: &meter) + let found = try index.find(query, meter: &meter) + precondition(found == (position >= 0 && position < width ? position : nil)) + try meter.release(UInt64(query.bytes.count)) + } + } + if sample >= 0 { + results.append(.init(algorithm: algorithm, shape: shape, operation: "12-lookups", status: "passed", width: width, sample: sample, + seconds: seconds(lookupStart.duration(to: clock.now)), work: meter.statistics()[.workUnits] - before, + logicalPeak: meter.statistics().peakLogicalRetainedBytes, inputSHA256: digest, failure: nil)) + } + } + return results + } + static func main() throws { + var samples: [Sample] = [] + for shape in ["ascii", "prefix", "unicode"] { + for width in [0, 1, 4, 8, 16, 32, 64, 256, 1024, 10000] { + samples += try run(ProbeScalarIndex.self, algorithm: "scalar", width: width, shape: shape) + samples += try run(ProbeLinearIndex.self, algorithm: "fingerprint-linear", width: width, shape: shape) + samples += try run(ProbeAVLIndex.self, algorithm: "array-AVL", width: width, shape: shape) + samples += try run(ProbeHashIndex.self, algorithm: "controlled-hash", width: width, shape: shape) + } + } + let encoder = JSONEncoder(); encoder.outputFormatting = [.prettyPrinted, .sortedKeys] + try encoder.encode(samples).write(to: URL(fileURLWithPath: CommandLine.arguments[1])) + print("INDEX_AB samples=\(samples.count) widths=10 shapes=3 algorithms=4 warmups=3 samples=9 lookupIterations=3 profile=offline constructionIterations=1") + } +} diff --git a/Tools/Release1_1Probes/LifetimeAnnotated.swift b/Tools/Release1_1Probes/LifetimeAnnotated.swift new file mode 100644 index 0000000..9e285e8 --- /dev/null +++ b/Tools/Release1_1Probes/LifetimeAnnotated.swift @@ -0,0 +1,13 @@ +// SPDX-License-Identifier: Apache-2.0 +final class Owner {} +public struct Cursor: ~Escapable { + let value: Int + @_lifetime(borrow owner) + init(owner: borrowing Owner) { value = 42 } + public subscript(_ key: String) -> Cursor { @_lifetime(copy self) get throws { self } } +} +func read(_ body: (borrowing Cursor) throws -> Result) rethrows -> Result { + let owner = Owner() + let cursor = Cursor(owner: owner) + return try body(cursor) +} diff --git a/Tools/Release1_1Probes/LookupBenchmark.swift b/Tools/Release1_1Probes/LookupBenchmark.swift new file mode 100644 index 0000000..5eeeaa4 --- /dev/null +++ b/Tools/Release1_1Probes/LookupBenchmark.swift @@ -0,0 +1,55 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import CryptoKit +@testable import JSON + +private struct Sample: Encodable { + let width: Int, shape: String, query: String, sample: Int + let iterations: Int, seconds: Double, work: UInt64, logicalPeak: UInt64 + let inputSHA256: String +} +@main struct LookupBenchmark { + static func main() throws { + var samples: [Sample] = [] + for width in [0,1,4,8,16,32,64,256,1024,10000] { + for shape in ["ascii", "prefix", "unicode"] { + let keys = (0.. 8 { queries.append(("first-filtered", keys[8])) } + for (name, key) in queries { + for sample in -3..<9 { + var meter = try JSONWorkMeter(limits: .offline, domain: .core) + let clock = ContinuousClock(); let start = clock.now + for _ in 0..<20 { + let value = try object.lookup(key, meter: &meter) + precondition(name == "miss" ? value == nil : value?.stringValue?.utf8.elementsEqual(key.utf8) == true) + } + let duration = start.duration(to: clock.now).components + if sample >= 0 { + let stats = meter.statistics() + samples.append(Sample(width: width, shape: shape, query: name, sample: sample, iterations: 20, + seconds: Double(duration.seconds) + Double(duration.attoseconds)/1e18, + work: stats[.workUnits], logicalPeak: stats.peakLogicalRetainedBytes, inputSHA256: digest)) + } + } + } + } + } + let encoder = JSONEncoder(); encoder.outputFormatting = [.sortedKeys] + try encoder.encode(samples).write(to: URL(fileURLWithPath: CommandLine.arguments[1])) + print("LOOKUP_BENCHMARK samples=\(samples.count) warmups=3 iterations=20 measured_samples=9") + } +} diff --git a/Tools/Release1_1Probes/MobileConsumer/Package.swift b/Tools/Release1_1Probes/MobileConsumer/Package.swift new file mode 100644 index 0000000..080f453 --- /dev/null +++ b/Tools/Release1_1Probes/MobileConsumer/Package.swift @@ -0,0 +1,10 @@ +// swift-tools-version: 6.4 +// SPDX-License-Identifier: Apache-2.0 +import PackageDescription +let package = Package(name: "MobileConsumer", platforms: [.macOS("27.0"), .iOS("27.0")], + products: [.library(name: "MobileConsumer", targets: ["MobileConsumer"])], + dependencies: [.package(name: "Swift-JSON", path: "../../..")], + targets: [.target(name: "MobileConsumer", dependencies: [ + .product(name: "JSON", package: "Swift-JSON"), .product(name: "JSONSchema", package: "Swift-JSON"), + .product(name: "JSONStreaming", package: "Swift-JSON"), .product(name: "JSONPatch", package: "Swift-JSON") + ])], swiftLanguageModes: [.v6]) diff --git a/Tools/Release1_1Probes/MobileConsumer/Sources/MobileConsumer/Consumer.swift b/Tools/Release1_1Probes/MobileConsumer/Sources/MobileConsumer/Consumer.swift new file mode 100644 index 0000000..0ab93df --- /dev/null +++ b/Tools/Release1_1Probes/MobileConsumer/Sources/MobileConsumer/Consumer.swift @@ -0,0 +1,14 @@ +// SPDX-License-Identifier: Apache-2.0 +import JSON +import JSONSchema +import JSONPatch +import JSONStreaming +public func mobileSDKProbe() throws -> Bool { + let value = try JSONParser().parse(#"{"count":9007199254740993,"name":"Ada"}"#) + let schema = try JSONSchemaDocument(JSONParser().parse(#"{"type":"object","properties":{"name":{"pattern":"^A"}},"required":["name"]}"#)) + let plan = try JSONSchemaCompiler().compile(schema) + let verdict = try JSONSchemaValidator().validate(value, against: plan) + let merged = try JSONMergePatch(.null).applying(to: value) + _ = JSONAsyncWriter() + return verdict.isValid && merged == .null +} diff --git a/Tools/Release1_1Probes/Nonescapable.swift b/Tools/Release1_1Probes/Nonescapable.swift new file mode 100644 index 0000000..b3e40ce --- /dev/null +++ b/Tools/Release1_1Probes/Nonescapable.swift @@ -0,0 +1,12 @@ +// SPDX-License-Identifier: Apache-2.0 +final class Owner {} +public struct Cursor: ~Escapable { + let value: Int + init(owner: borrowing Owner) { value = 42 } + public subscript(_ key: String) -> Cursor { get throws { self } } +} +func read(_ body: (borrowing Cursor) throws -> Result) rethrows -> Result { + let owner = Owner() + let cursor = Cursor(owner: owner) + return try body(cursor) +} diff --git a/Tools/Release1_1Probes/PublicReaderConsumer.swift b/Tools/Release1_1Probes/PublicReaderConsumer.swift new file mode 100644 index 0000000..a9e1389 --- /dev/null +++ b/Tools/Release1_1Probes/PublicReaderConsumer.swift @@ -0,0 +1,20 @@ +// SPDX-License-Identifier: Apache-2.0 +import Foundation +import JSON + +private struct Reading: Decodable { let count: UInt64; let name: String; let bytes: Data } +private let value = try JSONParser().parse(#"{"users":[{"count":18446744073709551615,"name":"sensor","bytes":"AAH/","nullable":null}]}"#) +private let report = try value.readWithReport { root in + let user = try root[JSONKey("users")][0] + let name = try user["name"].requiredString(); precondition(name == "sensor") + let count = try user["count"].requiredInteger(as: UInt64.self); precondition(count == UInt64.max) + let missing = try user.presence(of: "missing"); precondition(missing == .absent) + let nullable = try user.stringPresence(of: "nullable"); precondition(nullable == .null) + return try user.decode(Reading.self) +} +precondition(report.result.count == UInt64.max && report.result.bytes == Data([0, 1, 255])) +precondition(report.statistics.domain == .core && report.statistics[.numericWorkUnits] > 0) +private let escaped = try value.read { $0 } +do { _ = try escaped.requiredValue(); fatalError("Escaped cursor remained valid") } +catch let error as JSONFailure { precondition(error.code == .invalidOperation) } +print("PUBLIC_READER_CONSUMER passed work=\(report.statistics[.workUnits]) numeric=\(report.statistics[.numericWorkUnits])") diff --git a/Tools/Release1_1Probes/PublicTaskEscape.swift b/Tools/Release1_1Probes/PublicTaskEscape.swift new file mode 100644 index 0000000..1a298ee --- /dev/null +++ b/Tools/Release1_1Probes/PublicTaskEscape.swift @@ -0,0 +1,6 @@ +// SPDX-License-Identifier: Apache-2.0 +import JSON +func forbidden(_ cursor: JSONReadCursor) { + let closure: @Sendable () throws -> JSON = { try cursor.requiredValue() } + _ = closure +} diff --git a/Tools/Release1_1Probes/PublicTaskTransfer.swift b/Tools/Release1_1Probes/PublicTaskTransfer.swift new file mode 100644 index 0000000..17b339c --- /dev/null +++ b/Tools/Release1_1Probes/PublicTaskTransfer.swift @@ -0,0 +1,6 @@ +// SPDX-License-Identifier: Apache-2.0 +import JSON +func forbidden(_ cursor: JSONReadCursor) { + Task.detached { try cursor.requiredValue() } + _ = try? cursor.requiredValue() +} diff --git a/Tools/Release1_1Probes/README.md b/Tools/Release1_1Probes/README.md new file mode 100644 index 0000000..74afbec --- /dev/null +++ b/Tools/Release1_1Probes/README.md @@ -0,0 +1,36 @@ +# First milestone qualification subjects + +These files support the [1.1 plan](../../Documentation/Architecture/Release1.1/Plan.md). +They are not runtime products or a second JSON engine. Reader/index implementations +live under Tests/JSONTests/Release1_1 and use the actual native core meter via testable +imports. Consumer.swift exercises exported prototype shapes without private meters. +TaskEscape and TaskTransfer are intentional compiler-negative fixtures; they are +not targets in the mobile consumer package. + +SwiftDataShape is an explicit Version 1 Codable *shape* used only for SDK compilation; +it does not implement or qualify the strict persistence codec. The mobile consumer +imports all four products and compiles schema-resource/regex call sites. SDK builds, +simulator runtime correctness and device measurements are separate claims. + +Use Scripts/qualify_release_1_1.py through Scripts/ci.py where applicable. Index +measurements must run on an otherwise idle release-native host. Samples identify +input hashes, finite policy, work, logical peak and refusals. They do not measure +isolated allocator allocations, process RSS or physical mobile throughput. + +The small production increment is reviewed in +[BenchmarkReview](../../Documentation/Architecture/Release1.1/BenchmarkReview.md). +Its deterministic lookup-stage corpus can be repeated after building the real +release testable module on the supported toolchain: + +```sh +swift test -c release --filter wideLookup +xcrun swiftc -swift-version 6 -O -parse-as-library \ + -I .build/out/Products/Release -L .build/out/Products/Release -lJSON \ + Tools/Release1_1Probes/LookupBenchmark.swift -o .build/lookup-benchmark +.build/lookup-benchmark .build/lookup-samples.json +``` + +For A/B, build the baseline libraries in a separate worktree and compile the same +benchmark source against those libraries. Record both core source identities and +the benchmark/input hashes. Run measurements while local verification builds are +idle; preserve all samples, refusal outcomes and repeated-run variance. diff --git a/Tools/Release1_1Probes/SwiftDataShape.swift b/Tools/Release1_1Probes/SwiftDataShape.swift new file mode 100644 index 0000000..948801d --- /dev/null +++ b/Tools/Release1_1Probes/SwiftDataShape.swift @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: Apache-2.0 +import SwiftData + +public struct PayloadShape: Codable, Sendable { + private let utf8Base64: String + enum CodingKeys: String, CodingKey { case format, version, utf8Base64 } + public init(utf8Base64: String) { self.utf8Base64 = utf8Base64 } + public func encode(to encoder: any Encoder) throws { + var c = encoder.container(keyedBy: CodingKeys.self) + try c.encode("swift-json-value", forKey: .format) + try c.encode(1, forKey: .version) + try c.encode(utf8Base64, forKey: .utf8Base64) + } + public init(from decoder: any Decoder) throws { + let c = try decoder.container(keyedBy: CodingKeys.self) + guard try c.decode(String.self, forKey: .format) == "swift-json-value", + try c.decode(Int.self, forKey: .version) == 1 else { + throw DecodingError.dataCorrupted(.init(codingPath: [], debugDescription: "Unsupported payload format")) + } + utf8Base64 = try c.decode(String.self, forKey: .utf8Base64) + } +} +@Model final class ConfigurationSnapshot { + var name: String + var applicationSchemaVersion: Int + @Attribute(.codable) var payload: PayloadShape + @Attribute(.codable) var optionalPayload: PayloadShape? + init(name: String, payload: PayloadShape) { + self.name = name; applicationSchemaVersion = 1 + self.payload = payload; optionalPayload = nil + } +} diff --git a/Tools/Release1_1Probes/TaskEscape.swift b/Tools/Release1_1Probes/TaskEscape.swift new file mode 100644 index 0000000..9dc5544 --- /dev/null +++ b/Tools/Release1_1Probes/TaskEscape.swift @@ -0,0 +1,9 @@ +// SPDX-License-Identifier: Apache-2.0 +import JSON +import ReaderProbe +func illegalEscape(_ value: JSON) throws { + _ = try probeReadWithReport(value) { cursor in + let work: @Sendable () throws -> String = { try cursor["name"].requiredString() } + return work + } +} diff --git a/Tools/Release1_1Probes/TaskTransfer.swift b/Tools/Release1_1Probes/TaskTransfer.swift new file mode 100644 index 0000000..1b9e127 --- /dev/null +++ b/Tools/Release1_1Probes/TaskTransfer.swift @@ -0,0 +1,8 @@ +// SPDX-License-Identifier: Apache-2.0 +import JSON +import ReaderProbe +func transferCursor(_ value: JSON) throws { + _ = try probeReadWithReport(value) { cursor in + Task.detached { try cursor["name"].requiredString() } + } +}