build: enable fat LTO for release - #19
Open
0xLucqs wants to merge 76 commits into
Open
Conversation
A nationality-only statement (PredicateMode::Nat) unconditionally requested the birth_date/AgeOver attribute and vice versa — expected_mdoc_attributes() ignored the mode — so selectively-disclosed presentations failed extraction with ElementMissing on the other leg's element. The request and the verify-side disclosed-set pinning now follow uses_age()/uses_nat(), keeping cross-mode confusion fail-closed (an age-only statement rejects a two-attribute proof and vice versa). Adds a mode-binding unit test and a single-predicate e2e prove/verify test over the canonical v2 fixture. Port of 6c051d4 from q15/mobile-pool-lto.
Fix 1 (DoS): the verifier reconstructs tree-0 (interpolate + LDE + Merkle commit) from prover-supplied proof-shape fields before checking anything else, with no upper bound. A malicious proof claiming e.g. sha_log_n_rows = 30 forces a multi-gigabyte allocation (and a Box-leaked twiddle table) before rejection. Add fail-closed shape validation that runs BEFORE compute_canonical_preprocessed_root on every production verify path: - eu-id-prover: cap sha_log_n_rows (<= 20), sha_group_width (<= 8) and the digest-bind bridge log-sizes (<= 20) in verify_stark_with_config and the mdoc verifier, returning the new typed Error::ShapeTooLarge. - air-core: MAX_CANONICAL_PREPROCESSED_LOG_SIZE (25) backstop inside compute_canonical_preprocessed_root, rejecting any module whose max preprocessed log-size is oversized before the twiddle allocation. Caps derived from honest maxima + headroom (documented at each constant): SHA trace ~log 17 for a 64 KiB message (+3), group width fixed at 6 (+2), preprocessed tables top out at 3*6 = 18 (+7). Fix 2 (classical fail-closed): with the ec-coprocessor feature OFF, the legacy P256/DigestBind modules have witness-dependent preprocessed columns and no canonical reconstruction, so every verify path fails closed at the tree-0 root. Documented on the air-core trait default and the P256Verifier site; pinned by a feature-gated verify-entry test. Tests (all fast, run by default): - air-core: canonical_root_rejects_oversized_preprocessed_shape, canonical_root_fails_closed_without_reconstruct_override - eu-id-prover: shape_cap_rejects_oversized_sha_log_n_rows, shape_cap_accepts_values_at_the_boundary - eu-id-prover (feature off, ignored/slow): classical_verify_entry_fails_closed_not_panics
…ve-R e2e negative Mirror the prove-side rayon par_iter over sumcheck instances on the verify side. In verify_mdoc_p4b_circuit_bundle_profiled and verify_implemented_circuit_bundle_batch_with_projection_profiled the pure per-instance sumcheck (channel derivation + verify_circuit) now runs in parallel; the order-dependent claim walk (consistency_cursor / signer_state over consistency_claim_values) stays sequential and consumes the collected results in the original order. No transcript/channel or claim-order change. Add off_curve_r_monolithic_pipeline_rejects: injects an off-curve R (true R's x with y+1) through the wrong-R plumbing; the projective EC trace builder rejects it (AffineOutputMismatch) so no accepting monolithic proof can exist. Complements the standalone final_check_hint_binding_rejects_off_curve_r.
…ed booleanity already enforce them The 8 split-pack lookup tables (4 RoundSplit + 4 SigmaSplit) and their entire producer/consumer apparatus were vestigial: every SHA round/schedule function is computed from committed 32-bit boolean bit-planes, every keyed limb already has a recomposition constraint to those bits (which IS the range check the lookup advertised), and the packed-group cells fed nothing but the split-pack lookups. Deletion needs zero compensating constraints. Deleted across crates/stwo-sha256: the 8 split-pack producers + relations + SharedProducer arms, the AIR consumer lookups (wire_round_split_pack, wire_sigma_input_split) and their helpers, the dead packed trace columns (maj_grp, ch_grp, b/c/f/g_init, sched_sigma0/1_split — 56 base cols/row), the witness structs/generation, multiplicity + interaction + preprocessed generation, and the standalone tables.rs split-pack builders/TableId variants. Kept: the 4 range tables (Range2/4/5/16), all booleanity + recomposition constraints. Prover and verifier recompute the canonical root together, so the proof format simply changes (no shim). Shape: Layout::TOTAL_COLS 493 -> 437; preprocessed columns 46 -> 14; shared table producers now range-only. Negatives added (constraint_negative.rs): - rejects_maj_limb_off_bit_recomposition: a keyed limb off its bit-planes is rejected by the ungated recomposition constraint, not by any lookup. - rejects_non_boolean_bit_on_padding_row: ungated booleanity rejects a non-boolean bit-plane cell even on a padding row. Measured (fat-LTO release, ec-coprocessor, warm BENCH_ITERS=5; 1T = RAYON_NUM_THREADS=1): - total cells: 11.25 M -> 2.70 M (x0.24) - prove_ms_median: MT 364 (baseline ~961), 1T 950 (baseline ~2584) - verify_ms_median: 100 (MT) / 163 (1T) - proof_bytes: 2.16 MB (baseline 2.39 MB) - stark_prove_profile warm-MT: tree0_commit 31 -> 8.9, tree2_write 90 -> 32, engine 155 -> 126, stark total 493 -> 344 ms - stark_prove_profile 1T: tree0_commit 215 -> 65, engine 560 -> 310, stark total 1469 -> 927 ms
…ith spec encoding The DoS shape-cap commit added eu_id_prover::Error::ShapeTooLarge but the SDK's map_prover_error match was not updated, breaking the workspace build. ShapeTooLarge is a verify-side rejection, so it maps to ZkError::Verify. Also update device_authentication_bytes_are_tag24_wrapped: it still asserted the old double-wrapped bstr(tag24(bstr)) DeviceNameSpacesBytes encoding that 5742256 corrected to #6.24(bstr .cbor DeviceNameSpaces) per 18013-5.
…ack deletion Commit 85c7136 deleted the vestigial SHA split-pack lookup tables (46 -> 14 preprocessed columns in stwo-sha256), moving the mdoc tree-0 preprocessed commitment. Recompute the TS13 conformance pins at this revision via ts13_evidence_pack_n1_measurements: - TS13_PUBLISHED_AGE_OVER_18_PREPROCESSED_ROOT -> bc8ad8a2f9440f6602d4d3b07772e8bc28d3b7858cc875ba60372df82a9babcf - TS13_PUBLISHED_AGE_OVER_18_CIRCUIT_HASH -> d09852c1343dcf595dadbe8f1003f16710dd732cba471e58dc8cc2e52b84fc96 ts13_published_fixed_table_fingerprints() fingerprints the preprocessed tree by its root (single mdoc_preprocessed_tree entry), not per-table content, so it follows the root repin automatically. Bump TS13_CIRCUIT_REVISION 2 -> 3: the split-pack deletion changed the SHA-256 constraint set, matching the established precedent (revision 2 was bumped for a SHA constraint change). The revision folds into the tuple hash, so revision is the semantic circuit-identity marker. Pins verified green: eu-id-prover lib ts13 (9), mdoc_support ts13 --include-ignored (18, incl. end-to-end prove/verify + STARK-bound negatives), sdk lib (44, delegates to prover defaults). clippy clean.
Behind an off-by-default jni feature; each wrapper calls the existing C-ABI bench fn and serializes its result struct to a JSON string. No measurement logic changed.
arm64-only, minSdk 26, cargo-ndk release with fat LTO. Runs the eu-id FFI bench suite (identity cold + identity/mdoc/p256/sha256), emits one JSON document with device metadata and thermal readings; Firebase Test Lab Game Loop contract via intent.data. Makefile plumbing for bench-android-apk.
…parsing (identity v2 soundness) MdocWindowBind bound preimage windows at prover-supplied offsets; nothing proved an offset pointed at the semantically correct CBOR field, so a malicious prover could aim elementIdentifier/valueDigest/deviceKey windows at attacker-controlled bytes inside the signed structure. Replace it with MdocCborStream (per-byte grammar proof of the accepted CBOR profile) and MdocScope (DFA structural walk keyed by (stream_slot, byte_index)), plus committed per-proof cyclic claim masks replacing the claimed-sum blinder. Known cost: mdoc trace cells 1.82M -> 8.28M (4.55x); reduction follows in the next commit (DFA edge table height decoupling).
…47M cells (-58%) The scope walk's height was forced to the DFA transition-table height by needed = max(active_rows, table_edges): 348 columns at log 14 where the byte walk itself needs ~2k rows. Move the edge-tuple preprocessed columns, the committed multiplicity column, and the table-side LogUp yield into a narrow MdocScopeDfaTable component (12 cols) at the edge-count height, verifier-derived from the statement's DFA programs alone; the walk now sizes by its own byte count (prover-carried, hard-bounded [9,20]). The shared MdocScopeDfaRelation is drawn once and cancels across the component boundary in the global claimed-sum balance; the table's claimed sum joins the cyclic claim-mask ring (multiplicities are parse-path private). Padding multiplicity rows stay freshly blinded, gated by the preprocessed dfa_active flag. New negatives: cross-component balance rejects a tampered multiplicity; honest table trace satisfies the component. Measured (shape_dump, 2-attr fixture): mdoc cells 8,281,600 -> 3,472,896 (scope 5,701,632 -> 892,928: walk 340 cols @ log 11 + table 12 cols @ log 14). Cold prove (1 thread, mdoc_perf_probe) 2,872 -> 1,975 ms; verify 241 -> 235 ms; proof size unchanged. TS13 pins regenerated for circuit revision 6 via the evidence-pack flow. Gate: cargo test -p eu-id-prover --release all-targets green (138 passed); ts13_evidence_pack_n1_measurements green with the new pins.
Accept only the version 8 compressed proof envelope and reconstruct the public statement from validated caller inputs. Preserve the classical P-256 signature, trust, device-binding, revocation, validity, request-context, and private predicate checks while removing inactive proof paths. Bind the release to regenerated source manifests and the audited LTO profile.
Use the audited release profile across CI, Android, JVM, iOS, and benchmark harnesses. Add deterministic build comparison scripts, keep 12 build and Rayon workers, compile the current product API, and ignore only scoped Gradle and Kotlin build caches.
Delete obsolete benches, vectors, CLI documentation, historical plans, and helper tools that no longer describe the sole product flow. Remove stale file references and ignore only local agent state and generated build caches.
Prover speed (cold single-prove, the EUDI presentation path): - Parallelize Ligero batched_row_weights over claims (bit-identical field reduce). - Parallelize CBOR base-trace generation over rows (identical columns). - Parallelize the one-time build_basis_sums table and hoist warm_circle_tables to the start of the coprocessor prove so the build runs on a free rayon pool instead of serializing behind the cache lock during the parallel row encode. Proof size: - Re-sweep Ligero geometry 2x (row_len 512, codeword 8192, product domain 4096, proximity 3070, openings unchanged at 196). Opened columns halve; envelope ~2.68MB -> ~2.03MB. Soundness stays ~2^-133 (re-derived + re-pinned). - Recalibrate the structured claim-evaluator threshold for the new row length and add the point-length guard its factored path requires. Verified: make check, all crate tests (incl. unlinkability + fixed-shape + soundness pins + tamper negatives), and Firebase Test Lab on Pixel 8 / S24 Ultra / A54 (cold prove -9..-20%, peak RSS <=857MiB, proof 1.94MB).
Pack the digest-id uniqueness universe two ids per row (log16 -> log15), halving the max trace height that drives the FRI domain while covering the same u16 id space. The booleanity + LogUp multiplicity argument and the claim-mask (unlinkability) are unchanged; the packed eval batches the two relation fractions and the mask via finalize_logup_in_pairs. Right-size the product capacity to the real PID credential (MSO 6KiB -> 3.5KiB, still ~2x headroom over real ~1-2KiB MSOs), which drops the scope walk to log14 and the CBOR parsers to log12, shrinking the composition/ FRI domain and the total trace cells. Desktop prove ~728 -> ~665ms; single-prove peak RSS ~776 -> ~618 MiB. Phone (Firebase, Pixel8/S24U/A54): warm prove -5..-12%, RAM -20%. Verified: make check, full adversarial/unlinkability/tamper suites, and a Firebase Test Lab matrix on real devices.
The packed SHA-256 is the largest trace. Size the selected-item cap to the real disclosed attributes (birth_date, nationality — both well under 128 bytes) at 256 bytes, which brings the packed message set to 117 blocks and lets the SHA trace drop from log14 to log13, halving the largest component. The MSO cap trims to 3328 to keep the block count within the log13 capacity. NOTE: the 256-byte selected-item cap excludes the PID portrait attribute (a JPEG, kilobytes). This product proves only birth_date + nationality predicates, so portrait disclosure is out of scope; revisit the cap if it ever enters scope. Desktop prove ~665 -> ~575ms; single-prove peak RSS ~618 -> ~574 MiB. Phone cold prove (Firebase): S24U ~1.25s, Pixel8 ~2.77s, A54 ~3.16s. Verified: make check, full adversarial/unlinkability/tamper suites, Firebase matrix.
…nces The single MacBatch instance (8 GF(2^128) MAC tags, 84 sequential sumcheck rounds) was the Ligero critical path. Split it into 8 parallel MacHalf sub-instances plus a small MacCanonicality instance, collapsing the wall-clock critical path. Each half's Fiat-Shamir channel mixes the shared full_root, av, its own tag, and a unique half-index before any challenge (independence for the symmetric halves); each half's claims map to its own group_a/group_b slice of the SAME Ligero commitments, so the claim-batch binding is unchanged. The verifier mirrors the identical 8-way split + ordered merge. Soundness preserved (the 4 requirements hold); new adversarial tests reject wrong-position claims, swapped half order, and canonicality tampering. Structured-claim threshold re-tuned (verifier-only, both paths identical). Desktop prove ~575 -> ~545ms; verify ~190 -> ~173ms (the split also parallelizes verification). Proof +42KB (8 extra sumcheck transcripts). Verified: make check, full adversarial/unlinkability/tamper suites.
…floor The repo's 128-bit figure (pow 20 + blowup 2 * queries 54) uses stwo's optimistic 2-bits-per-query accounting. Under the standard Johnson-bound correlated-agreement conjecture the honest floor is ~1 bit/query, so 54 queries gave only ~74 bits. Raising to 84 queries gives ~104 conjectured bits (+ 20 PoW grinding). Prove ~545 -> ~558ms; verify ~173 -> ~178ms; proof 2.07 -> 2.53MB (~15KB/query). Verified: make check, full adversarial/ unlinkability/tamper suites, end-to-end identity proofs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Scheme-agnostic build improvement, cherry-picked from the quantum branch where it measured −32.6% prove time (interleaved profile-isolated min-of-3, single-thread) plus −6-7% verify. The bench profile already pinned fat-LTO precisely because release lacked it — this aligns the shipped binaries with the benched ones.
[profile.release]:lto = "fat",codegen-units = 1(merged into the existing section;debug = trueretained)🤖 Generated with Claude Code