Skip to content

feat(mint): implement epoch-based proof of liabilities with compact merkle-sum trees#1046

Draft
a1denvalu3 wants to merge 1 commit into
mainfrom
pol-mint
Draft

feat(mint): implement epoch-based proof of liabilities with compact merkle-sum trees#1046
a1denvalu3 wants to merge 1 commit into
mainfrom
pol-mint

Conversation

@a1denvalu3

Copy link
Copy Markdown
Collaborator

Description

This PR implements the Mint and Wallet sides of the Proof of Liabilities (PoL) scheme using synchronized epoch-based Sparse Merkle Sum Trees (MS-SMT) and OpenTimestamps (OTS) attestations.

Key Features:

  1. Aggregated Single-OTS Attestations: Solves the keyset bloat problem by dynamically aggregating all unexpired keysets' tree roots into a single deterministic global digest. Only one single OpenTimestamps calendar request is made per epoch, drastically reducing server load.
  2. Compact Merkle Proofs: Implements a bitmask-based optimization that reduces proof size from ~20KB to < 1KB (95%+ bandwidth reduction) by omitting empty default nodes.
  3. Continuous Lifecycle Audits for Inactive Keysets: Because inactive keysets cannot mint new ecash but can still redeem/spent outstanding tokens, the background loop keeps publishing new epochs containing the frozen Issued Tree and the updated/growing Spent Tree until the keyset's final_expiry is reached.
  4. Wallet CLI Solvency Auditing: Added cashu pol manifest and cashu pol audit commands. The wallet can trustlessly audit all its unspent tokens against both the Spent Tree (proving non-inclusion/double-spend protection) and the Issued Tree (proving 100% mint liability backing).
  5. Testing & Hardening: Includes comprehensive unit and integration tests, complete with respx mock HTTP calendar failovers and automated DB-backed transition validations. Runs fully offline with optional mock OTS mode (MINT_POL_MOCK_OTS=TRUE).

@codecov

codecov Bot commented Jun 10, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
709 1 708 88
View the top 1 failed test(s) by shortest run time
tests.wallet.test_wallet_v1_api::test_mint_and_split_and_state_and_restore_paths
Stack Traces | 0.006s run time
monkeypatch = <_pytest.monkeypatch.MonkeyPatch object at 0x7fc7ae3e9240>
api = <cashu.wallet.v1_api.LedgerAPI object at 0x7fc7ae3e8d30>

    @pytest.mark.asyncio
    async def test_mint_and_split_and_state_and_restore_paths(monkeypatch, api: LedgerAPI):
        output = BlindedMessage(id="kid", amount=1, B_="ab")
        proof = Proof(
            id="kid", amount=1, C=PrivateKey().public_key.format().hex(), secret="s1"
        )
        cast(Any, api).keysets = {"kid": object()}
    
        requests = []
    
        async def fake_request(self, method, path, **kwargs):
            requests.append((method, path, kwargs))
            if path == "mint/bolt11":
                return _response(
                    200,
                    {
                        "signatures": [
                            {
                                "id": "kid",
                                "amount": 1,
                                "C_": PrivateKey().public_key.format().hex(),
                            }
                        ]
                    },
                )
            if path == "swap":
                return _response(
                    200,
                    {
                        "signatures": [
                            {
                                "id": "kid",
                                "amount": 1,
                                "C_": PrivateKey().public_key.format().hex(),
                            }
                        ]
                    },
                )
            if path == "checkstate":
                return _response(
                    200,
                    {"states": [{"Y": proof.Y, "state": "UNSPENT"}]},
                )
            if path == "restore":
                return _response(
                    200,
                    {
                        "outputs": [{"id": "kid", "amount": 1, "B_": "ab"}],
                        "signatures": [
                            {
                                "id": "kid",
                                "amount": 1,
                                "C_": PrivateKey().public_key.format().hex(),
                            }
                        ],
                    },
                )
            if path == "mint":
                return _response(
                    200,
                    {
                        "signatures": [
                            {
                                "id": "kid",
                                "amount": 1,
                                "C_": PrivateKey().public_key.format().hex(),
                            }
                        ]
                    },
                )
            raise AssertionError(f"Unexpected path {path}")
    
        monkeypatch.setattr(
            "cashu.wallet.v1_api.httpx.AsyncClient", lambda **kwargs: object()
        )
        monkeypatch.setattr(api, "_request", MethodType(fake_request, api))
    
        promises = await api.mint(outputs=[output], quote="q", signature="sig")
        assert len(promises) == 1
    
        split_promises = await api.split([proof], [output])
>       assert len(split_promises) == 1
E       AssertionError: assert 2 == 1
E        +  where 2 = len(([BlindedSignature(id='kid', amount=1, C_='021d39455b2c7234e661ebc83eb0d80b97425a9e7e7139d0757b5864198e23bea3', dleq=None, pol_receipt=None)], None))

tests/wallet/test_wallet_v1_api.py:464: AssertionError

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@a1denvalu3

Copy link
Copy Markdown
Collaborator Author

The protocol specification has been drafted and submitted as Draft Pull Request cashubtc/nuts#388.

Updates Included:

  • Formally defines the Sparse Merkle Sum Tree (MS-SMT) leaf calculations and parent node hashing in plain-English prose.
  • Outlines the Synchronized OpenTimestamps Aggregation steps.
  • Establishes the Signed Transactional Proof of Liability Receipts scheme on an individual, order-preserving note basis (using keyset-amount private keys), allowing users to hold the mint cryptographically accountable for each unspent promise or spent input.
  • Standardizes the JSON schemas for responses and the cryptographic Fraud Challenge format.

@a1denvalu3

Copy link
Copy Markdown
Collaborator Author

⚡ Tree-Level Caching for Solvency Audits (PoL)

Hi team,

I have implemented an optimized Tree-Level Caching solution on the mint side to solve the performance and potential Denial of Service (DoS) vulnerability associated with on-the-fly tree reconstruction.

💡 The Problem with Endpoint-Level Caching

Normally, we cache HTTP responses at the route level using RedisCache. However, when a wallet queries inclusion proofs via /v1/pol/.../proofs/issued or /v1/pol/.../proofs/spent, the POST request body contains that individual wallet's unique active/spent secrets. Because each wallet's token payload is unique, HTTP-level caching yields a 0% cache hit rate, forcing the mint to run database queries and rebuild the entire $2^{256}$-leaf Sparse Merkle Sum Tree in-memory for every single request.

🚀 The Solution: Tree-Level Caching

Since historical epoch trees are completely static and immutable once finalized, we can cache the constructed tree objects themselves instead of the final HTTP responses.

We implemented a two-tier caching architecture directly inside the core tree builder:

  1. Tier 1 (Redis): If Redis caching is enabled (settings.mint_redis_cache_enabled), completed trees are serialized via high-performance pickle and saved under Redis keys pol:tree:issued:{keyset_id}:{epoch_index} and pol:tree:spent:{keyset_id}:{epoch_index}. This ensures horizontally scaled mint instances share a single cluster-wide cache.
  2. Tier 2 (In-Memory Fallback): If Redis is disabled, we fallback to a process-local, thread-safe memory dictionary (_FALLBACK_MEM_CACHE) capped at 20 tree pairs to prevent unbounded memory growth.
  3. Pre-Warming: Newly published epochs are automatically pre-warmed/cached as soon as the manifest is generated in update_pol_manifests, so the very first wallet to audit gets an instant response.

With this optimization, database queries and tree-construction CPU loops are run exactly once per epoch rather than once per wallet!

I have also added robust unit & integration tests covering both the Redis-primary and fallback in-memory caching layers (test_build_trees_caching in tests/mint/test_mint_pol.py), and formatted/resolved type-checking checks for repository stability.

Let me know if you have any feedback!

@a1denvalu3 a1denvalu3 force-pushed the pol-mint branch 2 times, most recently from 360119b to 99a2f62 Compare June 27, 2026 08:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant