Skip to content

port QuarkChain slavechain (MinorBlock) types, storage and chain plumbing#10

Open
blockchaindevsh wants to merge 12 commits into
masterfrom
port_qkc
Open

port QuarkChain slavechain (MinorBlock) types, storage and chain plumbing#10
blockchaindevsh wants to merge 12 commits into
masterfrom
port_qkc

Conversation

@blockchaindevsh

@blockchaindevsh blockchaindevsh commented Jun 7, 2026

Copy link
Copy Markdown

Addresses #1 and #7.

Summary

This PR makes goshard/base support the QuarkChain slavechain
(shardchain)
block and header as defined in goquarkchain,
ported as a parallel qkc/ package tree (~14k lines, 75 Go files). The
port is byte-compatible: headers and blocks serialize and hash exactly as in
goquarkchain (keccak256(serialize(header)), not RLP), and the storage
layer uses the identical on-disk key schema.

The root chain is not implemented here — it keeps running on the
original goquarkchain code; root block types are ported only as passive
dependencies and PrevRootBlockHash anchoring is validated against externally supplied
root blocks. Transaction execution is intentionally deferred (see Next
steps
).

What's included

Package Content
qkc/serialize QuarkChain's reflection-based binary codec
(verbatim, stdlib-only)
qkc/account Branch (fullShardId codec), Address, Identity +
python golden testdata
qkc/common (+hexutil) helpers, token-id codec, QKC hexutil fork
(lenient leading zeros)
qkc/params, qkc/config QKC EVM constants;
cluster/chain/shard/root configuration (JSON-identical)
qkc/types MinorBlock/MinorBlockHeader/MinorBlockMeta,
Transaction+EvmTransaction, Receipt, Log, Bloom,
TokenBalances, CalculateMerkleRoot, passive RootBlock
qkc/core/rawdb minor/root block accessors, canonical/head pointers
— on-disk compatible
qkc/consensus (+doublesha256) CommonEngine framework, difficulty
calculator, real double-sha256 PoW
qkc/core structural MinorBlockChain, genesis,
MinorBlockValidator
cmd/qkcshard standalone multi-shard runner (goquarkchain-slave
shape) + README

Byte/disk compatibility proof

All locked by goquarkchain's own golden vectors, ported with the code:
header hash b0b7dfab…9153d869, the 479-byte header size, the
token-trie root 0x848d4271…, CalculateMerkleRoot, RootBlock hash + signature, full
round-trips for every type, and the python-generated account/config
testdata.

Architecture

Parallel tree with minimal sharing: only geth's type-agnostic layers
are imported (crypto, rlp, common, ethdb, trie/triedb, log,
event, lru, params.ChainConfig as a type). geth's mainnet core/*,
consensus, Engine API, eth and miner are untouched, keeping the tree
upstream-mergeable.
Compile-level adaptations only (e.g. sha3.NewKeccak256
crypto.NewKeccakState,
modern trie constructors); no logic or format changes.

Entry point

go run ./cmd/qkcshard --mine --blocks 3                          #
in-memory demo
go run ./cmd/qkcshard --datadir /data/slave0 --fullshardid 2,3 --mine

One database per shard (/shard-0xN), idempotent genesis, engine
selected per shard from CONSENSUS_TYPE (real PoW for POW_DOUBLESHA256,
timed production otherwise). See cmd/qkcshard/README.md.

Testing

go test ./qkc/... — 8 packages green (golden vectors, rawdb round-trips,
genesis determinism, import/side-fork/orphan handling, database reopen,
real PoW mine-and-verify). go build ./... stays green tree-wide.

Next steps: integrating geth StateDB & EVM

Execution is the follow-up work item; every removed piece is marked
TODO(execution-issue) (20 seams). geth's core/state and core/vm are
type-agnostic and will be imported, not copied:

  1. State wiring: add stateCache to MinorBlockChain via
    state.NewMPTDatabase(triedb.NewDatabase(db, nil), state.NewCodeDB(db));
    implement State()/StateAt(root); genesis state-from-alloc replacing the
    empty-root placeholder; two-step persistence
    (statedb.Commit(...) + triedb.Commit(root, ...)) with trie GC in
    WriteBlockWithState.
  2. EVM execution: a qkc/core state processor — one vm.NewEVM per block
    with a vm.BlockContext built from MinorBlockHeader; per tx convert
    EvmTransaction → core.Message → core.ApplyMessage, assemble qkc
    Receipts; engine Finalize credits the coinbase.
  3. Validation completion: the seamed ValidateState checks (gas used,
    xshard cursor, coinbase amount, state root) and reorg-with-state.
  4. Pending decision — account/token model: single native token (reuse
    geth state unmodified) vs faithful multi-token semantics (fork core/state
    under qkc/ plus the QKC native-token precompiles and fullShardKey-aware
    CreateAddress). Block/header hashing is unaffected either way.
  5. Afterwards: txpool + miner template (blocks with real transactions),
    PoSW difficulty adjustment.

Note: goquarkchain's testBranch.json golden-vector file (160k
brute-force random vectors per shard size, 33MB) is deterministically sampled to 100
entries per shard size (1,600 total, all 16 shard sizes covered); the full
set was validated green once during the port.

blockchaindevsh and others added 9 commits June 7, 2026 09:29
Verbatim port of github.com/QuarkChain/goquarkchain/serialize, the
reflection-based binary codec used (instead of RLP) for all slavechain
headers, blocks and storage records. Pure stdlib, zero adaptations.
Includes the original golden byte-vector tests.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- account: Branch (fullShardId codec), Address (recipient+fullShardKey),
  Identity, keystore Account; python-generated golden testdata included.
  Adaptation: pborman/uuid -> google/uuid (only uuid in this module).
  goquarkchain's testBranch.json (160k brute-force random vectors per
  shard size, 33MB) is deterministically sampled to 100 entries per
  shard size (1600 total, all 16 sizes kept); the full set was validated
  green once during the port.
- common: IsP2/IntLeftMostBit/token id codec and misc helpers; includes
  the QuarkChain hexutil fork (leading-zero checks disabled), which the
  cluster config JSON format depends on.
- params: DenomsValue, DefaultConstantinople and QKC EVM constants.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Faithful port of cluster/config: QuarkChainConfig, ChainConfig,
ShardConfig/ShardGenesis/Allocation, RootConfig (passive dependency of
the shard code), SlaveConfig, PoW/PoSW configs. JSON format is identical;
verified against the python-generated cluster_config_template.json.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Byte-compatible port of goquarkchain/core/types: MinorBlock /
MinorBlockHeader / MinorBlockMeta, Transaction + EvmTransaction (RLP
inner encoding with Uint32 shard keys and token ids), Receipt, Log,
Bloom, TokenBalances, CrossShardTransactionDeposit, CalculateMerkleRoot
(custom keccak merkle tree) and DeriveSha, plus RootBlock/RootBlockHeader
as passive dependencies of the shard code.

Header hashing is keccak256(serialize(header)), not RLP. Adaptations are
compile-level only: sha3.NewKeccak256 -> crypto.NewKeccakState, the
ephemeral receipt trie and the TokenBalances token trie use the modern
trie/triedb API (trie.NewEmpty / trie.NewStateTrie).

Byte-parity is locked by the original golden vectors: header hash
b0b7dfab..., the 479-byte header size, the token-trie root 848d4271...,
CalculateMerkleRoot and full round-trips for every type.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On-disk compatible port of goquarkchain/core/rawdb: the exact key schema
(h/mn/b/r/LastBlock/...), minor & root block/header/receipt accessors,
canonical hash and head pointers, tx lookup entries, cross-shard deposit
lists, commit markers and chain config storage.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CommonEngine (header verification, local multi-threaded sealing, remote
sealer plumbing), EthDifficultyCalculator, FakeEngine for simulated
production, and the double-sha256 proof-of-work engine. Adaptations:
hashicorp/golang-lru -> geth common/lru (typed); Finalize signature uses
geth core/state (the execution layer operates on geth state). PoSW
construction is left to the execution work.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Structural subset of goquarkchain's shard chain, faithful where it does
not require the execution layer:

- Genesis: CreateRootBlock/CreateMinorBlock/SetupGenesisMinorBlock with
  original signatures; the genesis state-from-alloc is an execution seam
  and the block carries the empty trie root for now.
- MinorBlockValidator: full structural ValidateBlock (header rules, PoW
  seal, difficulty, linkage, TxHash/MetaHash recomputation, root-chain
  anchoring); ValidateState keeps the receipt-derived checks.
- MinorBlockChain: storage, canonical head management, structural
  InsertChain (validate -> persist -> advance head; side forks stored),
  future blocks, root-block passive reads, coinbase amount calculator.

Every removed execution-dependent piece is marked TODO(execution-issue):
state processing, State()/StateAt(), trie GC, PoSW, txpool and
reorg-with-state plug back into these seams.

End-to-end tests cover genesis determinism, import/side-fork/orphan
handling, database reopen and real double-sha256 mining.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Hosts one or more shards in a single process (the goquarkchain-slave
shape): per-shard pebble database under <datadir>/shard-0xN (the qkc
rawdb keys are per-chain singletons), idempotent genesis anchored to the
configured root block, per-shard engine selected from CONSENSUS_TYPE
(POW_DOUBLESHA256 = real PoW sealing, otherwise timed simulated
production) and parallel mining with graceful shutdown. Includes README.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The reviewed plan for porting QuarkChain's slavechain into this tree:
scope and boundaries (byte-compatible types, structural chain plumbing,
execution deferred, root chain external), the geth code-sharing
inventory and the phased implementation that landed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@qzhodl

qzhodl commented Jun 8, 2026

Copy link
Copy Markdown

Two things for this PR:

  • Too big to review well. It splits cleanly into a dependency-ordered stack of smaller PRs (serialize → common/account/config → types → rawdb → consensus → core), each with its own tests.

  • Mining shouldn't live in main. Both pyquarkchain and goquarkchain keep it as a separate module (internal miner + external RPC miner), not inlined into the entry point. The current cmd/qkcshard looks more like an e2e test.

blockchaindevsh and others added 3 commits June 8, 2026 21:36
Port of goquarkchain/consensus/simulate: a consensus.Engine whose seal
search sleeps to the configured block interval (and always verifies),
instead of doing real hashing. This lets non-PoW shards produce blocks
through the same engine interface as real PoW, so the miner can drive
both uniformly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Port of goquarkchain/cluster/miner: the MinerAPI contract
(CreateBlockToMine/InsertMinedBlock/GetTip/...) and the Miner that drives
production through the consensus engine — the internal commit->seal->result
loop plus the external GetWork/SubmitWork RPC mining path. Engine-agnostic:
simulated and real-PoW shards flow through the same loop. Adaptation: the
cluster service.ServiceContext dependency is dropped (mining is gated on
IsMining alone). Includes a test that mines blocks via double-sha256.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Addresses review feedback that mining should not live in the entry point.
The runner now implements miner.MinerAPI and forwards chain-head events to
the miner; the inline mining/sealing loop is gone. The per-shard engine is
selected from CONSENSUS_TYPE (doublesha256 for POW_DOUBLESHA256, otherwise
the simulate engine), so the entry point only wires things up. README
updated accordingly.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants