Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 103 additions & 5 deletions SPEC.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,11 @@ Primitives, in the exact order fields are listed for each message in §4:
and is time-zone independent. An instant outside the representable range of
`int64` UTC Unix nanoseconds (approximately the years 1678–2262) is out of scope
for v0 and MUST be treated as an encoding error rather than wrapped or clamped.
The reference encoder enforces this: `canon.Buffer.Time` appends nothing and
latches `canon.ErrTimeOutOfRange` (retrievable via `Buffer.Err()`) instead of
emitting a wrapped value, so two instants ~584 years apart can never alias to
identical bytes. A producer MUST reject via `canon.ValidTime` before signing;
a verifier/decoder MUST reject a message whose timestamp fails it.
- **repeated field** — an unsigned LEB128 varint *count* (the number of elements)
encoded FIRST, then each element's fields encoded in order, fully inline, using
the normal primitive rules above. A repeated element that is itself a string is
Expand All @@ -127,7 +132,7 @@ no leading version byte, no total-length prefix wrapping the message, and no
trailing bytes after the last field. The `Order:` list in §4 is exhaustive and
authoritative — the message's **signature field(s)** are the ONLY fields
excluded, and no field exists in the canonical bytes that is not named there.
For the six core messages of §4 that single excluded field is `Signature`. The
For the twelve core messages of §4 that single excluded field is `Signature`. The
Layer-C operator messages of §11 carry **two** signatures (`Sig0` and `Sig1`)
over the same canonical bytes; both are excluded exactly as `Signature` is here,
and, like `Signature`, neither appears in an `Order:` clause. A verifier
Expand Down Expand Up @@ -192,9 +197,14 @@ Domain tag: `sohocloud/assignment/v0`

Order: `JobID` (string), `NodeID` (string), `Spec.Workload` (string),
`Spec.Image` (string), `Spec.Args` (repeated string), `Spec.PrinterKind`
(string), `Fee.ContributorShareBps` (int64), `Fee.PlatformFeeBps` (int64),
(string), `Spec.GPUAPI` (string), `Spec.GPUMinVRAMMB` (int64),
`Fee.ContributorShareBps` (int64), `Fee.PlatformFeeBps` (int64),
`OfferedAt` (time).

`Spec.GPUAPI` ∈ {``, `vulkan`, `nnapi`, `cuda`, `metal`} — an advisory routing
hint like `PrinterKind`; empty means no GPU is required and `GPUMinVRAMMB`
MUST then be `0`.

The fee terms are attached inline so a node sees the split before it commits.
In the pull model there is no separate accept message.

Expand Down Expand Up @@ -224,6 +234,94 @@ Order: `CoordinatorID` (string), `Terms.ContributorShareBps` (int64),

`ContributorShareBps + PlatformFeeBps` SHOULD equal `10000`.

### 4.7 StorageLease — signed by the COORDINATOR
Domain tag: `sohocloud/lease/v0`

Order: `LeaseID` (string), `NodeID` (string), `ShardRef` (bytes),
`SizeClass` (int64), `Fee.ContributorShareBps` (int64),
`Fee.PlatformFeeBps` (int64), `IssuedAt` (time), `ExpiresAt` (time),
`Seq` (uint64).

Storage is a LEASE, not a job: an ongoing obligation to hold one sealed
shard for a bounded term. `ShardRef` is an opaque 32-byte content address
and `SizeClass` a quantized payload size — the protocol MUST NOT carry true
object sizes or stored content; encryption, padding, and sharding are
frontend concerns above this waist. Renewal is a new `StorageLease` for the
same `LeaseID` with strictly higher `Seq` (same rollback rule as §4.1). Fee
terms ride inline, as in §4.3.

### 4.8 LeaseDecline — signed by the NODE
Domain tag: `sohocloud/lease-decline/v0`

Order: `LeaseID` (string), `NodeID` (string), `Reason` (string),
`DeclinedAt` (time).

`Reason` ∈ {`local_policy`, `capacity`, `unavailable`}; `local_policy` is the
opt-out path (§5.1) for storage exactly as §4.4 is for jobs.

### 4.9 LeaseRelease — signed by the NODE
Domain tag: `sohocloud/lease-release/v0`

Order: `LeaseID` (string), `NodeID` (string), `ReleasedAt` (time).

A node may always stop holding (sovereignty includes leaving). The signed
release ends its metering; re-placing the shard is the frontend's concern.

### 4.10 ProofChallenge / ProofResponse — response signed by the NODE
Domain tag (response): `sohocloud/proof/v0`

A `ProofChallenge{LeaseID, Offset, Length, Nonce (16 bytes), IssuedAt}` is
NOT signed: nodes fetch challenges by polling over the authenticated channel
(pull model), and a challenge commits no one to anything. The signed
artifact is the response.

Response Order: `LeaseID` (string), `NodeID` (string), `Offset` (int64),
`Length` (int64), `Nonce` (bytes), `Digest` (bytes), `RespondedAt` (time).

`Digest` MUST be computed exactly as
`SHA-256(Nonce || uint64be(Offset) || uint64be(Length) || sealed[Offset : Offset+Length])`
over the sealed shard bytes. The response restates the challenged range and
nonce so it stands alone as a non-repudiable metering fact — the storage
counterpart of §4.5. A verifier MUST reject a response whose
`(LeaseID, Nonce)` it has already accepted: nonces are single-use, which is
what defeats replaying a recorded answer.

### 4.11 KeyRotation — signed by the node's CURRENT key
Domain tag: `sohocloud/node-rotate/v0`

Order: `NodeID` (string), `NewPublicKey` (bytes), `Algo` (string),
`RotatedAt` (time), `Seq` (uint64).

Possession of the outgoing key authorizes naming its successor. A verifier
MUST validate the signature against the key it currently holds for the node
(a rotation signed by the successor is self-installation and MUST fail),
MUST enforce strictly monotonic `Seq` per node, and MUST reject a rotation
whose `NewPublicKey` is not a well-formed key for `Algo` (for `ed25519`, not
exactly 32 bytes) or whose `Algo` is unknown — otherwise a verifier that
begins checking subsequent messages against a malformed `NewPublicKey` would
fault. Only after all of these pass does the verifier begin verifying
subsequent node messages against `NewPublicKey`. `Algo` ∈ {`ed25519`} in v0.

### 4.12 KeyRevocation — signed by the key being REVOKED
Domain tag: `sohocloud/node-revoke/v0`

Order: `NodeID` (string), `RevokedPublicKey` (bytes), `RevokedAt` (time),
`Seq` (uint64).

Kills a key with no successor. A verifier MUST verify the signature against
the key it **currently trusts** for `NodeID`, and MUST require that
`RevokedPublicKey` equals that trusted key. A revocation is only ever honored
against the exact key it names and kills. This is the anti-abuse binding: a
stranger can self-sign a revocation carrying a victim's `NodeID` and the
stranger's own key in `RevokedPublicKey`, but because that key is not the one
the coordinator trusts for the victim, the equality check fails and the
revocation is rejected — a node cannot be knocked offline by anyone who does
not already hold its current key. (Earlier drafts said "honor unconditionally";
that wording is superseded — the binding to the trusted key is mandatory.)
Given the current key, the safe reading of a revocation is "stop trusting this
key". Re-enrollment after revocation is out-of-band, via the same path as
first enrollment, never a wire message a key thief could forge.

---

## 5. Invariants
Expand Down Expand Up @@ -357,7 +455,7 @@ JSON first.
`MaxUint64`; both bools; the empty string and a multibyte UTF-8 string; and
`time` at the Unix epoch and at a fixed nanosecond instant.

- **`messages`** — one entry per signed message type (all six of §4). Each entry
- **`messages`** — one entry per signed message type (every signed message of §4). Each entry
records: `name`, `domain_tag`, `signer` (`node` or `coordinator`), the 32-byte
ed25519 `seed_hex` and the derived `public_key_hex`, the message `fields`, the
`canonical_bytes_hex` (the output of `CanonicalBytes()`, `Signature` excluded),
Expand Down Expand Up @@ -443,7 +541,7 @@ canonical.
orthogonal identity: the **operator** — a frontend (e.g. Cloudy) that terminates
member and node identity inside itself and presents a single rotating credential
to a coordinator. It is deliberately separate from node workload identity (§2):
the six core coordination messages (§4) are UNCHANGED and are NOT signed with an
the core coordination messages (§4) are UNCHANGED and are NOT signed with an
operator credential.

An operator holds **seven** Ed25519 keypairs (indices `0..6`) and signs each
Expand All @@ -468,7 +566,7 @@ only.
- **Migration seam.** The reference implementation routes operator signing and
verification through a `Signer`/`Verifier` interface, implemented with stdlib
Ed25519 today, precisely so the algorithm can be swapped by adding a `v1`
domain tag later without touching the six core messages. This is an
domain tag later without touching the core messages. This is an
implementation note; the wire contract is the field orders and tags below.

**Encoding of operator fields (no special casing).** Every field in the §11
Expand Down
66 changes: 64 additions & 2 deletions canon/canon.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,43 @@
package canon

import (
"crypto/ed25519"
"encoding/binary"
"errors"
"time"
)

// ErrTimeOutOfRange is latched on a Buffer when Time is given an instant
// outside the representable int64-UTC-nanoseconds range (SPEC §3). It is an
// encoding error, never a wrap or clamp.
var ErrTimeOutOfRange = errors.New("canon: time out of representable int64-nanosecond range")

// minRepresentableUnixNano / maxRepresentableUnixNano bound the instants that
// encode without int64 nanosecond overflow (approximately 1678..2262).
var (
minRepresentableUnixNano = time.Unix(0, minInt64)
maxRepresentableUnixNano = time.Unix(0, maxInt64)
)

const (
minInt64 = -1 << 63
maxInt64 = 1<<63 - 1
)

// ValidTime reports whether t encodes as int64 UTC Unix nanoseconds without
// overflow. Producers MUST validate timestamps with this before signing, and
// verifiers/decoders MUST reject a message whose timestamp fails it — an
// out-of-range instant is an encoding error, not a value to be wrapped
// (SPEC §3).
func ValidTime(t time.Time) bool {
u := t.UTC()
return !u.Before(minRepresentableUnixNano) && !u.After(maxRepresentableUnixNano)
}

// Buffer accumulates canonical bytes.
type Buffer struct {
b []byte
b []byte
err error
}

// New returns a Buffer whose first entry is a domain tag identifying the
Expand Down Expand Up @@ -86,13 +116,45 @@ func (buf *Buffer) Uint64(v uint64) *Buffer {
// monotonic-clock components are dropped so the encoding is stable across a
// JSON transport round-trip and across time zones.
func (buf *Buffer) Time(t time.Time) *Buffer {
if !ValidTime(t) {
// Latch an encoding error and append nothing rather than wrapping —
// a wrapped value would silently alias two instants ~584 years apart
// to identical bytes (SPEC §3). Callers surface this via Err().
if buf.err == nil {
buf.err = ErrTimeOutOfRange
}
return buf
}
return buf.Int64(t.UTC().UnixNano())
}

// Err returns the first encoding error latched during construction (currently
// only ErrTimeOutOfRange), or nil. A caller that builds canonical bytes from
// untrusted field values (a verifier or transport decoder) MUST check Err()
// and reject the message before trusting Sum().
func (buf *Buffer) Err() error {
return buf.err
}

// Sum returns a copy of the accumulated canonical bytes. The caller signs or
// verifies over exactly these bytes.
// verifies over exactly these bytes. When Err() is non-nil the returned bytes
// are incomplete by construction (the offending field was skipped, not
// wrapped), so any signature over them will fail to verify — callers should
// check Err() rather than relying on that.
func (buf *Buffer) Sum() []byte {
out := make([]byte, len(buf.b))
copy(out, buf.b)
return out
}

// VerifySig reports whether sig is a valid ed25519 signature by pub over msg.
// Unlike calling ed25519.Verify directly it is panic-safe: it rejects (returns
// false) when pub is not ed25519.PublicKeySize or sig is not
// ed25519.SignatureSize, instead of panicking. Every message Verify method
// routes through here so a malformed key on the wire can never crash a
// verifier.
func VerifySig(pub ed25519.PublicKey, msg, sig []byte) bool {
return len(pub) == ed25519.PublicKeySize &&
len(sig) == ed25519.SignatureSize &&
ed25519.Verify(pub, msg, sig)
}
44 changes: 44 additions & 0 deletions canon/canon_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package canon

import (
"crypto/ed25519"
"testing"
"time"
)

func TestVerifySigPanicSafe(t *testing.T) {
pub, priv, _ := ed25519.GenerateKey(nil)
msg := []byte("hello")
sig := ed25519.Sign(priv, msg)
if !VerifySig(pub, msg, sig) {
t.Fatal("valid signature rejected")
}
// Wrong-length key must return false, not panic.
if VerifySig([]byte{1, 2, 3}, msg, sig) {
t.Fatal("accepted a malformed public key")
}
// Wrong-length signature must return false, not panic.
if VerifySig(pub, msg, []byte{4, 5}) {
t.Fatal("accepted a malformed signature")
}
}

func TestValidTimeAndLatch(t *testing.T) {
if !ValidTime(time.Unix(1_700_000_000, 0)) {
t.Fatal("a normal instant reported out of range")
}
// Far outside the int64-nanosecond window (year ~5000).
far := time.Date(5000, 1, 1, 0, 0, 0, 0, time.UTC)
if ValidTime(far) {
t.Fatal("an out-of-range instant reported valid")
}
b := New("test/domain").String("x").Time(far).Uint64(7)
if b.Err() == nil {
t.Fatal("out-of-range Time did not latch an error")
}
// In-range time must not latch.
b2 := New("test/domain").Time(time.Unix(1, 0))
if b2.Err() != nil {
t.Fatalf("in-range Time latched an error: %v", b2.Err())
}
}
38 changes: 38 additions & 0 deletions coordinator/coordinator.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import (
"github.com/NTARI-RAND/sohocloud-protocol/employment"
"github.com/NTARI-RAND/sohocloud-protocol/fees"
"github.com/NTARI-RAND/sohocloud-protocol/identity"
"github.com/NTARI-RAND/sohocloud-protocol/lease"
"github.com/NTARI-RAND/sohocloud-protocol/listing"
"github.com/NTARI-RAND/sohocloud-protocol/liveness"
)
Expand Down Expand Up @@ -48,3 +49,40 @@ type Coordinator interface {
// Fees returns the coordinator's current signed fee declaration.
Fees(ctx context.Context) (fees.FeeDeclaration, error)
}

// StorageCoordinator is the OPTIONAL storage-lease surface. It is a separate
// capability interface, not new methods on Coordinator, so a coordinator can
// adopt the compute/print protocol without implementing storage yet — a
// consumer discovers support by type assertion and treats its absence as
// "this coordinator does not lease storage", never as an error. The same
// pull model and SPIFFE-binding discipline apply throughout.
type StorageCoordinator interface {
// PollLeases returns the storage leases currently offered to the calling
// node, and the open proof challenges against its held leases. Identity
// binding rules are identical to PollJobs.
PollLeases(ctx context.Context, id identity.NodeID) ([]lease.StorageLease, []lease.ProofChallenge, error)

// DeclineLease records a node's signed refusal of an offered lease.
DeclineLease(ctx context.Context, d lease.LeaseDecline) error

// ReleaseLease records a node's signed early exit from a held lease.
ReleaseLease(ctx context.Context, r lease.LeaseRelease) error

// SubmitProof records a node's signed proof of possession — the fact from
// which storage metering and payout derive. Implementations MUST reject a
// response whose (LeaseID, Nonce) was seen before: nonces are single-use.
SubmitProof(ctx context.Context, p lease.ProofResponse) error
}

// KeyLifecycleCoordinator is the OPTIONAL node-key rotation surface, split
// out for the same staged-adoption reason as StorageCoordinator.
type KeyLifecycleCoordinator interface {
// RotateKey verifies k against the node's CURRENT key, enforces strictly
// monotonic Seq, and thereafter trusts only k.NewPublicKey for the node.
RotateKey(ctx context.Context, k identity.KeyRotation) error

// RevokeKey kills a node key with no successor. Implementations MUST
// honor a validly signed revocation unconditionally; re-enrollment is
// out-of-band.
RevokeKey(ctx context.Context, k identity.KeyRevocation) error
}
21 changes: 11 additions & 10 deletions employment/employment.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,10 +18,12 @@ import (
// advisory routing hints, not enforcement — opt-out enforcement is local to the
// node (see Decline / listing.WorkloadOptIn).
type JobSpec struct {
Workload string // "compute" | "print" | "storage"
Image string
Args []string
PrinterKind string // set for print workloads; empty otherwise
Workload string // "compute" | "print" | "storage"
Image string
Args []string
PrinterKind string // set for print workloads; empty otherwise
GPUAPI string // advisory hint: "vulkan" | "nnapi" | "cuda" | "metal"; empty = no GPU required
GPUMinVRAMMB int64 // advisory minimum VRAM for GPU work; 0 when GPUAPI is empty
}

// Assignment is a coordinator's signed offer of a specific job to a specific
Expand Down Expand Up @@ -93,6 +95,8 @@ func (a Assignment) CanonicalBytes() []byte {
b.String(arg)
}
b.String(a.Spec.PrinterKind)
b.String(a.Spec.GPUAPI)
b.Int64(a.Spec.GPUMinVRAMMB)
b.Int64(int64(a.Fee.ContributorShareBps))
b.Int64(int64(a.Fee.PlatformFeeBps))
b.Time(a.OfferedAt)
Expand All @@ -107,8 +111,7 @@ func (a *Assignment) Sign(priv ed25519.PrivateKey) {
// Verify reports whether Signature is a valid coordinator signature over the
// assignment.
func (a Assignment) Verify(pub ed25519.PublicKey) bool {
return len(a.Signature) == ed25519.SignatureSize &&
ed25519.Verify(pub, a.CanonicalBytes(), a.Signature)
return canon.VerifySig(pub, a.CanonicalBytes(), a.Signature)
}

// CanonicalBytes returns the deterministic signing payload for the decline,
Expand All @@ -129,8 +132,7 @@ func (d *Decline) Sign(priv ed25519.PrivateKey) {

// Verify reports whether Signature is a valid node signature over the decline.
func (d Decline) Verify(pub ed25519.PublicKey) bool {
return len(d.Signature) == ed25519.SignatureSize &&
ed25519.Verify(pub, d.CanonicalBytes(), d.Signature)
return canon.VerifySig(pub, d.CanonicalBytes(), d.Signature)
}

// CanonicalBytes returns the deterministic signing payload for the job report,
Expand All @@ -154,6 +156,5 @@ func (r *JobReport) Sign(priv ed25519.PrivateKey) {

// Verify reports whether Signature is a valid node signature over the report.
func (r JobReport) Verify(pub ed25519.PublicKey) bool {
return len(r.Signature) == ed25519.SignatureSize &&
ed25519.Verify(pub, r.CanonicalBytes(), r.Signature)
return canon.VerifySig(pub, r.CanonicalBytes(), r.Signature)
}
Loading
Loading