Auto-Generate Protocol Types

[smell-of-curry]

The json/ exports are very close to being a usable source-of-truth for code generation, This pr is an auto-generator for protocol types for gophertunnel #370). However two concrete gaps are currently blocking that work. Both look fixable on the export side without changing the underlying protocol.

1. Optional fields are not representable in the current schema

The wire format uses brstd::optional<T> / std::optional<T> for many fields (a single byte presence flag followed by T only when present). In the JSON exports, those fields are emitted as ordinary properties and listed in "required", which is indistinguishable from a non-optional field.

Example — json/CameraInstructionPacket.json (r/21_u13, protocol 975), definition 7894722 (CameraInstruction):

"required": [
  "AttachToEntity", "Clear", "DetachFromEntity", "Fade",
  "FieldOfView", "RemoveTarget", "Set", "Spline", "Target"
]

The actual packet has every one of those wrapped in brstd::optional (matching changelogs back to r/21_u2 #12 and r/21_u3 #13, and the C++ source). A generator that trusts the JSON produces incorrect Marshal/Unmarshal code for this packet — and dozens of others (CameraPreset, CameraAimAssist*, DebugDrawerShape, etc.).

2. Several packets are missing from json/

The json/ directory currently contains 191 packet files, but a number of packets referenced in the changelogs and present in the HTML/dot exports have no JSON counterpart.

3. Minor: "x-format-version": "MISSING VERSION"

Several files (including CameraInstructionPacket.json at protocol 975) ship with "x-format-version": "MISSING VERSION". If the format version is meant to be stamped at export time, it looks like that step is failing. A real value here would let downstream tools gate on schema changes.

I have reached out to @JakeShirley regarding this:

Remaining Tasks:

• Properly handle the rest of the TODO's (requires handling of the complex mapping)
• Figure out lack of ids
• Remove all old protocol docs
• Make a clean way to share types between (currently it just creates them again in same file)

[Happy2018new]

It seems that the documents from mojang not support optional data type.
The following are some examples.

Debug Drawer Packet

Yours

func (x *ShapeDataPayload) Marshal(r protocol.IO) {
	r.Varuint64(&x.NetworkID)
	r.Uint8(&x.ShapeType)
	r.Vec3(&x.Location)
	r.Float32(&x.Scale)
	r.Vec3(&x.Rotation)
	r.Float32(&x.TotalTimeLeft)
	r.Int32(&x.Color)
	r.Varint32(&x.DimensionID)
	switch val := x.ExtraShapeData.(type) {
	case nil:
		var typeID uint32 = 0
		r.Uint32(&typeID)
		_ = val // nil value, nothing more to write
	}
}

But the real

// Marshal ...
func (x *DebugDrawerShape) Marshal(io IO) {
	io.Varuint64(&x.NetworkID)
	OptionalFunc(io, &x.Type, io.Uint8)
	OptionalFunc(io, &x.Location, io.Vec3)
	OptionalFunc(io, &x.Scale, io.Float32)
	OptionalFunc(io, &x.Rotation, io.Vec3)
	OptionalFunc(io, &x.TotalTimeLeft, io.Float32)
	OptionalFunc(io, &x.Colour, io.BEARGB)
	io.Varint32(&x.DimensionID)
	io.ShapeData(&x.ExtraShapeData)
}

Camera Instruction Packet

Yours

func (x *CameraInstructionEntry) Marshal(r protocol.IO) {
	protocol.Single(r, &x.Set)
	r.Bool(&x.Clear)
	protocol.Single(r, &x.Fade)
	protocol.Single(r, &x.Target)
	r.Bool(&x.RemoveTarget)
	protocol.Single(r, &x.FieldOfView)
	protocol.Single(r, &x.Spline)
	r.Int64(&x.AttachToEntity)
	r.Bool(&x.DetachFromEntity)
}

But the real

func (pk *CameraInstruction) Marshal(io protocol.IO) {
	protocol.OptionalMarshaler(io, &pk.Set)
	protocol.OptionalFunc(io, &pk.Clear, io.Bool)
	protocol.OptionalMarshaler(io, &pk.Fade)
	protocol.OptionalMarshaler(io, &pk.Target)
	protocol.OptionalFunc(io, &pk.RemoveTarget, io.Bool)
	protocol.OptionalMarshaler(io, &pk.FieldOfView)
	protocol.OptionalMarshaler(io, &pk.Spline)
	protocol.OptionalFunc(io, &pk.AttachToEntity, io.Int64)
	protocol.OptionalFunc(io, &pk.DetachFromEntity, io.Bool)
}

[TwistedAsylumMC]

Unfortunately the JSON docs still have a lot of discrepancies between what they say and what is actually required, as pointed out by Happy with a few examples. On top of this, there's still a large amount of packets not in the JSON format. Until all this issues are resolved, we will not be generating anything. If you're going to continue working on this, then it can stay open as a draft, but if you do not plan on turning this into something usable in the near future then I believe we should close this and come back to it at a better time

[HashimTheArab]

It can be done using prismarine js minecraft-data which has reliable full json definitions for packets and protocol
https://github.com/PrismarineJS/minecraft-data

Heres an example of that
https://github.com/axolotl-stack/axolotl-stack/tree/main/crates/valentine_gen

[EanBee]

The issue is still fully relevant. All three structural blockers cited remain unresolved in the current Mojang protocol exports, even though Mojang has made incremental progress and the protocol has advanced several releases.

What's changed since the comments (Dec 2025 → now)

The json/ exports come from Mojang/bedrock-protocol-docs, and Mojang has been actively working on them:

• Protocol advanced: The PR referenced protocol 975 (r/21_u13 era). main is now protocol 1001, MC 1.26.30 (latest json/ commit r/26_u3 protocol documentation (#30), 2026-05-20; changelog changelog_1001_05_18_26.md). Four new release branches exist: r/26_u0 … r/26_u3.
• More packets converted: json/ grew from 191 → 203 *Packet.json files. The changelog shows an ongoing "Converted to Cereal" migration (e.g. BossEventPacket: Converted to Cereal, broke binary compatibility, InventoryTransactionPacket: Converted to Cereal) — "Cereal" is the system that emits these JSON exports.

So the docs are improving, but the kind of problems flagged are structurally unchanged.

Issue-by-issue verification against the live protocol

1. Optional fields still not representable — STILL BROKEN.
The current json/CameraInstructionPacket.json (proto 1001) still lists every field as required, even though all 9 are brstd::optional on the wire (confirmed against PocketMine's readOptional calls and gophertunnel's OptionalMarshaler):

"7894722": {
    "title": "CameraInstruction",
    ...
    "required": [
        "AttachToEntity","Clear","DetachFromEntity","Fade",
        "FieldOfView","RemoveTarget","Set","Spline","Target"
    ]
}

I confirmed the same on CameraPresetsPacket.json: the CameraPreset definition marks all 22 properties required, including the known-optional Continue Targeting, Horizontal Rotation Limit, and Vertical Rotation Limit (all documented as brstd::optional in the r/21_u4 changelog). There is no x-optional / nullable / presence-flag marker anywhere in the schema.

Worse, required can't even be repurposed for this: it's actually being used for default-value semantics. In SplineInstruction, fields with a "default" (loadFromJson, splineIdentifier) are excluded from required, while wire-optional fields with no default are included. So required ≠ optionality — exactly the ambiguity Happy2018new described.

2. Missing packets — STILL TRUE. 26 packets are still only in the legacy html/ format and absent from json/, including critical ones a generator can't skip:

StartGamePacket, PlayerAuthInputPacket, ItemStackRequestPacket, ItemStackResponsePacket, AddActorPacket, AddPlayerPacket, AddItemActorPacket, LevelChunkPacket, SubChunkPacket, CraftingDataPacket, CreativeContentPacket, MovePlayerPacket, MoveActorDeltaPacket, SetActorDataPacket, ResourcePacksInfoPacket, … (StartGamePacket.json returns 404). Mojang's own Nov 2025 commit admits "compound tag packets are missing."

3. "x-format-version": "MISSING VERSION" — STILL TRUE. Universal across every packet I sampled (CameraInstructionPacket, CameraPresetsPacket, TextPacket, LoginPacket, DisconnectPacket, SetTimePacket, UpdateBlockPacket, AddVolumeEntityPacket) at proto 1001. (They do now carry x-minecraft-version and x-protocol-version, which weren't emphasized before, but the format-version stamp is still missing.)

[TwistedAsylumMC]

Thanks for the AI slop. I am fully aware this is still relevant which is why it's still open. Did not need an AI to tell me that all the issues I mentioned before are still present