Skip to content

Fix Owner reference-cycle handling across all serializers - #1107

Open
fuffc wants to merge 5 commits into
wieslawsoltes:masterfrom
fuffc:feature/stj-serializer-owner-reference-cycles
Open

Fix Owner reference-cycle handling across all serializers#1107
fuffc wants to merge 5 commits into
wieslawsoltes:masterfrom
fuffc:feature/stj-serializer-owner-reference-cycles

Conversation

@fuffc

@fuffc fuffc commented Jul 13, 2026

Copy link
Copy Markdown

Fix Owner reference-cycle handling across all serializers

Factory.InitLayout wires each dockable's Owner back-reference, producing a cyclic object graph (Dockable.Owner → Dock → VisibleDockables → Dockable). Saving and reloading such a layout was broken in four of the five serializers — ranging from silent identity duplication to process-killing crashes. This PR fixes each one and adds round-trip regression coverage.

What was broken

  • System.Text.Json — the per-item JsonConverterList restarted STJ's reference tracking on every element, breaking $id/$ref across list boundaries and duplicating ActiveDockable identity.
  • Xml / YamlListTypeConverter's post-deserialize tree walk had no cycle guard, so once the Owner back-reference was restored it recursed forever — an uncatchable StackOverflowException that kills the process.
  • Protobuf — protobuf-net 3.x has no working reference preservation (AsReference is obsolete-as-error), so the cycle either threw or came back with duplicate instances. A separate "Unexpected sub-type" failure also broke any custom Dockable subclass defined outside the Dock.Model.* assemblies.

Changes

  1. Fix System.Text.Json Owner reference-cycle handling — replace JsonConverterList/JsonConverterFactoryList with DockListTypeInfoModifier, a JsonTypeInfo resolver modifier that only overrides list instantiation, leaving STJ's built-in enumerable path (and its single ambient reference-tracking state) in control of the whole graph.
  2. Fix Xml/Yaml StackOverflowException on Owner reference cycles — add a reference-equality visited set to ListTypeConverter; the walk is idempotent, so visiting each instance once is sufficient.
  3. Fix Protobuf Owner cycle crash and ActiveDockable duplication — exclude Owner/OriginalOwner from the wire; ListTypeConverter rebuilds Owner structurally from tree containment (no InitLayout call required, matching the other serializers) and reconciles ActiveDockable/DefaultDockable/FocusedDockable by Id, guarding against null/duplicate Ids.
  4. Fix Protobuf Unexpected sub-type for custom Dockable subclasses — register each class under its nearest tracked base class as well as its closest interface.
  5. Preserve Dock list constructors for NativeAOT — statically root the built-in IList<IDockable> and IList<IDockWindow> creators so trimmed source-generated applications retain the required constructors.

Testing

  • DockLayoutReferencePreservationTests round-trips a wired layout through all five serializers and asserts ActiveDockable identity is preserved.
  • Protobuf-specific tests cover structural Owner rebuild and custom-subtype round-tripping.
  • System.Text.Json tests cover AOT-safe observable Dock collections.
  • GitHub Actions: all 9 checks pass, including the NativeAOT source-generation sample and packaging.

Closes #1104
Closes #1105

fuffc added 4 commits July 13, 2026 22:11
The per-item JsonConverterList restarted reference tracking on every
element, breaking $id/$ref across list boundaries. Replace it with a
JsonTypeInfo resolver modifier that only overrides list instantiation,
leaving the built-in enumerable path in control of the whole graph.
ListTypeConverter walked the deserialized tree with no cycle guard,
recursing forever once the Owner back-reference was restored. Add a
reference-equality visited set; the walk is idempotent.
protobuf-net 3.x cannot preserve references, so Owner/OriginalOwner are
excluded from the wire. ListTypeConverter now rebuilds Owner from tree
containment and reconciles ActiveDockable/DefaultDockable/FocusedDockable
by Id, and picks up the Xml/Yaml visited-set cycle guard.
protobuf-net dispatches through the CLR base-class chain, but subtype
registration only considered the closest interface. Register each class
under its nearest tracked base class too.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

2 participants