Skip to content

Deepen layers interop via lairs; consolidate onto canonical models (0.7.0)#8

Merged
aaronstevenwhite merged 7 commits into
mainfrom
deepen-layers-lairs-interop
Jun 30, 2026
Merged

Deepen layers interop via lairs; consolidate onto canonical models (0.7.0)#8
aaronstevenwhite merged 7 commits into
mainfrom
deepen-layers-lairs-interop

Conversation

@aaronstevenwhite

Copy link
Copy Markdown
Collaborator

Description

Deepens the bead.interop.layers integration with lairs, the read/write
client for the layers annotation format. bead now depends on lairs directly
and builds its layers projections from the canonical lairs.records models that
lairs generates from the layers lexicons, instead of a parallel hand-mirrored
copy of the schema.

The change has two halves:

Consolidation. Deletes the roughly 40 hand-mirrored layers models and the
snake/camel + MirrorIso machinery, and re-points the corpus-record, graph,
parse, and resource lenses onto the generated lairs.records models. Each lens
now returns a typed lairs model (or a small bead-side aggregate of them) as its
view, with the bead-only remainder in a JsonValue complement, so the
GetPut/PutGet laws still hold. The redundant Node lexicon validator and the
vendor/layers submodule are retired (validation is now inherent in the
generated models).

New surface. Plugs bead into the lairs ecosystem:

  • ItemLayersLens maps an Item's standoff spans and relations to span and
    relation AnnotationLayer records (a span anchors by tokenRefSequence with
    head_index to anchorTokenIndex, a Wikidata label_id to a knowledgeRef;
    a relation carries ArgumentRef source and target arguments).
  • BeadCodec, registered on the lairs.codecs entry point, round-trips an
    ItemCollection through the layers schema. lairs.codec("bead") resolves it
    and encode(decode(x)) == x is lossless.
  • corpus_io ingests a lairs.data.Corpus into bead models and emits bead data
    as a corpus (materialize to Arrow/Parquet, commit to a repository, publish to
    a PDS).
  • A bead layers CLI group (encode, decode, materialize, opt-in
    publish).

Raises the minimum didactic to >=0.9.0 and panproto to >=0.56.0 to match
lairs, and releases 0.7.0. Importing bead does not import lairs; the
dependency loads only when bead.interop.layers is used.

The five commits are scoped so the dependency bump is isolated first (so any
didactic 0.9 fallout is bisectable on its own), followed by the consolidation,
the new surface, the docs/release, and a ruff-format pass.

Motivation

lairs generates the canonical layers record models from the layers lexicons.
Maintaining a second, hand-written copy of those models inside bead was
redundant and had already drifted from the source of truth (for example
Expression.eprint_ref: str versus the generated eprintRefs: tuple[str, ...]).
Consolidating onto lairs.records removes the duplication and lets bead
participate in the lairs ecosystem as a first-class codec, rather than only
emitting layers-shaped JSON.

No associated issue.

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Refactoring (no functional changes)
  • Tests (adding or updating tests)

The breaking change is scoped to the bead.interop.layers public API (added in
0.6.0): the hand-mirror model exports are removed and the lens view types change
from dicts to typed lairs models.

Checklist

  • I have read the CONTRIBUTING guidelines
  • My code follows the project's style guidelines
  • I have run uv run ruff check . and uv run ruff format .
  • I have run uv run pyright with no errors
  • I have added tests that prove my fix/feature works
  • All tests pass (uv run pytest tests/)
  • I have updated documentation as needed

Testing

  • uv run pyright bead/: 0 errors.
  • uv run ruff check bead/ tests/ and uv run ruff format --check: clean.
  • uv run pytest tests/ -m "not slow_model_training": green. The
    slow_model_training tests (which train ML models, unrelated to this change)
    are deselected as usual.
  • Interop suite: 42 passed, 1 skipped (the live-PDS publish test skips without
    --run-integration).
  • uv run pytest tests/interop/test_layers_publish_integration.py --run-integration: passed. The test stands up a real bluesky PDS in docker,
    publishes a bead corpus, reads the record back via PdsClient, and verifies
    idempotency. It skips cleanly when docker is unavailable.
  • Verified import bead does not pull lairs into sys.modules, and
    lairs.codec("bead") resolves the registered codec.

Bump the minimum didactic to >=0.9.0 and panproto to >=0.56.0, and add
lairs>=0.5.0 (resolved from the local sibling via tool.uv.sources until it
is published). Importing bead does not import lairs.
Delete the hand-mirrored layers models and the snake<->camel / MirrorIso
machinery, and re-point the corpus-record, graph, parse, and resource lenses
onto the generated lairs.records models: each lens now returns a typed lairs
model (or a small bead-side aggregate of them) as its view, with the bead-only
remainder in a JsonValue complement, so the GetPut/PutGet laws still hold.

Retire the redundant Node lexicon validator and the vendor/layers submodule;
validation is now inherent in the lairs generated models. Migrate the interop
tests accordingly.
Register a 'bead' lairs codec (lairs.codecs entry point) that round-trips an
ItemCollection through the layers schema; map an Item's standoff spans and
relations to span/relation AnnotationLayer records via ITEM_LAYERS; ingest a
lairs Corpus into bead models and emit bead data as a corpus (materialize to
Arrow/Parquet, commit to a repository, publish to a PDS). Add a 'bead layers'
CLI group and a docker-PDS integration test for the publish path (gated behind
--run-integration). Make the spaCy tokenizer test skip on any import failure
(spaCy's pydantic.v1 raises ConfigError, not ImportError, on Python 3.14).
Rewrite the layers interop docs for the lairs-backed mapping, codec, item
bridge, corpus IO, and CLI; drop the retired mirror-model and submodule
references from the README and add a note on the --run-integration PDS tests.
Bump the version to 0.7.0 with a changelog entry.
lairs 0.5.0 is not yet on PyPI and the pip-based CI jobs ignore the
tool.uv.sources path redirect, so check out the public layers-pub/lairs repo
and pip-install it before installing bead, satisfying the lairs>=0.5.0
requirement.
lairs 0.5.0 is now published on PyPI, so drop the tool.uv.sources local-path
redirect and the CI source-checkout steps; the plain lairs>=0.5.0 requirement
resolves from PyPI for both local and CI installs.
@aaronstevenwhite aaronstevenwhite merged commit 7241773 into main Jun 30, 2026
8 checks passed
@aaronstevenwhite aaronstevenwhite deleted the deepen-layers-lairs-interop branch June 30, 2026 12:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant