Improve standard.site publishing reliability and content fidelity

[tripledoublev]

Summary

The current standard.site / Sequoia publishing flow for hypha.coop needs work in two areas:

1. Reliability: duplicate site.standard.document records are being created for the same post path.
2. Content fidelity: rich Jekyll post content is degraded when published to standard.site / ATProto.

This issue tracks the work needed to make standard.site publishing safe to run in CI and improve the quality of the published output.

Problems

1. Duplicate records on the PDS

Multiple recent Dripline posts have more than one site.standard.document record for the same path.

Examples found during audit:

• /dripline/who-took-this-photo
• /dripline/the-other-ai
• /dripline/we-built-you-a-garden
• /dripline/similarity-at-scale-perceptual-hashing
• /dripline/fixing-bad-opinions
• /dripline/introducing-rook-sovereign-intelligence

Root cause appears to be Sequoia's publish behavior:

• it decides create vs update from post.frontmatter.atUri
• if atUri is missing from frontmatter, it creates a new remote record even when local state previously knew the mapping
• renames / frontmatter drift / state drift can therefore cause silent duplicate publishes

There are also two payload families on the PDS:

• metadata-only records
• records with textContent

That suggests repeated republishes across different publish modes or Sequoia versions/config states.

2. Formatting and content loss in standard.site output

Current Sequoia output is plain-text-first and loses important post structure:

• custom Jekyll/Tachyons layout
• inline images beyond the cover image
• <figure> / <figcaption> structure
• embedded HTML and Liquid syntax
• richer attribution / acknowledgements
• social metadata that exists on the web version

This is especially visible on long-form Dripline posts where the site version carries meaning through image placement and semantic structure.

What has already been done

Work has already landed on a branch to reduce further damage:

• sync Sequoia state before publish
• fail CI if duplicate remote document paths already exist
• harden workflow and publish-related scripts

That work should be reviewed separately, but this issue tracks the broader product/architecture follow-up.

Proposed work

Reliability

• audit Sequoia publish / sync / inject round-trip in this repo
• ensure atUri survives renames and content updates
• decide whether frontmatter should remain the source of truth, or whether we need a repo-local guard/repair step before publish
• add a repeatable cleanup procedure for duplicate remote records
• document the operational workflow for fixing broken Sequoia state

Content fidelity

• evaluate whether Sequoia config can be extended to improve record quality
• test publishContent / textContentField behavior explicitly
• decide whether we should preprocess post content for ATProto instead of sending raw markdown/Jekyll source
• define a “good enough” derived representation for long-form Hypha posts on standard.site
• preserve more semantics where possible, especially images, captions, and attribution

Acceptance criteria

• publishing the same post repeatedly does not create duplicate site.standard.document records
• renaming a post does not orphan the existing remote document mapping
• CI detects duplicate remote paths before publish proceeds
• standard.site output is intentionally shaped rather than being a lossy byproduct of raw Jekyll source
• the repo has clear documentation for how standard.site publishing is expected to work

Notes

Relevant examples:

• at://did:plc:rxduhzsfgfpl2glle7vagcwl/site.standard.document/3mf5g2ci5n52j
• at://did:plc:rxduhzsfgfpl2glle7vagcwl/site.standard.document/3miwl7r7ecf22

These are distinct records for the same /dripline/who-took-this-photo path.

[tripledoublev]

A narrower follow-up issue may be useful for the duplicate-record/state-drift problem specifically: frontmatter atUri loss, .sequoia-state.json drift, rename handling, and remote duplicate cleanup. That can likely be scoped and resolved independently of the richer standard.site formatting/content-fidelity work.