Skip to content

Roadmap direction + first increments across the priority pillars - #29

Merged
RubenGlez merged 11 commits into
mainfrom
claude/roadmap-competitive-analysis-d7dvw7
Jul 3, 2026
Merged

Roadmap direction + first increments across the priority pillars#29
RubenGlez merged 11 commits into
mainfrom
claude/roadmap-competitive-analysis-d7dvw7

Conversation

@RubenGlez

@RubenGlez RubenGlez commented Jul 3, 2026

Copy link
Copy Markdown
Owner

Why

This branch sets the product direction and then ships a first, verified increment on each of the highest-priority pillars — turning the roadmap from a document into working features.

The wedge: the OpenAPI spec you never have to write, and never have to trust a cloud with. EasyDocs sits in the empty intersection competitors leave — generating an accurate spec from real traffic, entirely on your own infrastructure. The pillars, in priority order: C privacy/local-first → A docs-vs-reality drift → B measurable accuracy → D interoperable upstream. See ROADMAP.md for the full reasoning and the explicit anti-goals.

What's in here

Direction (docs)

  • ROADMAP.md — the wedge, the four pillars, what we deliberately won't build, and where the local dashboard fits (producer-side cockpit, not a consumer-facing portal). Boundary recorded in AGENTS.md.

Pillar C — privacy / local-first

  • Strict offline mode (privacy.offline: true): a hard guarantee, not best-effort redaction. Pins to a local Ollama model, ignores hosted API keys in the environment, and fails fast if a hosted provider is explicitly configured — nothing captured can leave the machine. Opens the regulated / air-gapped segment that traffic-based SaaS can't serve.
  • Redaction audit: collectSensitiveFields reads the x-easydocs-sensitive markers back out of processed specs. Surfaced two ways — a "Sensitive fields" panel in the dashboard, and a scriptable easydocs audit command (with --project / --markdown). Makes the PII-safe promise provable.

Pillar A — docs-vs-reality drift

  • The comparison only EasyDocs can make (it holds both the committed spec and the live traffic): computeDrift classifies divergence as undocumented / mismatch / unobserved. Exposed as easydocs drift <spec> (compares against captured traffic, or two files directly) and as a drift view in the dashboard (per-endpoint badges + breakdown panel). Informational only — never fails the build.

Pillar B — measurable accuracy

  • Publishable scoreboard: pnpm matrix --markdown > SCOREBOARD.md emits the per-provider/model accuracy scoreboard as committable Markdown (progress to stderr, clean stdout).
  • Per-section breakdown: aggregates the scorer's existing per-dimension scores so the scoreboard shows where a model is strong or weak (tags, responses, schema, parameters, request body, security), not just an overall mean.

Verification

Every increment was verified before commit, and the full workspace stays green throughout (build, typecheck, lint, tests):

  • Core: 116 tests (drift, offline provider resolution, capture pipeline, redaction audit).
  • Drift & audit CLIs: exercised end-to-end (text + Markdown + empty cases).
  • Dashboard drift and sensitive-fields views: driven in a real browser against a seeded DB.
  • Scoreboard aggregation: verified against real fixtures via the deterministic scorer.

Note: the accuracy eval.yml gate skips without provider secrets (by design), so real scoreboard numbers must be generated where keys are available.

🤖 Generated with Claude Code

Generated by Claude Code

claude added 11 commits July 1, 2026 10:24
@claude
docs: add product roadmap and competitive positioning
497e7b7 
Define the strategic wedge (accurate OpenAPI from real traffic, generated
on your own infrastructure) and the four pillars that defend it, in priority
order: privacy/local-first, docs-vs-reality drift, measurable accuracy, and
interoperable upstream. Includes explicit anti-goals and rough phasing.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
docs: clarify the local dashboard's place in the roadmap
7cf044d 
Distinguish the producer-side local dashboard (kept, central to pillars 1-2)
from a hosted consumer-facing docs portal (anti-goal). Add a section on where
the dashboard fits and sharpen the anti-goal wording to remove the apparent
contradiction.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
docs: record the dashboard boundary in AGENTS.md
140cddd 
Point contributors to ROADMAP.md and codify the producer-side vs
consumer-facing dashboard boundary so future dashboard work respects the
strategic line.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
feat(core,cli): docs-vs-reality drift detection
48b50e3 
Add the flagship drift check: compare a committed OpenAPI spec against the
spec EasyDocs derives from real traffic, and classify the divergence as
undocumented (observed but not in spec), unobserved (documented but not seen),
or mismatch (spec contradicts reality). This is the one comparison only
EasyDocs can make, since it holds both the committed spec and live traffic.

- core: new spec/drift engine (computeDrift/renderDrift/isEmptyDrift/driftCount)
  built on the existing diffSpecs, scoped to the endpoint contract by default so
  metadata differences do not create false drift. Exported from the package root
  and the @easydocs/core/spec/drift subpath, with tests.
- cli: new 'easydocs drift <spec> [observed]' command. One arg reads observed
  traffic from the local capture DB; two args compare files. Supports --project
  and --markdown; informational only, always exits 0.
- docs: README, CLI README, and CHANGELOG.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
feat(dashboard): docs-vs-reality drift view
3d29cc4 
Surface drift in the local producer-side dashboard: read the committed spec
from EASYDOCS_SPEC_PATH (default ./openapi.json), compare it against the spec
derived from observed traffic, and show where documentation diverged.

- lib/db: fetchDrift() loads the committed spec, builds the observed spec, and
  computes drift plus a per-endpoint finding count for sidebar badges.
- new /api/drift route (local only — the comparison never leaves the machine).
- Dashboard sidebar: a drift toggle with a total count, and a rose dot on each
  drifted endpoint alongside the existing conflict dot.
- new DriftSummary panel breaking findings into undocumented / mismatch /
  documented-but-unobserved sections.

Keeps the dashboard a producer-side cockpit per the roadmap — no consumer-facing
portal, just the surface where the privacy and drift pillars become visible.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
feat(core): strict offline mode for regulated environments
fae563f 
Add privacy.offline — a hard local-first guarantee, not just best-effort
redaction. When enabled, EasyDocs pins the AI provider to a local Ollama model,
ignores any hosted API keys present in the environment, and fails fast at
startup if a hosted provider is explicitly configured. No captured payload can
ever reach a third-party service, so the regulated / air-gapped segment that is
structurally locked out of traffic-based SaaS can use EasyDocs.

- provider: resolveProvider/resolveModel take an offline flag; new
  isHostedProvider helper. Offline resolves to ollama or throws on a hosted
  provider.
- capture: validates offline config at capturer creation (fail fast), threads
  the flag through redaction (kept local, never redacted) and buildOperation.
- types: privacy.offline boolean.
- exports resolveModel/resolveProvider/isHostedProvider/Provider from the root.
- tests for provider resolution and the capture pipeline; README + CHANGELOG.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
feat(evals): publishable Markdown accuracy scoreboard
ef19cc7 
Add 'pnpm matrix --markdown' to emit the per-provider/model spec-accuracy
scoreboard as a committable Markdown report (pnpm matrix --markdown >
SCOREBOARD.md). Progress goes to stderr so stdout is clean Markdown. Serves the
measurable-accuracy pillar: a published, defensible number that backs the
claims of the drift and local-first pillars.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
feat(core,dashboard): redaction audit — show what EasyDocs protects
1c3f753 
Make the PII-safe promise provable: read the x-easydocs-sensitive markers back
out of processed specs and show exactly which fields are protected.

- core: new pure collectSensitiveFields(operation) that walks parameters,
  requestBody, and responses and returns each flagged field with its location,
  de-duplicated. Exported from the root and the pure @easydocs/core/privacy/audit
  subpath (safe for the browser bundle), with tests.
- dashboard: a 'Sensitive fields' panel (emerald toggle + count) listing flagged
  fields grouped by endpoint, computed client-side from the loaded specs. Sits
  alongside the drift panel as the second place the privacy pillar becomes visible.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
feat(evals): per-section accuracy breakdown in the scoreboard
056aed3 
The scorer already computes per-dimension scores; the matrix threw them away.
Aggregate them so the scoreboard shows where each model is strong or weak
(tags, responses, response schema, parameters, request body, security), not
just an overall mean — making the accuracy moat granular and diagnosable.

- score: new pure meanByDimension() that averages each acc:<section> across the
  fixtures where it applied.
- matrix: capture namedScores per fixture, render an 'Accuracy by section' table
  in both the console and --markdown scoreboard.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
feat(core,cli): easydocs audit command for the redaction inventory
326217d 
Add a scriptable/CI surface for the redaction audit, mirroring the dashboard's
'Sensitive fields' panel: 'easydocs audit' reads the stored specs and lists
every field flagged sensitive, grouped by endpoint. Supports --project and
--markdown (for PR comments). Lets a pipeline assert exactly what EasyDocs is
protecting.

- core: renderAudit() + EndpointAudit type in privacy/audit (pure), exported and
  tested.
- cli: new 'audit' command reading endpoints from the capture DB.
- docs: CLI README + CHANGELOG.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@claude
Merge origin/main into roadmap branch
e68a636 
Reconcile with v0.8.0/v0.8.1 released on main:
- Combine core/cli exports: main's fail-able diff (classifyDiff/shouldFail/
  renderClassifiedDiff/FailOn) alongside this branch's drift + audit commands.
- main independently shipped the Markdown accuracy scoreboard and per-section
  breakdown (BENCHMARK.md); dropped this branch's duplicate matrix/score changes
  in favor of main's released version.
- Merge CHANGELOG (kept [Unreleased] atop the 0.8.x releases) and AGENTS.md.

Full workspace verified green after the merge: build, typecheck, lint, and
tests (core 131, cli 11). All three CLI commands (diff, drift, audit) exercised.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012hKk3suH6Hfjqkph6aPVt6
@RubenGlez
RubenGlez merged commit 4a8c28b into main Jul 3, 2026
3 checks passed
@RubenGlez
RubenGlez deleted the claude/roadmap-competitive-analysis-d7dvw7 branch July 3, 2026 10:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@RubenGlez @claude