chore: add live-credential smoke test across github/gitlab/gitea before merging push/pull/delete changes

[ClaudiaFang]

Context

Real-provider testing has already caught provider/API behavior that mocked requestUrl cannot reliably cover. The repository supports GitHub, GitLab, and Gitea, so this issue remains the foundation for headless real-provider integration testing.

Previous E2E infrastructure also exposed an important constraint: Node-only test harness code (fetch, globalThis, node:crypto, node:child_process, node:util) can be picked up by the Obsidian scanner even though it is test-only. The revised design keeps those responsibilities out of the plugin TypeScript test surface.

Goal

Run production provider code against real GitHub / GitLab / Gitea repositories with isolated test branches, independent remote verification, and cleanup that remains safe under concurrency and cancellation.

flowchart LR
    CI[GitHub Actions] --> P[Shell harness: provision]
    P --> R[(Dedicated test repo)]
    CI --> T[TypeScript integration tests]
    T --> S[Production provider service]
    S --> R
    R --> V[Shell harness: independent verify]
    V --> C[Shell harness: cleanup]

Loading

Responsibility boundary

Shell / CI harness

Use Shell + standard Git CLI for test-environment work:

• clone/fetch the dedicated test repository;
• create an isolated branch for each run;
• seed initial Git state;
• independently verify branch/tree/content/commit state;
• delete the current test branch;
• garbage-collect stale test branches.

Use gh / glab only when provider-specific API behavior cannot be expressed cleanly with standard Git. Keep Docker/service startup for Gitea at the GitHub Actions layer rather than spawning it from Node.

Do not reintroduce Node-only provisioning/verifier scripts into the plugin E2E suite.

TypeScript integration tests

TypeScript tests should only exercise the code under test:

• instantiate the real production provider service;
• testConnection;
• list/read/push/update/delete;
• batch operations;
• move/rename semantics;
• conflict/concurrency protections;
• root-path handling and provider-specific behavior.

Independent verification must not reuse the provider implementation under test.

Git authentication

CI secrets should be injected as job-scoped environment variables and exposed to Git through GIT_ASKPASS.

Do not:

• embed tokens in remote URLs;
• persist tokens into .git/config;
• pass tokens as normal CLI arguments;
• print tokens in logs or artifacts.

The askpass helper contains no credential itself; it only reads the job-scoped environment value when Git requests authentication.

Branch isolation

Every provider job receives a unique remote branch. A cancelled/stuck run must never share state with a newer run.

PR run:

e2e/pr/123/github/run-987654-1

Branch-only run:

e2e/branch/feat-settings-compat-a31f2c/github/run-987654-1

Identity should include:

• PR number, or sanitized source-branch name + short hash;
• provider;
• github.run_id;
• github.run_attempt.

Cleanup lifecycle

Cleanup is best-effort; correctness must not depend on cleanup always running.

1. Per-job cleanup

   • if: always();
   • delete only the current run branch.

2. PR / source-branch lifecycle cleanup

   • PR closed/merged → remove e2e/pr/<number>/**;
   • source branch deleted → remove its e2e/branch/<name>/** namespace.

3. Scheduled janitor

   • periodically remove stale e2e/** branches older than a TTL;
   • covers runner crash, force cancellation, network failure, or failed cleanup.

A new run always allocates a new branch, so an orphan from an older run is harmless until the janitor removes it.

CI shape

Keep provider and Obsidian compatibility matrices independent. This issue owns the provider matrix only:

provider / GitHub
provider / GitLab
provider / Gitea

Each provider job follows the same lifecycle:

provision -> seed -> TypeScript integration -> independent verify -> cleanup

Do not create an Obsidian-version × provider Cartesian matrix.

Provider contract coverage

Shared scenarios

• testConnection
• pushFile create/update
• getFile
• pushBatch
• deleteFile
• deleteBatch
• listFilesDetailed
• getBlob
• root-path handling
• move/rename verification
• conflict/concurrency safety

GitHub additional scenarios

• GraphQL batch commit verification
• GraphQL errors[] handling
• symlink push (mode 120000)

Acceptance criteria

• GitHub / GitLab / Gitea real-provider jobs run independently in CI.
• Provision / verify / cleanup use Shell + Git CLI rather than Node-only E2E infrastructure.
• Production TypeScript provider code is the code under test.
• Independent verification does not reuse the provider implementation.
• Every run has an isolated remote branch.
• Concurrent/cancelled runs cannot overwrite one another.
• PR close/merge and source-branch deletion clean their test namespaces.
• Scheduled janitor removes orphaned branches.
• Tokens never appear in remote URLs, Git config, command arguments, logs, or artifacts.
• Obsidian scanner remains clean with the real-provider test infrastructure present.

Follow-up

GUI/real-Obsidian automation remains tracked separately in #115 and should reuse this provider sandbox/lifecycle rather than duplicate it.

[ClaudiaFang]

Revised plan: real-provider E2E in one branch / one PR

Updated 2026-08-03. This replaces the earlier oversized proposal with a scope that can be implemented and reviewed in one branch and one PR, while still qualifying as real E2E for the plugin's core sync path.

Definition of “real E2E”

The suite must:

1. instantiate the production GitHubService, GitLabService, or GiteaService;
2. run them through SyncManager with an in-memory Vault where applicable;
3. execute real HTTP requests against GitHub, GitLab, and Gitea;
4. create real commits, trees, blobs, moves, and deletions;
5. verify the remote result with an independent raw API verifier, not through the service under test.

This PR does not attempt full Obsidian UI automation. Obsidian event ordering and modal interaction remain a short manual smoke checklist. The automated E2E boundary is the core sync logic + provider production code + real Git server.

---

Branch and PR

• Branch: test/real-provider-e2e
• PR: test(e2e): verify sync operations against GitHub, GitLab and Gitea

One branch, one PR, reviewable through separate commits.

---

Architecture

e2e/
├── vitest.e2e.config.ts
├── setup.ts
├── config.ts
├── reporter.ts
├── shim/
│   ├── obsidian.ts
│   └── fake-vault.ts
├── providers/
│   ├── provider-adapter.ts
│   ├── github-adapter.ts
│   ├── gitlab-adapter.ts
│   └── gitea-adapter.ts
├── provision/
│   ├── sandbox.ts
│   ├── github-sandbox.ts
│   ├── gitlab-sandbox.ts
│   └── gitea-sandbox.ts
├── suites/
│   ├── service-contract.e2e.test.ts
│   ├── sync-manager.e2e.test.ts
│   └── github-symlink.e2e.test.ts
└── fixtures/
    ├── text.ts
    └── binary.ts

scripts/
├── e2e-gitea-up.sh
├── e2e-gitea-down.sh
└── e2e-sweep-branches.mjs

docs/testing/
└── real-provider-e2e.md

NPM scripts:

{
  "test:e2e": "vitest run --config e2e/vitest.e2e.config.ts",
  "test:e2e:smoke": "vitest run --config e2e/vitest.e2e.config.ts --testNamePattern='@smoke'",
  "test:e2e:full": "vitest run --config e2e/vitest.e2e.config.ts"
}

The E2E suite must use a separate Vitest config so the current mocked obsidian module and mocked requestUrl from tests/setup.ts cannot leak into real-provider tests.

---

Load-bearing Obsidian shim

e2e/shim/obsidian.ts wraps native fetch and reproduces the requestUrl response contract expected by BaseGitService.safeRequest().

Required behavior:

• HTTP 4xx/5xx returns a response and does not throw automatically;
• only DNS/TLS/timeout/network failures throw;
• returns { status, headers, text, json, arrayBuffer };
• header names are normalized to lowercase;
• body bytes are read once and exposed as text/arrayBuffer;
• json is a lazy getter and throws when the body is not JSON;
• defines window.setTimeout, required by GitHub stale-head retry logic;
• sanitizes tokens before any failure is printed.

The suite must import the existing production service files. It must not reimplement GitHub/GitLab/Gitea request logic in the test harness.

---

Environments

GitHub

Use a dedicated sandbox repository, e.g.:

firstsun-dev/git-files-sync-e2e

Use a fine-grained PAT restricted to that one repository with Contents read/write. Do not write E2E branches into the plugin repository and do not reuse a broad organization token.

GitLab

Use a dedicated gitlab.com sandbox project with a project access token:

• scope: api;
• role: Developer or Maintainer;
• project restricted to the E2E sandbox.

write_repository alone is insufficient because GitLabService uses the REST API rather than Git-over-HTTP.

Gitea

Run a pinned Gitea container in CI and locally, initially:

docker.gitea.com/gitea:1.27.0

Provision at runtime:

1. wait for /api/v1/version;
2. create an admin test user;
3. generate an access token;
4. create an initialized repository;
5. execute tests;
6. destroy the container.

Gitea requires no stored repository secret and can therefore run on fork PRs.

---

Configuration and secret handling

Local and CI runs use the same file-based configuration:

{
  "github": {
    "owner": "firstsun-dev",
    "repo": "git-files-sync-e2e",
    "tokenFile": ".e2e/secrets/github.token"
  },
  "gitlab": {
    "baseUrl": "https://gitlab.com",
    "projectId": "<project-id>",
    "tokenFile": ".e2e/secrets/gitlab.token"
  },
  "gitea": {
    "baseUrl": "http://127.0.0.1:3000",
    "owner": "e2e",
    "repo": "git-files-sync-e2e",
    "tokenFile": ".e2e/secrets/gitea.token"
  }
}

Rules:

• add .e2e/ to .gitignore;
• no token CLI arguments;
• test code does not directly read provider tokens from process.env;
• CI masks each token before writing a chmod 600 token file;
• reporter redacts every configured token substring;
• request headers and secret-bearing bodies are never uploaded as artifacts;
• each CI job receives --require-provider=<provider> and fails if zero tests execute;
• a missing required secret is an explicit failure, never a silent skip/green result.

---

Sandbox lifecycle and concurrency

Each test job creates its own branch:

e2e-github-<run-id>-<attempt>
e2e-gitlab-<run-id>-<attempt>
e2e-gitea-<run-id>-<attempt>

Initial implementation deliberately avoids / in branch names because provider branch-path encoding should be tested separately rather than making the whole harness depend on it.

Lifecycle:

read seed/default head
  -> create ephemeral branch
  -> provision fixtures with raw provider API
  -> execute production service / SyncManager
  -> independently verify remote state
  -> delete branch in finally

Cleanup guarantees:

• afterAll plus outer try/finally teardown;
• startup sweeper removes e2e-* branches older than 24 hours;
• E2E_KEEP_BRANCH=1 retains a failing branch for debugging;
• tests within one provider run serially;
• GitHub, GitLab, and Gitea jobs may run in parallel.

---

Independent verification

A service must never validate its own write using its own getFile() or listFilesDetailed() implementation.

Provider adapters use raw API calls to verify:

• branch head before and after;
• exact commit-count delta;
• remote tree paths;
• raw blob content and binary bytes;
• file modes, including 120000 symlinks;
• old path removal and new path creation for moves;
• blob SHA cross-check using gitBlobSha.

This prevents symmetric write/read bugs from passing the test.

---

Service-contract matrix

Smoke tier — every relevant internal PR

Run for all three providers unless marked otherwise:

• testConnection: valid branch and missing branch;
• pushFile: create UTF-8/CJK content;
• getFile: real round-trip;
• pushFile: update an existing file using the current SHA;
• missing file returns { content: '', sha: '' };
• listFilesDetailed: path and SHA are correct;
• getBlob: content and SHA are correct;
• pushBatch: mixed create/update and exactly one new commit;
• commitBatch: real move, old path gone, new path exists, one commit;
• deleteFile: file is gone remotely;
• deleteBatch: all files gone and exactly one new commit;
• rootPath: writes and filtered listing stay inside the configured root;
• batch result returns usable SHA values where the interface/provider supports them.

Provider-specific smoke cases:

GitHub

• GraphQL createCommitOnBranch succeeds against the real API;
• HTTP 200 with GraphQL errors[] is surfaced as an exception;
• pushSymlink creates tree mode 120000 and the expected target;
• GitLab/Gitea expose no pushSymlink capability.

GitLab

• create vs update actions behave correctly;
• move uses the Commits API move action and correct previous path.

Gitea

• branch -> commit resolution works;
• blob -> tree -> commit -> ref flow works;
• batch deletion through null tree entries works.

Full tier — main, manual dispatch, scheduled API-drift run

• binary push/pull byte identity;
• paths containing spaces, CJK, #, +, and nested directories;
• stale SHA update conflict;
• atomic batch failure leaves the remote tree unchanged;
• approximately 30-file batch for payload/concurrency behavior;
• non-existent path inside deleteBatch with provider behavior pinned down;
• GitHub immediate push-then-delete stale-head retry;
• GitLab pagination;
• .gitignore retrieval;
• branch names containing / as an explicit encoding regression case.

---

SyncManager real-provider E2E

The first PR includes six high-value scenarios using an in-memory fake Vault and a real provider service:

1. Push new file

   • remote file exists;
   • syncMetadata.lastSyncedSha records the real remote SHA.

2. Push unchanged file

   • second push is skipped;
   • remote commit count does not change.

3. Pull remote update

   • raw provider API changes the remote file;
   • SyncManager.pullFile() updates the Vault and metadata.

4. Conflict protection

   • both local and remote change after the last sync;
   • push must not silently overwrite the remote;
   • metadata must not be falsely marked synced.

5. Rename / move

   • call trackRename(oldPath, newPath);
   • push removes the old remote path and creates the new one;
   • exactly one commit is created.

6. Batch push-all path

   • multiple Vault files are pushed through the batch-capable path;
   • all remote files are correct;
   • exactly one commit is created for the batch.

Additional conflict-modal choices, complete gitignore interaction, chunk-boundary stress, and every symlink mode stay out of this first PR unless implementation is unexpectedly small.

---

CI wiring

Add jobs to the existing .github/workflows/ci.yml:

changes
├── e2e-gitea
├── e2e-github
└── e2e-gitlab
        ↓
       CI (reusable workflow, including release)

Triggers:

• internal PR touching sync/provider paths: Gitea + GitHub + GitLab smoke;
• fork PR: Gitea smoke only;
• push to main: all three full tier;
• workflow_dispatch: choose provider/tier/keep-branch;
• weekly schedule: all three full tier for API-drift detection.

Relevant path filter:

src/services/**
src/logic/sync-manager.ts
src/utils/git-blob-sha.ts
src/utils/path.ts
src/utils/symlink.ts
e2e/**
package.json
package-lock.json
.github/workflows/ci.yml

Release gating

The current shared reusable workflow performs semantic release internally. Therefore the E2E jobs must run before the caller invokes the reusable CI job.

The CI job should needs the three E2E jobs and allow only successful or intentionally skipped provider jobs. This ensures a real-provider failure blocks the shared workflow and release rather than becoming a post-release canary.

Gitea should become the required E2E status check because it runs without secrets and works on fork PRs. GitHub/GitLab checks should remain non-required at branch-protection level so fork PRs cannot be wedged by unavailable secrets; internal PR release gating still depends on them through workflow job dependencies.

---

Commit breakdown inside the single PR

1. test(e2e): add real requestUrl shim and secure config loader
2. test(e2e): add provider sandboxes and service contract suite
3. test(e2e): add SyncManager real-provider scenarios
4. ci(e2e): run provider checks before shared CI and release
5. docs(e2e): document setup credentials cleanup and troubleshooting

---

Acceptance criteria

• existing lint, build, and unit tests remain green;
• Gitea provisions from a clean container without manual setup;
• GitHub, GitLab, and Gitea smoke suites each pass twice consecutively;
• pushBatch, commitBatch, and deleteBatch prove one-commit behavior;
• every remote assertion uses the independent verifier;
• no current-run branch remains after a successful run;
• a cancelled/crashed run is cleaned by the next sweeper;
• missing credentials cannot produce a zero-test green job;
• CI logs and uploaded artifacts contain no token value;
• fork PRs can complete the required Gitea check;
• a required provider E2E failure prevents the reusable CI/release workflow from starting;
• docs/testing/real-provider-e2e.md includes local setup, CI setup, cleanup, and failure investigation instructions.

---

Explicit non-goals for this PR

• full Obsidian desktop UI automation;
• exhaustive modal interaction testing;
• hundreds/thousands of file load testing;
• a large Gitea version compatibility matrix;
• production monitoring or alerting for sandbox failures;
• refactoring provider APIs unrelated to enabling E2E.

Those can follow once the core real-provider harness is stable.

[ClaudiaFang]

Backlog priority review — 2026-08-04

Priority: P1

Reason: Mock tests cannot verify real provider API semantics, and a live test has already caught production-relevant behavior. This is a reliability and credential-handling guardrail for every future push/pull/delete change.

Scheduling note: Build the repeatable local smoke-test harness before adding another provider or materially changing provider services. Keep credentials file-based and out of agent transcripts, and document cleanup of disposable test data.

[ClaudiaFang]

2026-08-07 update: keep #57 headless/core; split GUI E2E to #115

After the recent sync correctness fixes, this issue remains the real-provider/core E2E foundation. Full Obsidian desktop GUI automation is intentionally split into #115 so this suite stays deterministic and useful as a release gate.

Execution order

Implement providers in this order:

1. Gitea — validate the harness first with a pinned disposable container and no external SaaS secret dependency.
2. GitHub — validate the highest-risk GraphQL batch path, stale-head handling, and symlink mode.
3. GitLab — validate REST commit actions and the recently fixed blob-SHA vs revision semantics.

Do not debug all three providers at once. Get one complete vertical slice green before adding the next provider.

Credential clarification

• GitHub: dedicated sandbox repository + fine-grained PAT restricted to that repository.
• GitLab: use a dedicated sandbox project. Prefer a project-scoped token with api where available; otherwise use a dedicated low-blast-radius test account PAT with api. write_repository alone is not sufficient for the REST API paths used by GitLabService.
• Gitea: provision the user/token/repository at runtime inside the pinned container.

Promote #101 regression into smoke tier

The recently fixed GitLab identity bug must be an explicit real-provider smoke case:

single pull
-> local edit
-> batch push
-> no false remote conflict

Assertions:

• GitFile.sha remains blob/content identity;
• provider revision is used only for write/concurrency control;
• syncMetadata.lastSyncedSha is not polluted with a commit revision;
• mixed single/batch flows do not report a false remote conflict.

CI gate adjustment

Use an explicit aggregator rather than wiring the reusable CI directly to three provider jobs:

e2e-gitea  --\
e2e-github ----> e2e-gate -> shared CI -> semantic-release
e2e-gitlab --/

e2e-gate should run with if: always() and distinguish:

• required provider success;
• legitimate skip because credentials are unavailable for that event type;
• real failure.

This avoids a skipped secret-backed provider job accidentally skipping the downstream reusable CI while still preventing a real provider failure from reaching release.

Scope boundary with #115

#57 owns:

• production provider services against real Git servers;
• SyncManager real-provider scenarios with a fake/in-memory Vault;
• independent raw API verification;
• deterministic release gating.

#115 owns:

• launching real Obsidian Desktop with Playwright/Electron;
• settings / commands / modal / Vault-event wiring;
• real GUI push/pull flows;
• GUI smoke across all providers and exhaustive GUI coverage on a canonical provider.

The two layers should share sandbox/verifier helpers where practical, but GUI E2E must not replace the headless/core suite.

[ClaudiaFang]

Revision: run providers as a parallel matrix on the self-hosted runner fleet

Use the existing Ansible-managed GitHub Actions runners from firstsun-dev/infra-config as the default execution environment for real-provider E2E.

Current runner fleet characteristics already fit this workload:

• org-level firstsun-dev runners;
• 10 ephemeral runners;
• labels currently include self-hosted, linux, x64, 32gb-ram;
• runner containers are privileged and mount /var/run/docker.sock;
• the runner image already includes Xvfb and Playwright/Chromium dependencies.

Replace three hand-written provider jobs with one matrix

provider-e2e:
  name: E2E / ${{ matrix.provider }}
  runs-on: [self-hosted, linux, x64, 32gb-ram]
  strategy:
    fail-fast: false
    max-parallel: 3
    matrix:
      provider: [github, gitlab, gitea]

  steps:
    - uses: actions/checkout@v6
    - uses: actions/setup-node@v6
      with:
        node-version: '22'
        cache: npm
    - run: npm ci
    - name: Prepare provider sandbox
      run: node scripts/e2e-runner.mjs prepare --provider=${{ matrix.provider }}
    - name: Run provider E2E
      run: node scripts/e2e-runner.mjs test --provider=${{ matrix.provider }}
    - name: Cleanup provider sandbox
      if: always()
      run: node scripts/e2e-runner.mjs cleanup --provider=${{ matrix.provider }}

The wrapper continues to own provider selection and credential loading; custom provider flags must not be passed directly to Vitest.

Required/optional semantics

The same matrix should run in all supported events, but provider requirements differ:

• internal PR touching sync/provider paths: GitHub + GitLab + Gitea required;
• push to main: GitHub + GitLab + Gitea required, full tier;
• workflow_dispatch: selected provider(s)/tier;
• fork PR: Gitea required; GitHub/GitLab cells may report a clear intentional skip when protected credentials are unavailable;
• scheduled API-drift run: all three required.

A required provider with missing credentials must fail explicitly. An optional provider skipped because the event cannot receive secrets must emit a visible notice and must not be confused with a successful test execution.

Release gating simplification

Because the provider checks are one matrix job, the caller workflow can gate the reusable CI/release workflow on the aggregate matrix result:

provider-e2e [github | gitlab | gitea in parallel]
                |
                v
        shared Obsidian CI
                |
                v
        semantic-release

This removes the need for three independent provider job IDs. Keep a separate e2e-gate only if later we need more complex policy aggregation; it is not required for the initial matrix implementation if required/optional semantics are enforced inside the matrix runner.

Sandbox isolation for parallel execution

Every matrix cell must be isolated by provider + workflow run:

e2e-<provider>-<run-id>-<attempt>

GitHub/GitLab use independent ephemeral branches in their dedicated sandbox repositories/projects.

Gitea can be provisioned through Docker from the self-hosted runner, but it must not assume a fixed global container name, Docker network, or host port because multiple matrix/CI jobs may share the same Docker host. Use run-specific container/network names and dynamically allocated connectivity.

Implementation order

The harness can still be developed vertically in this order:

1. Gitea contract works locally/on one runner;
2. GitHub contract;
3. GitLab contract + bug(sync): normalize GitLab blob SHA and commit revision semantics #101 regression;
4. switch workflow on with matrix.provider and max-parallel: 3;
5. prove all three cells pass concurrently twice;
6. enable release gating.

The final CI architecture is parallel even though implementation/debugging is incremental.

[ClaudiaFang]

Status update — 2026-08-16

Core implementation is done and merged; two follow-ups are still open.

Merged to main

• Shell/Git real-provider E2E harness (e2e/** + scripts/e2e-harness.sh) for GitHub, GitLab, and Gitea — production provider services under test, independent raw-API verification, isolated per-run branches (e2e/pr/<n>/**, e2e/branch/<name>/**), E2E_KEEP_BRANCH debug retention. Landed via test/real-provider-e2e in two phases (Phase 1: Shell/Git harness replacing the earlier Node harness; Phase 2: multi-run isolation) plus a follow-up retry fix for transient runner network flakes (PR ci(e2e): retry provider E2E step on transient runner network failures #125).
• CI wiring: single provider-e2e matrix job (github/gitlab/gitea) gates the shared CI/release workflow, with path-filtered triggering on sync/provider-relevant changes plus a weekly API-drift schedule.
• Scheduled janitor: sweeps stale e2e/** branches — confirmed running successfully on schedule.
• Credential handling: tokens injected as job-scoped env vars, no tokens in URLs/config/logs — per the original design.

Known gap

• Gitea leg is temporarily disabled in CI (ci.yml, provider-e2e job) — container provisioning against this runner fleet's Docker topology needs more investigation (bridge-IP reachability, health-check timing). Suite/harness code itself is untouched and passes locally (npm run test:e2e -- --provider gitea). Since Gitea is also the no-secret leg that normally covers fork PRs, fork PRs currently get no E2E coverage until this is re-enabled.

Still open

• PR docs: surface real-provider E2E testing #123 (draft) — documents the real-provider E2E strategy in README/CONTRIBUTING + adds docs/testing/test-matrix.md and test-scenarios.md. No code changes, ready for review.
• PR chore: clean up Obsidian scan compatibility boundaries #119 (draft, opened 2026-08-07) — predates the Phase 1/2 Shell rebuild and is now CONFLICTING against main; superseded by the work that landed since. Recommend closing in favor of the current state, unless there's a piece of it worth salvaging.

Next step

Re-enable the Gitea CI leg (see TODO comment in ci.yml), then this issue's acceptance criteria are essentially met except for the fork-PR coverage gap it currently causes.