GitBucket Testing Pipeline — Overview

[michael-conrad]

Overview

Establish a comprehensive, fully-Java testing pipeline for the GitBucket MCP plugin. All testing runs via Gradle tasks — no Python, no Docker, no external dependencies beyond Java.

The pipeline is a four-phase verification system where each phase bootstraps the next:

Phase 0: In-JVM Service Oracle Tests
  → defines "what correct behavior IS" for every GitBucket service method
  ↓
Phase 1: In-JVM Scalatra Route Discovery
  → discovers all API routes via reflection (not source parsing)
  → outputs route manifest (JSON)
  ↓
Phase 2: External HTTP API Probe
  → probes every discovered route against live GitBucket instances
  → sweep testing across versions for regression detection
  → outputs JSON + markdown compatibility matrices
  ↓
Phase 3: MCP Handler Oracle Verification
  → proves MCP plugin handlers match service-layer oracle results
  → proves MCP succeeds where REST API fails
  → (blocked on MCP plugin implementation)

Why This Pipeline

GitBucket has no official API documentation. The only reliable source of truth is the source code. Previous testing approached this from outside via Python scripts and HTTP — which can only test REST endpoints, cannot call internal services, and cannot discover routes dynamically.

This pipeline turns GitBucket's own runtime into the oracle:

• Phase 0 uses GitBucket's ServiceSpecBase test pattern to define correct behavior at the service level
• Phase 1 uses JVM reflection to discover routes from the compiled classpath — no source parsing
• Phase 2 tests those routes against any .war version for regression detection
• Phase 3 closes the loop by proving the MCP plugin matches the oracle

Key Design Decisions

Decision	Rationale
All Java/Gradle — no Python	Single ecosystem, no Node.js or Docker, Gradle tasks for everything
Reflection over source parsing	Routes discovered from compiled classpath — canonical and version-accurate
.war download from GitHub Releases	Tests any version without rebuild; default latest, optional sweep
Route manifest from current classpath	Sweep probes current routes against older versions → regression/improvement matrix
JSON + markdown output	JSON is canonical/machine-readable; markdown is derived/human-readable
git clone --mirror for test data	Real commit history from public repos (gitbucket/gitbucket); no private data
Programmatic issue/PR creation	Deterministic canonical dataset (3 issues, 2 PRs) on top of git mirror

Specs

Spec	Issue	Phase	Implementable Now?
Testing Pipeline (Phases 0-2)	#12	0, 1, 2	Yes
MCP Handler Verification (Phase 3)	#13	3	No — blocked on MCP plugin

Related

• Contents API regression in 4.46.0: GET /contents returns 404, PUT /contents returns 500 (was working in 4.45.0) gitbucket/gitbucket#4006 — Contents API regression (probe will provide reproducible evidence)
• [SPEC] GitBucket API Local Test Probe — Source-Based Endpoint Discovery Against Local Java Instance Brothertown-Language/snea-shoebox-editor#1058 — Superseded (Python probe replaced by Java pipeline)
• [SPEC] GitBucket API Capability Map & MCP Plugin Roadmap #10 — API capability map (probe output will auto-generate updated version matrices)