TESTING.md

Testing Strategy

This document outlines the testing approach for GraphRAG. The goal is real assertions that verify behavior, not just happy path passthrough.

---

Testing Philosophy

1. Every interface has a mock - Enables isolated unit testing
2. Integration tests use real containers - Validates actual behavior
3. Table-driven tests - Exhaustive edge case coverage
4. Race detection required - Catches concurrency bugs
5. Error paths tested - Failure modes are first-class

---

Test Categories

Category	Purpose	Location	Run Command
Unit	Test isolated logic	*_test.go	go test ./...
Integration	Test with real backends	*_integration_test.go	go test -tags=integration ./...
Benchmark	Performance regression	*_bench_test.go	go test -bench=. ./...
Race	Concurrency safety	All tests	go test -race ./...

---

Coverage Targets

Package	Target	Notes
Core interfaces	90%+	Critical path
Store implementations	85%+	Integration-heavy
Embedding providers	85%+	API-dependent
Pipeline	90%+	Core logic
Stages	85%+	Each stage independent

---

Mock Implementations

Every interface has a corresponding mock in */mock/:

// embedding/mock/embedder.go
type Embedder struct {
    EmbedFunc      func(ctx context.Context, texts []string) ([][]float32, error)
    EmbedQueryFunc func(ctx context.Context, query string) ([]float32, error)
    dimension      int
}

func NewEmbedder(dimension int) *Embedder {
    return &Embedder{dimension: dimension}
}

func (m *Embedder) Embed(ctx context.Context, texts []string) ([][]float32, error) {
    if m.EmbedFunc != nil {
        return m.EmbedFunc(ctx, texts)
    }
    // Default: return zero vectors
    result := make([][]float32, len(texts))
    for i := range result {
        result[i] = make([]float32, m.dimension)
    }
    return result, nil
}

Mock Patterns

Configurable responses:

mock := storemock.NewVectorStore()
mock.SearchFunc = func(ctx context.Context, v []float32, limit int, f graphrag.Filter) ([]store.SearchResult, error) {
    return []store.SearchResult{
        {ID: "doc1", Score: 0.95},
        {ID: "doc2", Score: 0.87},
    }, nil
}

Error injection:

mock.SearchFunc = func(ctx context.Context, v []float32, limit int, f graphrag.Filter) ([]store.SearchResult, error) {
    return nil, graphrag.ErrConnectionFail
}

Call counting:

mock := embmock.NewEmbedder(1024)
// ... run test ...
assert.Equal(t, 3, mock.EmbedCallCount())

---

Unit Test Patterns

Table-Driven Tests

func TestDocument_Validate(t *testing.T) {
    tests := []struct {
        name    string
        doc     Document
        wantErr error
    }{
        {
            name:    "valid document",
            doc:     Document{ID: "abc123", Content: "Hello world"},
            wantErr: nil,
        },
        {
            name:    "empty ID",
            doc:     Document{ID: "", Content: "Hello world"},
            wantErr: ErrInvalidInput,
        },
        {
            name:    "empty content",
            doc:     Document{ID: "abc123", Content: ""},
            wantErr: ErrInvalidInput,
        },
    }

    for _, tt := range tests {
        t.Run(tt.name, func(t *testing.T) {
            err := tt.doc.Validate()
            if tt.wantErr != nil {
                assert.ErrorIs(t, err, tt.wantErr)
            } else {
                assert.NoError(t, err)
            }
        })
    }
}

Subtest Organization

func TestPipeline(t *testing.T) {
    t.Run("Build", func(t *testing.T) {
        t.Run("empty pipeline fails", func(t *testing.T) {...})
        t.Run("single stage succeeds", func(t *testing.T) {...})
        t.Run("multiple stages chain", func(t *testing.T) {...})
    })

    t.Run("Execute", func(t *testing.T) {
        t.Run("returns top K results", func(t *testing.T) {...})
        t.Run("respects context cancellation", func(t *testing.T) {...})
        t.Run("stage error stops pipeline", func(t *testing.T) {...})
    })
}

---

Integration Tests

Test Isolation Pattern

Integration tests run in parallel by default. Each test MUST have isolated data to prevent interference.

Required Pattern:

//go:build integration

type testEnv struct {
    store       *neo4j.Store
    vectorStore *qdrant.Store
    fixture     *testutil.TestFixture
}

func setupTestEnv(t *testing.T, testName string) *testEnv {
    cfg := testutil.LoadTestConfig()

    // Unique prefix per test (timestamp-based)
    fixture := testutil.NewFixture("mytest_" + testName)

    store, err := neo4j.New(
        neo4j.WithURI(cfg.Neo4jURI),
        neo4j.WithAuth(cfg.Neo4jUser, cfg.Neo4jPass),
    )
    require.NoError(t, err)

    // Unique collection per test
    vectorStore, err := qdrant.New(
        qdrant.WithGRPC(cfg.QdrantHost, cfg.QdrantPort),
        qdrant.WithCollection(fmt.Sprintf("test_%s", fixture.Prefix())),
    )
    require.NoError(t, err)

    env := &testEnv{store: store, vectorStore: vectorStore, fixture: fixture}
    t.Cleanup(func() {
        cleanupTestEnv(t, env)
        store.Close()
        vectorStore.Close()
    })
    return env
}

func cleanupTestEnv(t *testing.T, env *testEnv) {
    ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
    defer cancel()

    // PREFIX-SPECIFIC cleanup (NOT global DETACH DELETE)
    query := fmt.Sprintf(
        "MATCH (n:Document) WHERE n.id STARTS WITH '%s' DETACH DELETE n",
        env.fixture.Prefix(),
    )
    _ = env.store.ExecuteWrite(ctx, query)
}

Critical Rules:

1. Never use global cleanup like MATCH (n) DETACH DELETE n - this deletes other tests' data
2. Always use prefix-specific cleanup with WHERE n.id STARTS WITH '<prefix>'
3. Create unique Qdrant collections per test using fixture.Prefix()
4. Use hash-based mock embeddings - same text = same embedding for deterministic results

Docker Compose Test Infrastructure

For integration tests, use docker-compose.test.yml with port offset:

Service	Production Port	Test Port
Neo4j	7687	17687
Qdrant	6334	16334
Redis	6379	16379

# Start test containers
docker compose -f docker-compose.test.yml up -d

# Run integration tests
go test -tags=integration ./...

Testcontainers Setup

//go:build integration

package neo4j_test

import (
    "context"
    "testing"

    "github.com/testcontainers/testcontainers-go"
    "github.com/testcontainers/testcontainers-go/modules/neo4j"
)

func TestNeo4jStore_Integration(t *testing.T) {
    ctx := context.Background()

    // Start Neo4j container
    container, err := neo4j.RunContainer(ctx,
        testcontainers.WithImage("neo4j:5.26-community"),
        neo4j.WithAdminPassword("testpassword"),
    )
    require.NoError(t, err)
    defer container.Terminate(ctx)

    // Get connection details
    uri, err := container.BoltUrl(ctx)
    require.NoError(t, err)

    // Create store
    store, err := neo4jstore.New(
        neo4jstore.WithURI(uri),
        neo4jstore.WithAuth("neo4j", "testpassword"),
    )
    require.NoError(t, err)
    defer store.Close()

    // Run tests
    t.Run("Store and Get", func(t *testing.T) {
        doc := &graphrag.Document{
            ID:      "test-doc-1",
            Content: "Test content",
        }

        err := store.Store(ctx, doc)
        require.NoError(t, err)

        retrieved, err := store.Get(ctx, doc.ID)
        require.NoError(t, err)
        assert.Equal(t, doc.Content, retrieved.Content)
    })
}

Shared Test Fixtures

// internal/testutil/fixtures.go

func TestDocuments() []*graphrag.Document {
    return []*graphrag.Document{
        {ID: "doc1", Content: "Authentication uses JWT tokens"},
        {ID: "doc2", Content: "Caching layer uses Redis"},
        {ID: "doc3", Content: "Database is PostgreSQL"},
    }
}

func TestVectors(dimension int) [][]float32 {
    // Deterministic test vectors
    return [][]float32{
        makeVector(dimension, 0.1),
        makeVector(dimension, 0.2),
        makeVector(dimension, 0.3),
    }
}

---

Error Path Testing

Connection Failures

func TestVectorStore_ConnectionFailure(t *testing.T) {
    store, err := qdrant.New(
        qdrant.WithHost("nonexistent.local", 6334),
        qdrant.WithTimeout(100 * time.Millisecond),
    )
    require.NoError(t, err) // Constructor doesn't connect

    ctx := context.Background()
    _, err = store.Search(ctx, testVector, 10, nil)

    assert.ErrorIs(t, err, graphrag.ErrConnectionFail)
}

Rate Limiting

func TestEmbedder_RateLimitRetry(t *testing.T) {
    attempts := 0
    server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
        attempts++
        if attempts < 3 {
            w.Header().Set("Retry-After", "1")
            w.WriteHeader(429)
            return
        }
        json.NewEncoder(w).Encode(voyageResponse{...})
    }))
    defer server.Close()

    embedder, _ := voyage.New(
        voyage.WithBaseURL(server.URL),
        voyage.WithAPIKey("test-key"),
        voyage.WithRetry(5, 100*time.Millisecond),  // maxAttempts, baseDelay
    )

    _, err := embedder.Embed(context.Background(), []string{"test"})

    assert.NoError(t, err)
    assert.Equal(t, 3, attempts)
}

Context Cancellation

func TestPipeline_ContextCancellation(t *testing.T) {
    slowStage := &SlowStage{delay: 5 * time.Second}

    pipeline := retrieval.NewPipeline().
        AddStage(stages.Semantic(mockStore, mockEmbedder, 50)).
        AddStage(slowStage).
        Build()

    ctx, cancel := context.WithTimeout(context.Background(), 100*time.Millisecond)
    defer cancel()

    _, err := pipeline.Execute(ctx, "query", 10)

    assert.ErrorIs(t, err, context.DeadlineExceeded)
}

---

Benchmark Tests

func BenchmarkPipeline_SemanticOnly(b *testing.B) {
    ctx := context.Background()
    pipeline := presets.Minimal(mockVectorStore, mockEmbedder)

    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        pipeline.Execute(ctx, "test query", 10)
    }
}

func BenchmarkEmbedder_BatchSize(b *testing.B) {
    embedder := embmock.NewEmbedder(1024)
    texts := generateTexts(100)

    for _, batchSize := range []int{1, 10, 50, 100} {
        b.Run(fmt.Sprintf("batch_%d", batchSize), func(b *testing.B) {
            for i := 0; i < b.N; i++ {
                for j := 0; j < len(texts); j += batchSize {
                    end := min(j+batchSize, len(texts))
                    embedder.Embed(context.Background(), texts[j:end])
                }
            }
        })
    }
}

---

Race Detection

All tests run with race detector in CI:

# .github/workflows/ci.yml
- name: Test with Race Detector
  run: go test -race -v ./...

Common race conditions to watch for:

• Concurrent map access in context values
• Shared embedder/store state
• Pipeline result aggregation

---

CI Pipeline

name: CI

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-go@v5
        with:
          go-version: '1.22'

      - name: Lint
        run: |
          go vet ./...
          go fmt ./... && git diff --exit-code

      - name: Unit Tests
        run: go test -race -coverprofile=coverage.out ./...

      - name: Integration Tests
        run: go test -race -tags=integration ./...

      - name: Upload Coverage
        uses: codecov/codecov-action@v4
        with:
          files: coverage.out

---

Test Organization

pkg/graphrag/
├── store/
│   ├── neo4j/
│   │   ├── store.go
│   │   ├── store_test.go              # Unit tests
│   │   ├── store_integration_test.go  # Integration tests
│   │   └── testdata/                  # Test fixtures
│   └── mock/
│       ├── graph.go                   # GraphStore mock
│       ├── vector.go                  # VectorStore mock
│       └── cache.go                   # CacheStore mock
│
├── embedding/
│   ├── voyage/
│   │   ├── embedder.go
│   │   ├── embedder_test.go
│   │   └── testdata/
│   │       └── responses.json         # Recorded API responses
│   └── mock/
│       └── embedder.go
│
├── rerank/
│   ├── voyage/
│   │   └── reranker_test.go
│   ├── cohere/
│   │   └── reranker_test.go
│   └── mock/
│       └── reranker.go
│
├── retrieval/
│   ├── pipeline_test.go
│   ├── pipeline_bench_test.go
│   └── stages/
│       ├── semantic_test.go
│       ├── hydrate_test.go
│       ├── graph_test.go
│       └── rerank_test.go
│
└── internal/
    └── testutil/
        ├── containers.go              # Testcontainer helpers
        ├── fixtures.go                # Shared test data
        └── assertions.go              # Custom assertions