Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .claude/rules/configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ Proposed names (descriptive: once implemented, the code is the reference).
| `Checkpoints:BlobConnectionString`, `Checkpoints:Container` | workers | Azurite locally |
| `Consumer:InitialBackoff`, `Consumer:MaxBackoff`, `Consumer:MaxUnclassifiedAttempts` | workers | handler loop; max back-off 30 s, 5 attempts (`resilience.md`) |
| `Teamtailor:BaseUrl`, `Teamtailor:ApiKey`, `Teamtailor:ApiVersion` | workers | API version `20240904` |
| `Teamtailor:SignatureKey` | Ingress.Api | must equal `FakeTeamtailor:SignatureKey` |
| `Teamtailor:QualifiedStageName` | workers | `Qualified` locally. Only Matching.Worker maps stages, but `AddTeamtailorApi` validates it on start in both |
| `Teamtailor:Resilience:*` | workers | `HttpStandardResilienceOptions` of the API client (e.g. `Retry:MaxRetryAttempts` = 2); unset keys keep the library defaults, but the section itself must exist in every `*:Resilience` row or the host fails to build |
| `Crm:BaseUrl` | Sync.Worker | |
Expand Down
4 changes: 1 addition & 3 deletions .claude/rules/messaging.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,7 @@ Transport metadata goes in `EventData.Properties`, never in the envelope. The on

## Ingress.Api

1. Read the raw body (enable buffering). Verify signature v2 through the verifier interface from
`Infrastructure.Teamtailor` (see teamtailor rule). Invalid → 401.
MVP: signature verification is done last; until then the endpoint skips it.
1. Read the raw body once: the event id is computed over the bytes as sent.
2. Parse the webhook through the parser interface from `Infrastructure.Teamtailor`, build the envelope,
insert into `ingress.Outbox` in one local transaction, return 200.
3. If the insert fails → 503. Teamtailor may not retry; this loss is accepted (see known limitations).
Expand Down
4 changes: 2 additions & 2 deletions .claude/rules/testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ paths:
| `TalentSync.Matching.Domain.Tests` | Unit, no I/O | Suggestion validation, `ConsultantProfile` behaviour |
| `TalentSync.Matching.Infrastructure.Tests` | Unit, no network | Matching Platform client: request body, status mapping (4xx → Rejected), HTTP pipeline. CV masking. Enrichment adapter over `FakeChatClient`: outcome per mode, timeout, open circuit, shutdown |
| `TalentSync.Sync.Application.Tests`, `TalentSync.Matching.Application.Tests` | Unit, no I/O | Use cases: each fetch result and port result → use case result. Ports replaced by small hand-written stubs |
| `TalentSync.Infrastructure.Teamtailor.Tests` | Unit, no network | Signature v2, webhook parsing, stage mapping, response mapping (404 → Deleted, 4xx → Rejected). Internals via `InternalsVisibleTo` |
| `TalentSync.Infrastructure.Teamtailor.Tests` | Unit, no network | Webhook parsing, stage mapping, response mapping (404 → Deleted, 4xx → Rejected). Internals via `InternalsVisibleTo` |
| `TalentSync.Infrastructure.Messaging.Tests` | Unit, no I/O | Consumer skeleton: inbox and parking lot mapping, failure classification, handler loop |
| `TalentSync.Infrastructure.Observability.Tests` | Unit, no network | `AddObservability` with the SDK enabled on a test server (the request span is a root), `SpanExportPolicy` in front of an in-memory exporter. Internals via `InternalsVisibleTo` |
| `TalentSync.Sync.Infrastructure.Tests` | Unit, no network | CRM client: status mapping (404 on delete = done, 4xx → Rejected), HTTP pipeline. Internals via `InternalsVisibleTo` |
Expand Down Expand Up @@ -79,4 +79,4 @@ pass by editing a report header or the test. The evaluation itself never runs in
## Packages

Ask before adding any package, including test helpers such as `Microsoft.Extensions.TimeProvider.Testing`
(`FakeTimeProvider` for the signature replay window).
(`FakeTimeProvider`).
16 changes: 16 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# dotnet format --verify-no-changes runs in CI (Build job). end_of_line is left unset on purpose: the index holds LF,
# Windows checkouts get CRLF through core.autocrlf, and both must pass. charset is unset too: dotnet ef writes
# migrations with a BOM, so every new migration would fail the check.
root = true

[*]
indent_style = space
indent_size = 4
insert_final_newline = true
trim_trailing_whitespace = true

[*.{csproj,props,targets,slnx,xml,json,yml,yaml}]
indent_size = 2

[*.md]
trim_trailing_whitespace = false
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,9 @@ jobs:
- name: Build
run: dotnet build --no-restore -c Release

- name: Check formatting
run: dotnet format --verify-no-changes --no-restore

# Builds again instead of passing bin/ between jobs: a restore and build take under a minute, artifacts of
# every project's output would be slower and fragile.
test:
Expand Down
5 changes: 2 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,7 @@ Key names, owners and gotchas: `.claude/rules/configuration.md` (loads when `app
In scope, minimal version of everything:
docker compose (Event Hubs emulator, Azurite, SQL Server), the three simulators, Ingress with outbox and relay,
Sync.Worker (inbox, ACL, CRM upsert, resilience, parking lot), Matching.Worker (Qualified rule, LLM enrichment
with stored results and fallback, profile upsert). Teamtailor signature v2 verification is in scope but done last.
with stored results and fallback, profile upsert).
Observability (ADR 010): OpenTelemetry traces, metrics and logs, Aspire Dashboard locally.

Matching.Worker ignores `candidate.*` events in the MVP (rule D6).
Expand All @@ -130,8 +130,7 @@ architecture tests.
- `dotnet test` (exit code 8 = no tests ran, e.g. a project without tests yet or a filter that matched nothing)
- One scenario: `dotnet test --project tests/TalentSync.IntegrationTests --filter-method "*.S2_*"`
- Unit tests only: `dotnet test --project tests/TalentSync.Matching.Domain.Tests`
- EF migration, example for Sync (descriptive: verify once the first `DbContext` exists; needs the `dotnet-ef`
tool and `Microsoft.EntityFrameworkCore.Design`, ask before adding them):
- EF migration, example for Sync (`dotnet-ef` comes from the tool manifest `dotnet-tools.json`, run `dotnet tool restore` once):
`dotnet ef migrations add <Name> --project src/Sync/TalentSync.Sync.Infrastructure --startup-project src/Sync/TalentSync.Sync.Worker --context SyncDbContext`
- Anthropic key (I set it myself): `dotnet user-secrets set "Llm:Anthropic:ApiKey" "<key>" --project src/Matching/TalentSync.Matching.Worker`,
then `dotnet user-secrets set "Llm:Provider" "Anthropic" --project src/Matching/TalentSync.Matching.Worker`
88 changes: 77 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ matching platform, with LLM enrichment of consultant profiles.
A portfolio project: a simplified model of a real-world integration. Not production code, but designed to behave
correctly under failure: duplicate deliveries, retries, crashes and downstream outages.

> **Status: work in progress.** Documentation and acceptance scenarios first, code in small steps.

## How it works

```mermaid
Expand All @@ -30,24 +28,80 @@ Key decisions:
- **Notify-then-fetch**: events carry only IDs; workers fetch current state from the API.
- **Outbox** in Ingress; **at-least-once** delivery; **idempotent upserts** keyed by `candidateId`.
- **Parking lot** instead of a dead-letter queue, which Event Hubs doesn't have.
- **LLM results stored** by candidate and CV hash before the profile upsert, so retries replay the same decision.
- **LLM results stored** by candidate, CV hash and prompt version before the profile upsert, so retries replay the
same decision.

More: [architecture](docs/architecture.md), [diagrams](docs/diagrams.md), [domain](docs/domain.md),
[acceptance scenarios](docs/acceptance-scenarios.md).

Stack: .NET 10, Azure Event Hubs SDK (no messaging framework), SQL Server 2025 + EF Core,
`Microsoft.Extensions.Http.Resilience` (Polly v8), `Microsoft.Extensions.AI`, xUnit, Docker Compose.
`Microsoft.Extensions.Http.Resilience` (Polly v8), `Microsoft.Extensions.AI` with the Anthropic SDK (Claude),
OpenTelemetry + Aspire Dashboard, xUnit v3 + Testcontainers, Docker Compose, GitHub Actions.

## Observability

One webhook is one trace across the three hosts, through the outbox and Event Hubs
([ADR 010](docs/adr/010-observability.md)). Below: a job application reaches the qualified stage. Ingress stores
the webhook and the relay publishes it. Matching.Worker checks its inbox, fetches the application and the candidate,
finds no stored enrichment, calls Claude, stores the result, upserts the profile, writes the inbox and checkpoints.
Sync.Worker ignores `job_application.*` events, so its branch is only a checkpoint.

![Trace of a qualified job application in the Aspire Dashboard](docs/images/trace-application-qualified.jpeg)

## Testing

- **Acceptance scenarios** ([docs/acceptance-scenarios.md](docs/acceptance-scenarios.md)) are the definition of done:
Given/When/Then examples of the domain rules and invariants, written before the code. Each automated scenario has
exactly one integration test named after its ID, e.g. `S4` (CRM outage delays but does not lose events) or `M3`
(LLM timeout yields a `NotEnriched` profile). The tests run the three hosts and the simulators in process against
SQL Server, Azurite and the Event Hubs emulator in Testcontainers, and break the simulators on purpose.
- **Unit tests** per project: the domain model, use cases, failure classification in the consumer skeleton, HTTP
clients and their resilience, the Teamtailor anti-corruption layer.
- **CI** (GitHub Actions) builds and runs everything on every PR, with a test report and a coverage summary.
- **Prompt evaluation** ([ADR 009](docs/adr/009-prompt-evaluation.md)): the enrichment prompt is scored offline
against a golden set of fictional CVs (`eval/cases/`, including a prompt injection and a Polish CV) for validity,
recall, precision, hallucinations, seniority and stability. Run by hand, never in CI, because it calls the real
model; reports are in [`eval/reports/`](eval/reports/). CI only checks that a report exists for the current prompt.

## Decisions

The reasoning is in [architecture decision records](docs/adr/README.md):

| ADR | Decision |
|---|---|
| [001](docs/adr/001-project-structure.md) | Project structure by context, then by layer; a project wherever the compiler should enforce a rule |
| [002](docs/adr/002-event-hubs-over-service-bus.md) | Event Hubs over Service Bus: replay and independent consumers, at the cost of a DLQ |
| [003](docs/adr/003-notify-then-fetch.md) | Notify-then-fetch: the event is a signal, the API state is the truth |
| [004](docs/adr/004-transactional-outbox.md) | Transactional outbox in Ingress, one relay |
| [005](docs/adr/005-idempotency.md) | Idempotent effects, per-worker inbox, checkpoint after processing, stored LLM results |
| [006](docs/adr/006-partition-key-candidate-id.md) | Partition key = `candidateId`: ordering per candidate |
| [007](docs/adr/007-llm-enrichment.md) | LLM enrichment: own port, PII masking, validation, timeout and fallback to `NotEnriched` |
| [008](docs/adr/008-source-of-truth-and-sync-direction.md) | One-way sync from Teamtailor, ownership per field |
| [009](docs/adr/009-prompt-evaluation.md) | Offline evaluation of the enrichment prompt |
| [010](docs/adr/010-observability.md) | OpenTelemetry, one trace per webhook through the outbox and Event Hubs |

## Running locally

Requires .NET 10 SDK and Docker. Target commands while the setup is being built:
Requires .NET 10 SDK and Docker.

Tests need nothing else: the integration tests start SQL Server, Azurite and the Event Hubs emulator with
Testcontainers and run the hosts and simulators in process.

```bash
docker compose -f deploy/docker-compose.yml up -d
dotnet build
dotnet test
```

For a manual run, set the SQL password once in `deploy/.env` (copy `deploy/.env.example`) and give the three hosts
the connection string in user secrets:

```bash
for p in Ingress/TalentSync.Ingress.Api Sync/TalentSync.Sync.Worker Matching/TalentSync.Matching.Worker; do
dotnet user-secrets set "ConnectionStrings:Sql" \
"Server=localhost,1433;Database=TalentSync;User Id=sa;Password=<SQL_SA_PASSWORD>;TrustServerCertificate=True" \
--project src/$p
done
```

`Llm:Provider` is `Fake` by default, so the system runs without an LLM key. For Claude, keep the key in user secrets
and switch the provider (every real call is billed):

Expand All @@ -56,10 +110,11 @@ dotnet user-secrets set "Llm:Anthropic:ApiKey" "<key>" --project src/Matching/Ta
dotnet user-secrets set "Llm:Provider" "Anthropic" --project src/Matching/TalentSync.Matching.Worker
```

To click through the whole flow by hand (success, CRM outage, candidate without a CV), start the containers and the
three hosts with `deploy/run-local.sh` (Git Bash; on Windows each host opens in its own Windows Terminal tab), then use
To click through the whole flow by hand (success, CRM outage, candidate without a CV), run `deploy/run-local.sh`
(Git Bash). It starts the containers, builds, applies the EF migrations and starts the three hosts (on Windows each
in its own Windows Terminal tab). Then use
[`docs/e2e.http`](docs/e2e.http). Traces, metrics and logs of the three hosts are in the Aspire Dashboard at
http://localhost:18888 (ADR 010).
[http://localhost:18888](http://localhost:18888) (ADR 010).

## Simplifications

Expand All @@ -70,6 +125,15 @@ http://localhost:18888 (ADR 010).
- Undocumented Teamtailor behaviour is an explicit assumption, isolated in the adapter: candidate id in job
application webhooks, `404` = deleted resource, stage via `include=stage`.

## Known limitations

Accepted for the MVP and listed in [architecture](docs/architecture.md#known-limitations-to-be-covered-by-adrs), e.g.:

- If SQL Server is down when a webhook arrives, Ingress returns 503 and Teamtailor may not retry, so the event is
lost. A reconciliation job against the API would close this gap.
- One outbox relay replica: several would publish a candidate's events out of order.
- A profile is not refreshed when the candidate's CV changes, only on the next qualification.

## Working with a coding agent

The code is written in small steps with a coding agent. `CLAUDE.md` and `.claude/rules/` hold its instructions,
Expand All @@ -79,4 +143,6 @@ a test pass.

---

Not affiliated with Teamtailor; built from its public documentation.
Not affiliated with Teamtailor; built from its public documentation.

Shared for review as a portfolio project; no license is granted for reuse.
14 changes: 12 additions & 2 deletions deploy/run-local.sh
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Starts everything for a local run: docker compose (infrastructure and simulators), then Ingress.Api, Sync.Worker
# and Matching.Worker. On Windows each host gets its own Windows Terminal tab (stop it with Ctrl+C there);
# Starts everything for a local run: docker compose (infrastructure and simulators), build, EF migrations, then
# Ingress.Api, Sync.Worker and Matching.Worker. On Windows each host gets its own Windows Terminal tab (stop it with Ctrl+C there);
# elsewhere they run in this terminal and Ctrl+C stops all three.
# Containers keep running: docker compose -f deploy/docker-compose.yml stop
set -euo pipefail
Expand All @@ -23,6 +23,16 @@ docker compose -f deploy/docker-compose.yml up -d --wait --force-recreate --rene
echo "Building..."
dotnet build TalentSync.slnx --nologo --verbosity quiet

# Before the hosts start, so neither the relay nor a worker meets a missing table. Applies only pending migrations.
# The connection string comes from each host's user secrets (README, "Running locally").
echo "Applying migrations..."
dotnet tool restore >/dev/null
dotnet ef database update --no-build --project src/Ingress/TalentSync.Ingress.Api --context IngressDbContext
dotnet ef database update --no-build --project src/Sync/TalentSync.Sync.Infrastructure \
--startup-project src/Sync/TalentSync.Sync.Worker --context SyncDbContext
dotnet ef database update --no-build --project src/Matching/TalentSync.Matching.Infrastructure \
--startup-project src/Matching/TalentSync.Matching.Worker --context MatchingDbContext

if command -v wt.exe >/dev/null 2>&1; then
for i in "${!titles[@]}"; do
# MSYS_NO_PATHCONV: Git Bash would otherwise turn `/k` into a Windows path.
Expand Down
14 changes: 1 addition & 13 deletions docs/acceptance-scenarios.md
Original file line number Diff line number Diff line change
Expand Up @@ -163,7 +163,7 @@ Scenario: M8 Candidate update is ignored by Matching

## Ingress

The test sends the request to Ingress itself. I3–I4 are done last in the MVP (signature v2).
The test sends the request to Ingress itself.

```gherkin
# Illustrates: invariant 10, ADR 004 (transactional outbox)
Expand All @@ -180,18 +180,6 @@ Scenario: I2 Event Hubs outage delays but does not lose events
Then Ingress responds 200
When Event Hubs becomes available
Then the hub receives the event with that EventId

# Illustrates: Teamtailor signature v2
Scenario: I3 Invalid signature is rejected
When a webhook with a TT-Signature computed with the wrong key is sent to Ingress
Then Ingress responds 401
And ingress.Outbox has no row for that EventId

# Illustrates: Teamtailor signature v2 (replay window)
Scenario: I4 Stale timestamp is rejected
When a correctly signed webhook with a timestamp 6 minutes in the past is sent to Ingress
Then Ingress responds 401
And ingress.Outbox has no row for that EventId
```

## Observability
Expand Down
2 changes: 0 additions & 2 deletions docs/adr/003-notify-then-fetch.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,6 @@ The webhook is treated as a signal, and the Teamtailor API as the source of trut
`GET /v1/job-applications/{id}?include=stage`) and acts on that, not on the webhook.
- A 404 from the API maps to `Deleted`, never to an error.

This is **not** a workaround for an unsigned payload: signature v2 signs the whole body, and Ingress verifies it.

## Consequences

- Stale or reordered events are harmless: each worker acts on the current API state, whatever order the events
Expand Down
3 changes: 1 addition & 2 deletions docs/adr/004-transactional-outbox.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ API, the CRM or the Matching Platform; its only job is to accept the webhook and

Ingress persists the event before answering, and a separate relay publishes it.

- The webhook endpoint verifies the signature, parses the webhook, builds the envelope (IDs only) and inserts it
- The webhook endpoint parses the webhook, builds the envelope (IDs only) and inserts it
into `ingress.Outbox` in one local transaction, then returns 200. If the insert fails, it returns 503.
- The outbox relay is a `BackgroundService` inside Ingress.Api. In a loop it selects unpublished rows
`WITH (UPDLOCK, READPAST, ROWLOCK)` ordered by `Id`, publishes each with partition key `candidateId` through a
Expand All @@ -38,4 +38,3 @@ Ingress persists the event before answering, and a separate relay publishes it.

- **Publishing to Event Hubs directly from the webhook endpoint.** Rejected: the webhook would then depend on Event
Hubs being available, and an event that fails to publish is answered with an error that Teamtailor may not retry.
TODO: confirm this rationale.
Loading
Loading