Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions deploy/host.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# One image per host: pass HOST, the project name (TalentSync.Ingress.Api, TalentSync.Sync.Worker,
# TalentSync.Matching.Worker). Build context is the repository root, for Directory.Build.props and Directory.Packages.props.
# global.json is not copied: its SDK pin would fail the build whenever the image's SDK is in a lower feature band.
# All of src/ is copied before the restore, so any source change repeats it: a host references most of src/, and listing
# every project file by hand would drift from the references.
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG HOST
WORKDIR /src
COPY Directory.Build.props Directory.Packages.props ./
COPY src/ src/
RUN dotnet publish "$(find src -name "${HOST}.csproj")" -c Release -o /app

FROM mcr.microsoft.com/dotnet/aspnet:10.0
ARG HOST
ENV HOST_DLL=${HOST}.dll
WORKDIR /app
COPY --from=build /app .
USER $APP_UID
ENTRYPOINT ["sh", "-c", "exec dotnet \"$HOST_DLL\""]
23 changes: 23 additions & 0 deletions deploy/migrate.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#!/bin/sh
# Entry point of the migrations image, run by the Helm hook Job before any host rolls out (ADR 011): creates the
# database if needed, then applies the idempotent script of each schema.
# SQL_SERVER, SQL_DATABASE, SQL_USER from the environment; sqlcmd reads the password from SQLCMDPASSWORD itself.
# SQL_TRUST_SERVER_CERTIFICATE=true only for a local SQL Server with its self-signed certificate.
set -eu

trust=""
if [ "${SQL_TRUST_SERVER_CERTIFICATE:-false}" = "true" ]; then
trust="-C"
fi

# -b: stop at the first failing batch with a non-zero exit code, so the Job fails and Helm stops the upgrade.
# -I: QUOTED_IDENTIFIER ON. sqlcmd defaults to OFF, unlike ADO.NET, and filtered indexes fail without it.
run() {
/opt/mssql-tools18/bin/sqlcmd -S "$SQL_SERVER" -U "$SQL_USER" $trust -b -I "$@"
}

run -Q "IF DB_ID(N'$SQL_DATABASE') IS NULL CREATE DATABASE [$SQL_DATABASE]"
for schema in ingress sync matching; do
echo "Applying $schema"
run -d "$SQL_DATABASE" -i "/migrations/$schema.sql"
done
44 changes: 44 additions & 0 deletions deploy/migrations.Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Idempotent migration scripts of the three DbContexts and the sqlcmd that applies them (ADR 011).
# Build context is the repository root.
FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
WORKDIR /src
COPY dotnet-tools.json Directory.Build.props Directory.Packages.props ./
RUN dotnet tool restore
COPY src/ src/
# dotnet ef builds but does not restore.
RUN for host in Ingress/TalentSync.Ingress.Api Sync/TalentSync.Sync.Worker Matching/TalentSync.Matching.Worker; do \
dotnet build "src/$host" || exit 1; \
done
# dotnet ef builds each host's services to find its DbContext. It runs as Development, so appsettings.Development.json
# supplies the other keys. The connection string is a placeholder: generating a script never connects.
ENV ConnectionStrings__Sql="Server=unused"
RUN dotnet ef migrations script --no-build --idempotent --output /scripts/ingress.sql \
--project src/Ingress/TalentSync.Ingress.Api --context IngressDbContext \
&& dotnet ef migrations script --no-build --idempotent --output /scripts/sync.sql \
--project src/Sync/TalentSync.Sync.Infrastructure --startup-project src/Sync/TalentSync.Sync.Worker --context SyncDbContext \
&& dotnet ef migrations script --no-build --idempotent --output /scripts/matching.sql \
--project src/Matching/TalentSync.Matching.Infrastructure --startup-project src/Matching/TalentSync.Matching.Worker --context MatchingDbContext
# EF wraps each migration in a transaction but does not set XACT_ABORT. Without it a failing statement does not stop
# the batch, so its COMMIT keeps the rest of the migration and its history row: half a migration marked as applied.
# EF writes a UTF-8 BOM, which sqlcmd skips only at the start of a file, so it is removed before the line goes in.
RUN for f in /scripts/*.sql; do \
{ printf 'SET XACT_ABORT ON;\nGO\n'; sed '1s/^\xEF\xBB\xBF//' "$f"; } > "$f.tmp" && mv "$f.tmp" "$f" || exit 1; \
done

# sqlcmd from Microsoft's package repository, as documented for Ubuntu. The standalone tools image is from 2017 and
# the SQL Server image is over 1.5 GB. The base is the Ubuntu of the .NET 10 images.
FROM ubuntu:24.04
# libgssapi-krb5-2: the ODBC driver loads it without declaring it, so removing curl would remove it too.
RUN apt-get update \
&& apt-get install -y --no-install-recommends ca-certificates curl \
&& curl -sSLo /tmp/packages-microsoft-prod.deb https://packages.microsoft.com/config/ubuntu/24.04/packages-microsoft-prod.deb \
&& dpkg -i /tmp/packages-microsoft-prod.deb \
&& apt-get update \
&& ACCEPT_EULA=Y apt-get install -y --no-install-recommends mssql-tools18 libgssapi-krb5-2 \
&& apt-get purge -y curl \
&& apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/* /tmp/packages-microsoft-prod.deb
COPY --from=build /scripts/ /migrations/
COPY --chmod=755 deploy/migrate.sh /migrations/migrate.sh
USER ubuntu
ENTRYPOINT ["/migrations/migrate.sh"]
20 changes: 16 additions & 4 deletions docs/adr/011-kubernetes-deployment.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,10 @@ and migrations on start would race between replicas.
on an unreachable node).
- **Workers:** a Deployment each, at most as many replicas as partitions (4).
- **Probes:** `/health/live` only, never dependency checks: a SQL outage must not restart every pod.
- **Migrations:** an EF migration bundle per `DbContext`, run as a `pre-install` / `pre-upgrade` hook Job. Helm runs
it before the hosts roll out and stops the upgrade if it fails.
- **Migrations:** an idempotent EF migration script per `DbContext`, generated when the image is built and applied
with `sqlcmd` by a `pre-install` / `pre-upgrade` hook Job. The database is ready before any host rolls out; if a
migration fails, Helm stops the upgrade and every host keeps its previous version. The Job needs only the SQL
connection.
- **Secrets** in a Kubernetes Secret created from the gitignored `deploy/.env` by the local run script, never in
chart values; the chart only references it by name.
A Kubernetes Secret is base64, not encryption, and readable by anyone who can read Secrets or create pods in the
Expand All @@ -39,7 +41,10 @@ and migrations on start would race between replicas.
effects are idempotent and LLM results are stored (invariants 2, 3, 5).
- Ingress.Api waits briefly before stopping (`preStop`), so a webhook that arrives after SIGTERM is not refused and
lost.
- Migrations must stay compatible with the previous code version, since old pods run during a rollout.
- A failed migration is rolled back by its own transaction; migrations that ran before it stay applied. Being
backward compatible, they need no rollback, and the next upgrade skips them.
- Migrations must stay compatible with the previous code version, since old pods run during a rollout. Schema
changes are forward only: a Helm rollback restores the previous pods, not the previous schema.
- Not verified yet: whether pods in kind on Docker Desktop (Windows) reach the containers in docker compose.

## Alternatives considered
Expand All @@ -49,4 +54,11 @@ and migrations on start would race between replicas.
- **SQL Server and Event Hubs in the cluster.** Rejected: production uses managed services.
- **Kustomize.** Plain YAML, built into `kubectl`. Rejected: the migration Job would need its ordering in a script,
and there is no release history or rollback.
- **Migrations on host start.** Rejected: replicas race.
- **An init container per host, migrating its own schema.** A failed migration would block only its host. Rejected:
hosts could run different versions for a while, every pod start would run migrations, and replicas starting
together would need a lock.
- **Migrations in the host process (`Database.Migrate()`).** Rejected: replicas race, and the application would need
DDL rights and migration code.
- **An EF migration bundle.** Rejected for now: without a design-time `DbContext` factory it builds the host's
services, so it would need the worker's configuration and secrets. Worth it once a migration renames or drops
a column, which an idempotent script can fail on.
6 changes: 4 additions & 2 deletions docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,9 @@ eval/ offline evaluation of the enrichment
reports/ committed reports; local/ is gitignored
run-eval.sh runs the evaluation locally; provider, runs, output and model as variables at the top
deploy/ docker-compose.yml (infra + simulators + Aspire Dashboard), simulator.Dockerfile,
eventhubs/Config.json, run-local.sh (containers, migrations, the three hosts)
eventhubs/Config.json, run-local.sh (containers, migrations, the three hosts),
host.Dockerfile (one image per host), migrations.Dockerfile + migrate.sh
(idempotent EF scripts applied with sqlcmd, ADR 011)
docs/ domain.md, architecture.md, acceptance-scenarios.md, diagrams.md,
adr/ (index: adr/README.md), e2e.http (end-to-end demo by hand),
images/ (screenshots for the README)
Expand Down Expand Up @@ -215,4 +217,4 @@ Written ADRs are in `docs/adr/` (index: `docs/adr/README.md`). This table summar
| [008](adr/008-source-of-truth-and-sync-direction.md) | Source of truth and sync direction: one way from Teamtailor, ownership per field, full replace of the fields we own (last write wins) |
| [009](adr/009-prompt-evaluation.md) | Offline evaluation of the enrichment prompt: golden set of fictional CVs, metrics in code, report per `(PromptVersion, GoldenSetHash)`, run manually never in CI; a CI test only checks that a report exists for the current pair |
| [010](adr/010-observability.md) | Observability: OpenTelemetry over OTLP, Aspire Dashboard locally, one `Infrastructure.Observability` project referenced by the hosts, trace context propagated explicitly through the outbox and `EventData` properties, one consumer span per attempt |
| [011](adr/011-kubernetes-deployment.md) | Kubernetes deployment: only our hosts in the cluster (kind locally), infrastructure in docker compose, one Helm chart, relay switched by `OutboxRelay:Enabled` and run as a one-replica StatefulSet, migrations as a Helm hook Job |
| [011](adr/011-kubernetes-deployment.md) | Kubernetes deployment: only our hosts in the cluster (kind locally), infrastructure in docker compose, one Helm chart, relay switched by `OutboxRelay:Enabled` and run as a one-replica StatefulSet, migrations as a Helm hook Job before any host rolls out |
Loading