Skip to content

Commit 38376f5

Browse files
Pigbibicodex
Pigbibi
and
codex
committed
feat: publish weekly producer contract artifact
Co-Authored-By: Codex <noreply@openai.com>
1 parent 7c4104d commit 38376f5

6 files changed

Lines changed: 628 additions & 4 deletions

File tree

‎.github/workflows/rss_source_pipeline.yml‎

Lines changed: 60 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,14 @@ on:
2323
required: false
2424
default: "50"
2525
type: string
26+
period_start:
27+
description: "Manual weekly period Monday UTC (YYYY-MM-DD)."
28+
required: true
29+
type: string
30+
as_of:
31+
description: "Manual weekly as_of Sunday UTC (YYYY-MM-DD)."
32+
required: true
33+
type: string
2634
commit_outputs:
2735
description: "Commit generated live CSV outputs back to data/live."
2836
required: false
@@ -32,7 +40,7 @@ on:
3240
- "false"
3341
- "true"
3442
schedule:
35-
- cron: "15 12 * * 6"
43+
- cron: "15 0 * * 1"
3644

3745
permissions:
3846
contents: write
@@ -46,8 +54,8 @@ jobs:
4654
runs-on: ubuntu-latest
4755
timeout-minutes: 30
4856
steps:
49-
- uses: actions/checkout@v6
50-
- uses: actions/setup-python@v6
57+
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6
58+
- uses: actions/setup-python@ece7cb06caefa5fff74198d8649806c4678c61a1 # v6
5159
with:
5260
python-version: "3.11"
5361
- name: Install package
@@ -75,6 +83,47 @@ jobs:
7583
--watchlist "${WATCHLIST_PATH}" \
7684
--events data/output/rss_source_pipeline/source_events.csv \
7785
--output data/output/rss_source_pipeline/source_tracker.csv
86+
- name: Build weekly contract artifact
87+
env:
88+
INPUT_PERIOD_START: ${{ inputs.period_start || '' }}
89+
INPUT_AS_OF: ${{ inputs.as_of || '' }}
90+
PRODUCER_REF: ${{ github.sha }}
91+
run: |
92+
set -euo pipefail
93+
if [ "${GITHUB_EVENT_NAME}" = "workflow_dispatch" ]; then
94+
test -n "${INPUT_PERIOD_START}"
95+
test -n "${INPUT_AS_OF}"
96+
PERIOD_START="${INPUT_PERIOD_START}"
97+
AS_OF="${INPUT_AS_OF}"
98+
RUN_MODE=manual
99+
else
100+
RUN_DATE="$(date -u +%F)"
101+
read -r PERIOD_START AS_OF < <(RUN_DATE="${RUN_DATE}" python3 - <<'PY'
102+
import os
103+
from datetime import date, timedelta
104+
105+
today = date.fromisoformat(os.environ["RUN_DATE"])
106+
start = today - timedelta(days=today.weekday() + 7)
107+
print(start.isoformat(), (start + timedelta(days=6)).isoformat())
108+
PY
109+
)
110+
RUN_MODE=scheduled
111+
fi
112+
GENERATED_AT="$(date -u +%Y-%m-%dT%H:%M:%S.%6NZ)"
113+
python scripts/write_weekly_artifact.py \
114+
--base-dir . \
115+
--output-dir data/output/rss_source_pipeline/weekly-artifact \
116+
--feed-status data/output/rss_source_pipeline/source_fetch_status.json \
117+
--period-start "${PERIOD_START}" \
118+
--as-of "${AS_OF}" \
119+
--generated-at "${GENERATED_AT}" \
120+
--run-mode "${RUN_MODE}" \
121+
--producer-ref "${PRODUCER_REF}" \
122+
--source-provenance official_political_event_tracking_research_v1 \
123+
data/output/rss_source_pipeline/source_items.csv \
124+
data/output/rss_source_pipeline/source_events.csv \
125+
data/output/rss_source_pipeline/source_tracker.csv \
126+
data/output/rss_source_pipeline/source_fetch_status.json
78127
- name: Publish live CSV outputs to repository
79128
env:
80129
COMMIT_OUTPUTS: ${{ github.event_name == 'schedule' && 'true' || github.event.inputs.commit_outputs || 'false' }}
@@ -108,8 +157,15 @@ jobs:
108157
git push
109158
fi
110159
- name: Upload RSS source artifact
111-
uses: actions/upload-artifact@v7
160+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
112161
with:
113162
name: rss-source-pipeline
114163
path: data/output/rss_source_pipeline/
115164
if-no-files-found: error
165+
- name: Upload weekly contract artifact
166+
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7
167+
with:
168+
name: political-event-weekly-v1
169+
path: data/output/rss_source_pipeline/weekly-artifact/weekly_manifest.json
170+
if-no-files-found: error
171+
retention-days: 30

‎docs/weekly_producer_artifact_contract.md‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# Weekly producer artifact contract
2+
3+
This is the producer-side boundary for `political_event_weekly.v1`. It is a
4+
pure local adapter: callers provide the completed UTC week, `as_of`, actual
5+
UTC `generated_at`, producer full SHA, provenance identifier, input files, and
6+
feed status. The adapter never infers dates, reads the wall clock, fetches
7+
feeds, or uploads files.
8+
9+
## Artifact
10+
11+
- Name: `political-event-weekly-v1`
12+
- File set: exactly `weekly_manifest.json`
13+
- Retention: 30 days, configured by the consuming artifact workflow; it is not
14+
inferred from report content.
15+
- Bytes: canonical `political_event_weekly.v1` manifest bytes from the merged
16+
weekly manifest serializer.
17+
18+
The input file paths are repository-relative POSIX paths under the trusted
19+
`base_dir`. They must be regular, non-symlink files. Their SHA-256 and CSV row
20+
counts are read from those exact files. The feed status file must be one of
21+
the declared inputs, and its counters must agree with every feed entry. Any
22+
failed, stale, missing, partial, malformed, or mismatched input fails before
23+
the output directory is created.
24+
25+
The CLI requires all period, timestamp, producer, provenance, and path inputs
26+
explicitly. Upload availability is an external workflow gate: callers must
27+
only upload the returned artifact after successful local readback; an upload
28+
step that cannot accept the exact single-file artifact must fail the workflow,
29+
not publish a success signal.
30+
31+
The RSS source workflow now supplies this boundary without changing fetching:
32+
scheduled runs execute Monday UTC after the previous ISO week is complete and
33+
derive that week using the documented producer rule; manual runs require both
34+
`period_start` and `as_of`. It uploads the single manifest with a 30-day
35+
retention setting. Other workflows and downstream consumers are unchanged.

‎scripts/write_weekly_artifact.py‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
#!/usr/bin/env python3
2+
"""Write a producer-owned ``political_event_weekly.v1`` manifest artifact."""
3+
4+
from __future__ import annotations
5+
6+
import argparse
7+
from datetime import datetime
8+
from pathlib import Path
9+
import sys
10+
11+
sys.path.insert(0, str(Path(__file__).resolve().parents[1] / "src"))
12+
13+
from political_event_tracking_research.weekly_contract import WeeklyContractError
14+
from political_event_tracking_research.weekly_producer import write_weekly_artifact_from_files
15+
16+
17+
def _generated_at(value: str) -> datetime:
18+
if not value.endswith("Z"):
19+
raise argparse.ArgumentTypeError("generated_at_invalid")
20+
try:
21+
return datetime.fromisoformat(value[:-1] + "+00:00")
22+
except ValueError as error:
23+
raise argparse.ArgumentTypeError("generated_at_invalid") from error
24+
25+
26+
def main() -> None:
27+
parser = argparse.ArgumentParser(description="Write a validated weekly producer artifact.")
28+
parser.add_argument("paths", nargs="+", help="Explicit input files, relative to --base-dir.")
29+
parser.add_argument("--base-dir", required=True)
30+
parser.add_argument("--output-dir", required=True)
31+
parser.add_argument("--feed-status", required=True)
32+
parser.add_argument("--period-start", required=True)
33+
parser.add_argument("--as-of", required=True)
34+
parser.add_argument("--generated-at", required=True, type=_generated_at)
35+
parser.add_argument("--run-mode", required=True, choices=("scheduled", "manual"))
36+
parser.add_argument("--producer-ref", required=True)
37+
parser.add_argument("--source-provenance", required=True)
38+
args = parser.parse_args()
39+
try:
40+
write_weekly_artifact_from_files(
41+
args.paths,
42+
base_dir=args.base_dir,
43+
output_dir=args.output_dir,
44+
feed_status_path=args.feed_status,
45+
period_start=args.period_start,
46+
as_of=args.as_of,
47+
generated_at=args.generated_at,
48+
run_mode=args.run_mode,
49+
producer_ref=args.producer_ref,
50+
source_provenance=args.source_provenance,
51+
)
52+
except WeeklyContractError as error:
53+
raise SystemExit(error.code) from None
54+
55+
56+
if __name__ == "__main__":
57+
main()

0 commit comments

Comments
 (0)