Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
beee3f9
standardise configs across pipelines
eglucas Jun 25, 2026
01acfcd
pipeline READMEs and template
eglucas Jun 25, 2026
6428378
WIP repo-level README
eglucas Jun 25, 2026
1187ba7
update running-pipelines.md
eglucas Jun 29, 2026
5ca6b2a
WIP adding pipelines#
eglucas Jun 29, 2026
bfd1e3c
Merge branch 'dev' into pipeline-documentation
eglucas Jul 1, 2026
f5d3bd4
update pipeline docs
eglucas Jul 2, 2026
e3d428d
update documentation
eglucas Jul 2, 2026
2b59f84
fix typos
eglucas Jul 2, 2026
9eed7ec
editing docs content
eglucas Jul 2, 2026
0df7502
revert example pipeline changes to preserve tests
eglucas Jul 2, 2026
5761aa5
Replace stale infrastructure README with headings-only skeleton
AlexRossKnox Jul 27, 2026
cef69bd
Add infrastructure readme
AlexRossKnox Jul 27, 2026
fb5c570
add infrastructure refs to linked docs
eglucas Jul 31, 2026
efd8b4e
Merge pull request #83 from nestauk/pipeline-documentation
eglucas Jul 31, 2026
0857419
update repo-level README with infra section
eglucas Jul 31, 2026
a0aa5a6
Merge pull request #92 from nestauk/update-readme-infra-section
eglucas Jul 31, 2026
ec706b7
TDCV changes in config and zero-VAT electricity periods in config and…
eglucas Aug 20, 2026
439b76b
Notebook for checking that changes worked correctly
eglucas Aug 20, 2026
e332281
Fix gold node for handling VAT
eglucas Aug 26, 2026
bb1571d
Updated checks notebook
eglucas Aug 26, 2026
710384c
Merge pull request #93 from nestauk/85-energy_price_cap_levels_annex_…
eglucas Aug 26, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 0 additions & 25 deletions .github/PULL_REQUEST_TEMPLATE/dev_template.md

This file was deleted.

30 changes: 26 additions & 4 deletions .github/PULL_REQUEST_TEMPLATE/prod_template.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,31 @@
## Dev → Prod Promotion Checklist
<!-- PR template for promoting from dev to prod -->

---

## Description

### Pipeline description
<!-- Which pipeline(s) does this promote? What changed? -->

## Type of promotion
- [ ] Existing pipeline - code/data update
- [ ] New pipeline - first promotion to prod

## Instructions for reviewer

<!-- Anything they should pay particular attention to -->

## Checklist

### Pipeline run
- [ ] Manual pipeline run completed successfully on dev after merging to dev
- [ ] Manual pipeline run completed successfully on dev after merging to dev (link the run below)
- [ ] Dev S3 output spot-checked; row counts and values look reasonable
- [ ] No unexpected nulls or schema changes in output parquet files

### Image freshness
- [ ] `dev-latest` in ECR was rebuilt from the tip of `dev` that includes this PR's changes
(check the "Build and push Docker Image to ECR" run in Actions, as promotion only re-tags
the existing image, it doesn't rebuild)

### Code quality
- [ ] run-tests workflow passed on this PR
- [ ] No hardcoded credentials or environment-specific values
Expand All @@ -17,5 +35,9 @@
- [ ] No changes to silver/gold dataset column names or types that would break Superset
<!-- If there are column/type changes, describe them here and provide instructions on changes needed in Superset -->

### Notes
### After merging
- [ ] Trigger the "Run pipeline in prod" workflow to actually refresh prod data — merging this
PR only re-tags the image, it does not run the pipeline. Requires approval from at least one of named CODEOWNERS.

## Notes
<!-- Any context the approver should know before merging -->
31 changes: 31 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
<!-- PR template for merges to dev -->

---

## Description

<!-- What does this change, and why? -->

Fixes # <!-- Issue number -->

## Type of change

- [ ] New pipeline
- [ ] Change to an existing pipeline
- [ ] Package / infrastructure change
- [ ] Docs only

## Checklist

- [ ] `uv run pytest` passes locally
- [ ] `uv run pre-commit run --all-files` passes (ruff, gitleaks, etc.)
- [ ] If this adds a new pipeline: it's registered in `pipelines.yaml` and has its own `README.md`
- [ ] I've merged the latest `dev` into this branch

## Testing notes

<!-- How did you verify this works? e.g. ran locally with `--stage all`, ran via the "Test pipeline in dev" GitHub Actions workflow -->

## Instructions for reviewer

<!-- Anything they should pay particular attention to -->
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,11 @@ infrastructure/.env
*.parquet
/data/

# Notebooks
# Scratch notebooks aren't meant to be committed — see docs/DEVELOPMENT.md
*.ipynb
.ipynb_checkpoints/

# Logs
*.log

Expand Down
6 changes: 3 additions & 3 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ repos:

### Python Tools ###
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.15.20
rev: v0.16.4
hooks:
- id: ruff-check
args: [--fix]
Expand All @@ -30,15 +30,15 @@ repos:
name: "🐍 python · Format with Ruff"

- repo: https://github.com/abravalheri/validate-pyproject
rev: v0.25
rev: '0.26'
hooks:
- id: validate-pyproject
name: "🐍 python · Validate pyproject.toml"
additional_dependencies: ["validate-pyproject-schema-store[all]"]

### Data & Config Validation ###
- repo: https://github.com/python-jsonschema/check-jsonschema
rev: 0.37.4
rev: 0.38.0
hooks:
- id: check-github-workflows
name: "🐙 github-actions · Validate gh workflow files"
Expand Down
Loading
Loading