Summary
The nightly job completes successfully even though it processes only about half of the inputs it receives. The skipped inputs are not reported, retried, or reflected in the job status, so the resulting output looks healthy while being incomplete.
Expected behavior
Every input should be processed exactly once, or the job should clearly report which inputs were skipped and fail or retry when the final processed count does not match the input count.
Impact
Nightly results can be incomplete without an alert. Any downstream consumer that assumes a successful run contains all inputs may make decisions from partial data.
Acceptance criteria
- Record the total input count and processed, skipped, failed, and retried counts for each nightly run.
- Ensure the job does not silently drop inputs; skipped or failed inputs are retried or listed in the run output.
- Mark the run failed, or otherwise raise an actionable alert, when the processed result is incomplete.
- Add a regression test covering a full nightly input set and verifying that every input is accounted for.
Summary
The nightly job completes successfully even though it processes only about half of the inputs it receives. The skipped inputs are not reported, retried, or reflected in the job status, so the resulting output looks healthy while being incomplete.
Expected behavior
Every input should be processed exactly once, or the job should clearly report which inputs were skipped and fail or retry when the final processed count does not match the input count.
Impact
Nightly results can be incomplete without an alert. Any downstream consumer that assumes a successful run contains all inputs may make decisions from partial data.
Acceptance criteria