Nightly job silently skips half of its inputs

[shunk031]

Summary

The nightly job reports success after processing only half of its inputs. The omitted inputs are not reported or retried, so the output appears healthy even though it is incomplete.

Impact

Each nightly run can publish partial results without an alert. Downstream consumers that trust the successful job status may make decisions from missing data.

Steps to reproduce

1. Run the nightly job with a known set of inputs.
2. Compare the input IDs and count with the IDs and count in the output.
3. Observe that only half of the inputs are processed while the job exits successfully and emits no warning or error.

Expected behavior

Every input is processed exactly once. If an input cannot be processed, the job records its ID, retries it when appropriate, and fails or raises an actionable alert when the final input and output sets do not match.

Actual behavior

The job processes only half of the inputs and completes successfully without identifying the omitted inputs.

Acceptance criteria

• Identify and fix the condition that drops half of the inputs.
• Add a regression test that verifies the complete input set is processed.
• Record input, processed, skipped, failed, and retried counts for each nightly run.
• Fail or raise an actionable alert when any input is unaccounted for, including enough information to identify it.