Skip to content

Distinct initiated cells can receive the same cell_uid #72

Description

@RBhupi

Pipeline stage where the bug occurs

Acquisition — downloading or queuing NEXRAD files

Adapt version

0.1.4

Radar site and data source

KMRX

What happened?

A live KMRX run (2026-08-12 00:14:15 UTC) produced two distinct initiating cells in the same scan with the
same cell_uid:

  • cell label 18TKR02LX0GX
  • cell label 23TKR02LX0GX

cell_volume_stats correctly failed because its primary key is
(run_id, scan_time, cell_uid). Earlier tracking persistence can also upsert one
of the colliding cells_by_scan rows, losing an observation before the failure.

Root Cause

New-cell identity uses a quantized v1 birth signature composed of time,
latitude/longitude, reflectivity, ZDR, and area. Distinct cells may share every
quantized value, so they deterministically receive the same UID. This is a
signature collision, not necessarily a digest/hash collision.

Steps to reproduce

(adapt_env) adapt-repo % adapt run-nexrad config.yaml --radar KMRX
ARM Adapt v0.1.3.post39+dirty
Copyright © 2026, UChicago Argonne, LLC.
See the LICENSE (https://arm-doe.github.io/Adapt/license.html) for terms and disclaimer.

2026-08-12 00:12:39,865 - adapt.run - INFO - ── Adapt run 2026AUG12-0012-KMRX ─────────────────────────────
site KMRX · NEXRAD · mode realtime
pipeline aws_nexrad v0.1.3.post39+dirty · commit — · python 3.14.4 · macOS-26.6.1-arm64-arm-64bit-Mach-O
config /Users/bhupendra/Desktop/adapt-repo/collections/KMRX/config_run_2026AUG12-0012-KMRX.json (sha 8fe8194a) · modules: ingest, detection, projection, analysis, tracking, cell_volume_stats
────────────────────────────────────────────────────────────
2026-08-12 00:12:39,865 - adapt.run - INFO - methods:

Full error message or traceback

2026-08-12 00:14:15,208 - adapt.runtime.processor - ERROR - Scan failed | scan=d75eff37114948b6 elapsed=23.87s ValueError: cell_volume_stats: submitted DataFrame has duplicate primary keys
Traceback (most recent call last):
  File "/Users/bhupendra/projects/arm_adaptive_scaning/Adapt/src/adapt/runtime/processor.py", line 419, in _run_scan
    self._router.persist(self._post_modules, ext_result, enrich_meta)
    ~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bhupendra/projects/arm_adaptive_scaning/Adapt/src/adapt/persistence/output_router.py", line 44, in persist
    self._dispatch(module.name, spec, result, meta)
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/bhupendra/projects/arm_adaptive_scaning/Adapt/src/adapt/persistence/output_router.py", line 143, in _dispatch
    ).write(df)
      ~~~~~^^^^
  File "/Users/bhupendra/projects/arm_adaptive_scaning/Adapt/src/adapt/persistence/module_output.py", line 41, in write
    self._reject_duplicate_primary_keys(frame)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^
  File "/Users/bhupendra/projects/arm_adaptive_scaning/Adapt/src/adapt/persistence/module_output.py", line 82, in _reject_duplicate_primary_keys
    raise ValueError(f"{self._spec.table}: submitted DataFrame has duplicate primary keys")
ValueError: cell_volume_stats: submitted DataFrame has duplicate primary keys

Configuration file (redact credentials)

Environment

MacOS

Checklist

  • I checked the documentation and existing issues
  • I can reproduce this consistently
  • This occurs on the latest release

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions