Skip to content

Reconciler fails to apply interfaces when dcim.device.primary_ip4 spans multiple change sets #545

Description

@Triomat

Diode version: v2.0.0

Deployment type: docker compose

Deployment type version: netboxlabs/diode Docker images tagged 2.0.0 (diode-ingester, diode-reconciler, diode-auth)

Diode SDK type: diode-sdk-python

Diode SDK version: v2.0.0 (bundled with netboxlabs/orb-agent:2.10.0)

Diode NetBox Plugin version: v1.13.0

NetBox version: v4.6.1-5.0.1 (NetBox v4.6.1, netbox-docker 5.0.1)

Steps to reproduce

  1. Deploy Diode v2.0.0 with the default docker compose environment.
  2. Run netboxlabs/orb-agent:2.10.0 with the device_discovery backend. Reproduced with two different platforms and NAPALM drivers:
    • Cisco Nexus 9K via nxos_ssh
    • Aruba AOS-CX (e.g. 6200F) via aruba_aoscx
  3. Configure the policy against a switch that has:
    • many physical interfaces, and
    • one management interface carrying the IP used as the agent's scan target.
  4. Use default processor settings:
    INGESTION_LOG_PROCESSOR_BATCH_SIZE=10
    AUTO_APPLY_PROCESSOR_BATCH_SIZE=100
    
  5. Let the agent ingest the device. It emits the device entity first, then one entity per interface.
  6. Inspect the Diode ingestion_logs table for object_type = 'dcim.interface' and state = 4.

Example interface entity emitted by the agent:

{
  "interface": {
    "name": "1/1/2",
    "type": "1000base-t",
    "device": {
      "name": "LAB-SW-01",
      "site": {"name": "Lab"},
      "device_type": {"model": "Aruba 6200F", "manufacturer": {"name": "Aruba"}},
      "primary_ip4": {"address": "203.0.113.10/24"}
    },
    "enabled": false
  }
}

The IP 203.0.113.10/24 is configured on interface vlan201, emitted as a separate entity/change set.

Expected behavior

All interface entities should be reconciled and created in NetBox regardless of whether the device record was already present when an individual interface change set was generated.

Observed behavior

Many interface change sets fail with the same error on both the Cisco N9K and the Aruba device:

apply error for entity <id>: {"dcim.device":{"primary_ip4":["The specified IP address (203.0.113.10/24) is not assigned to this device."]}}

For the Aruba test device, 13 interfaces applied successfully while 19 failed. The failures are timing-dependent: interfaces whose change sets were generated after the device already existed in NetBox succeed; those generated while the device was still being created fail.

Root cause (from local analysis)

Each entity becomes one ingestion log, and each ingestion log produces one change set. The graph contains a cycle:

Interface --device--> Device --primary_ip4--> IPAddress --assigned_object_interface--> Interface

Because the management interface and the physical interfaces are in different change sets, a physical-interface change set that creates the inline device tries to set primary_ip4 before the management-interface change set has linked that IP to the device. NetBox validation rejects the device update.

The diode-netbox-plugin transformer marks dcim.device.primary_ip4 as a circular reference and uses post_create, but that logic operates within a single change set and cannot resolve the assignment across change sets.

Workaround

Set both processor batch sizes to 1 to force strictly sequential processing:

INGESTION_LOG_PROCESSOR_BATCH_SIZE=1
AUTO_APPLY_PROCESSOR_BATCH_SIZE=1

With the device entity processed first, subsequent interface change sets see the device as a noop and skip the primary_ip4 update, allowing interfaces to apply. This reduces throughput and may still fail on a completely empty NetBox if the management IP does not yet exist.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions