Adding bulker to Vnet Orch

[ypcisco]

What I did

Implements bulk processing for VNET tunnel routes to improve performance and reduce per-route SAI overhead when handling large-scale route operations.

Key Changes

• Added context structures (TunnelRouteContext, RouteOrchContext, VNetRouteBulkContext) to track batched operations — route details, SAI op type, and a status_index into a shared status deque — with a staging area (toBulk_) to collect requests before flushing.
• Routed overlay tunnel routes through a new EntityBulker<sai_route_api_t> and reused RouteOrch's existing bulker for non-subnet routes, so both route types flush in the same pass.
• Implemented two-phase doTask: parse/queue all requests first, flush both bulkers once, then post-process using saved context.
• Moved post-processing after the flush: each route's SAI status is checked first, and state-DB/ref-count/CRM/flow-counter updates run only on success.
• Added per-entry exception handling in doTask (parse errors drop the entry; SAI failures keep it) so one bad request can't abort the batch.
• Enhanced error handling to track per-route success/failure within a batch and retry failed routes on the next iteration by leaving them in m_toSync.

Why I did it

Previously each VNET tunnel route was programmed with an individual synchronous SAI call, so large route churn (bring-up, failover, config push) was slow and blocked the orchagent. Bulking amortizes the SAI round-trips and aligns VNET route handling with the bulker pattern already used by RouteOrch.

How I verified it

Added vnet_lib.py helpers (create_vnet_routes_appdb, set_vnet_routes_appdb, delete_vnet_routes_appdb, check_vnet_route_exists) using a buffered ProducerStateTable to force real batching, plus the following VS integration tests:

• Test 38 (Bulk route scale) — programs 50 routes sharing endpoints; asserts all land in ASIC_DB, collapse into a single shared NHG, appear in STATE_DB, and are fully removed on bulk delete.
• Test 39 (Bulk mixed operations) — flushes add + update + delete in one batch; asserts each op takes effect and the resulting ASIC route count matches.
• Test 40 (Deferred dependency) — queues routes before the tunnel/VNET exist; asserts they aren't programmed until dependencies are created, then all get programmed (retry path).
• Test 41 (Bulk delete with custom monitoring) — bulk-deletes 20 custom-monitored routes; asserts every VNET_MONITOR_TABLE session and STATE_DB entry is cleaned up.
• Test 42 (Bulk NHG ref-counting) — routes across two endpoint groups, migrates 25 from A→B, deletes the rest; asserts NHG create/share/delete follows ref-counts (2 NHGs → NHG_A removed at refcount 0 → all cleaned up).
• Test 43 (SET-to-inactive) — SETs an active BFD route to new endpoints with monitors down; asserts the ASIC route is removed but the STATE_DB key is retained as inactive, and delete then removes it.

Details if related

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[Copilot]

Pull request overview

This PR introduces bulk (batched) SAI programming for VNET tunnel routes (and queues non-subnet routes via RouteOrch) to improve performance when many routes are processed, by separating request parsing/queueing from a single bulk flush and then per-entry post-processing.

Changes:

• Added bulk context structures and new tunnel-route bulker in VNetRouteOrch, plus a custom doTask() that queues all requests, flushes once, then post-processes based on per-route SAI status.
• Updated RouteBulkContext to be movable and exposed RouteOrch bulkers publicly to enable cross-orch batching.
• Added new DVS tests to validate scale, mixed operations, dependency deferral, monitoring cleanup, and NHG refcount behavior.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
orchagent/vnetorch.h	Adds new bulk context structs and member fields to support batching and post-processing.
orchagent/vnetorch.cpp	Implements bulk queue/flush/post flow and new tunnel route bulk helpers/post handlers.
orchagent/routeorch.h	Makes RouteBulkContext movable and exposes RouteOrch bulkers needed for external batching.
tests/vnet_lib.py	Adds a helper to assert VNET tunnel route presence in STATE_DB.
tests/test_vnet.py	Adds bulk-focused integration tests (scale, mixed ops, dependency deferral, monitoring cleanup, NHG refcount).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines will not run the associated pipelines, because the pull request was updated after the run command was issued. Review the pull request again and issue a new run command.

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).

[mssonicbld]

/azp run

[azure-pipelines]

Azure Pipelines:
Successfully started running 1 pipeline(s).