Skip to content

Commit e74378f

Browse files
authored
Merge pull request #1 from NetX-lab/sync/sonic-integration
Initial commit
2 parents 89cfcb3 + f3e2797 commit e74378f

24 files changed

Lines changed: 409 additions & 173 deletions

File tree

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,3 +51,4 @@ docs/.source/
5151
docs/.env.local
5252
docs/_legacy/
5353
.playwright-mcp/
54+
CLAUDE*

CONTRIBUTING.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ Thanks for helping improve NetOpsBench. This project is an early-stage DCN troub
55
## Development Setup
66

77
```bash
8-
git clone https://github.com/netx-lab/NetOpsBench.git
8+
git clone https://github.com/NetX-lab/NetOpsBench.git
99
cd NetOpsBench
1010
python -m venv .venv
1111
source .venv/bin/activate

README.md

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
# NetOpsBench
22

33
<p align="center">
4-
<strong>The first closed-loop benchmark for agentic DCN troubleshooting.</strong>
4+
<strong>An Interactive Arena for AI Infrastructure Diagnosis</strong>
55
</p>
66

77
<p align="center">
8-
<a href="https://github.com/netx-lab/NetOpsBench/actions/workflows/test.yml"><img alt="Tests" src="https://github.com/netx-lab/NetOpsBench/actions/workflows/test.yml/badge.svg"></a>
9-
<a href="https://github.com/netx-lab/NetOpsBench/actions/workflows/docs-pages.yml"><img alt="Docs" src="https://github.com/netx-lab/NetOpsBench/actions/workflows/docs-pages.yml/badge.svg"></a>
8+
<a href="https://github.com/NetX-lab/NetOpsBench/actions/workflows/test.yml"><img alt="Tests" src="https://github.com/NetX-lab/NetOpsBench/actions/workflows/test.yml/badge.svg"></a>
9+
<a href="https://github.com/NetX-lab/NetOpsBench/actions/workflows/docs-pages.yml"><img alt="Docs" src="https://github.com/NetX-lab/NetOpsBench/actions/workflows/docs-pages.yml/badge.svg"></a>
1010
<a href="https://www.python.org/downloads/"><img alt="Python" src="https://img.shields.io/badge/python-3.11%2B-blue"></a>
1111
<a href="LICENSE"><img alt="License" src="https://img.shields.io/badge/license-MIT-green"></a>
1212
</p>
@@ -27,7 +27,7 @@
2727
- Complete benchmark documentation and methodology
2828
- Cross-model evaluation results and scoring framework
2929

30-
NetOpsBench evaluates RCA agents inside a real reproducible data-center network loop: build a SONiC-VS / Containerlab fabric, inject a controlled fault, observe Pingmesh and telemetry symptoms, call an agent, and score the diagnosis against ground truth.
30+
NetOpsBench evaluates agentic RCA inside a real reproducible data-center network loop: build a SONiC-VS / Containerlab fabric, inject a controlled fault, observe Pingmesh and telemetry symptoms, call an agent, and score the diagnosis against ground truth.
3131

3232
It is built for researchers and engineers who want to compare LLM-backed, symbolic, heuristic, or hybrid troubleshooting strategies on the same operational benchmark, not just on static logs or hand-written prompts.
3333

@@ -38,7 +38,7 @@ It is built for researchers and engineers who want to compare LLM-backed, symbol
3838
| Capability | What NetOpsBench gives you |
3939
|---|---|
4040
| Run the network, not just logs | Faults execute against live SONiC/Containerlab topologies with traffic and observability. |
41-
| Bring any RCA agent | Pass a Python object with `diagnose(context)` into the public SDK. |
41+
| Bring any agentic strategy | Pass a Python object with `diagnose(context)` into the public SDK. |
4242
| Score localization, not only detection | Measure fault type, device, interface, runtime, tool calls, and token cost. |
4343
| Scale with isolated worker pools | Run suites across independent labs and merge results into one `BenchmarkReport`. |
4444

@@ -47,7 +47,7 @@ It is built for researchers and engineers who want to compare LLM-backed, symbol
4747
NetOpsBench runtime execution requires Linux because Containerlab depends on Linux networking primitives.
4848

4949
```bash
50-
git clone https://github.com/netx-lab/NetOpsBench.git
50+
git clone https://github.com/NetX-lab/NetOpsBench.git
5151
cd NetOpsBench
5252

5353
python -m venv .venv
@@ -106,3 +106,16 @@ Contributions are welcome for benchmark scenarios, fault types, SDK ergonomics,
106106
## License
107107

108108
NetOpsBench is released under the MIT License. See [LICENSE](LICENSE).
109+
110+
## Citation
111+
112+
If you use NetOpsBench in your research, please cite:
113+
114+
```bibtex
115+
@software{netopsbench2026,
116+
author = {Yang, Yitao and Xu, Hong},
117+
title = {{NetOpsBench}: An Interactive Arena for Agentic RCA in AI infrastructure},
118+
year = {2026},
119+
url = {https://github.com/netx-lab/NetOpsBench},
120+
}
121+
```

docs/content/docs/benchmark/methodology.mdx

Lines changed: 45 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,6 +17,50 @@ Each benchmark case pairs a generated scenario with ground truth:
1717

1818
The agent returns a `DiagnosisResult` containing a verdict, location, confidence, evidence, reasoning, and optional metadata such as tool calls or token usage.
1919

20+
## Benchmark test suite
21+
22+
### Topology scales
23+
24+
All topologies use a spine-leaf fabric running SONiC-VS switches. Scale increases both the number of network devices and the density of clients, making diagnosis progressively harder as the fault signal is diluted across a larger Pingmesh matrix.
25+
26+
| Scale | Spines | Leafs | Clients | Network devices |
27+
|---|---:|---:|---:|---:|
28+
| XS | 2 | 2 | 2 | 4 |
29+
| Small | 2 | 4 | 8 | 6 |
30+
| Medium | 4 | 8 | 16 | 12 |
31+
| Large | 4 | 16 | 64 | 20 |
32+
33+
"Network devices" counts switches only. Clients are Linux containers attached to leaf switches and act as Pingmesh endpoints.
34+
35+
### Fault coverage
36+
37+
The benchmark covers twelve canonical fault types across five categories. Each type corresponds to a distinct failure mode that an operations agent may encounter in a real DCN.
38+
39+
| Category | Fault types | Difficulty |
40+
|---|---|---|
41+
| Link | `link_down`, `link_flapping` | easy – hard |
42+
| Routing | `blackhole_route`, `static_route_misconfig`, `bgp_neighbor_misconfig`, `route_policy_misconfig` | medium – hard |
43+
| Impairment | `mtu_mismatch`, `packet_loss`, `packet_corruption`, `high_latency` | medium – hard |
44+
| System | `device_down` | easy |
45+
| ACL | `acl_misconfig` | hard |
46+
47+
All twelve fault types appear in every topology scale. Each scenario instance is placed on a randomly selected device that matches the fault template's role constraint (spine, leaf, or client-facing interface).
48+
49+
### Scenario breakdown
50+
51+
Each scale combines fault scenarios with a set of healthy negative samples. Healthy cases exercise false-positive control: the network is operating normally and the correct answer is `network_healthy`.
52+
53+
| Scale | Fault cases | Healthy cases | Total |
54+
|---|---:|---:|---:|
55+
| XS | 12 | 2 | 14 |
56+
| Small | 12 | 3 | 15 |
57+
| Medium | 24 | 4 | 28 |
58+
| Large | 48 | 4 | 52 |
59+
60+
### Scenario generation
61+
62+
Scenarios are defined in `scenarios/specs/fault_campaign.yaml`, which specifies a `count_per_scale` entry for each fault template. Running `netopsbench benchmark prepare --scales <scale>` materialises that spec into individual YAML scenario files under `scenarios/generated/<scale>/`. The generator picks a qualifying device and interface at random for each instance, so every preparation pass produces a structurally identical but topologically varied set.
63+
2064
## Core scoring dimensions
2165

2266
| Dimension | Meaning |
@@ -36,7 +80,7 @@ Fault-case score is verdict-gated. If the agent misses the fault verdict, locali
3680
When an interface is applicable, NetOpsBench uses a localization-first score:
3781

3882
```text
39-
score = 0.6 * device_correct + 0.4 * interface_correct
83+
score = 0.5 * device_correct + 0.5 * interface_correct
4084
```
4185

4286
When interface ground truth is unavailable, device localization receives the full score.

docs/content/docs/benchmark/results.mdx

Lines changed: 20 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -21,23 +21,24 @@ They do not replace your own benchmark run. Provider defaults, model versions, p
2121

2222
| Dimension | Values |
2323
|---|---|
24-
| Models | Kimi, DeepSeek, OpenAI, MiniMax |
25-
| Topology scales | XS (14 scenarios), Small (15), Medium (28), Large (52) |
24+
| Models | Kimi K2.6, MiniMax M2.7, DeepSeek V4 Pro, OpenAI GPT-5.4 |
25+
| Topology scales | XS → Large (4 scales, 14 – 52 scenarios each) |
26+
| Fault types | 12 canonical types across link, routing, impairment, system, and ACL categories |
2627
| Quality metrics | Detection rate, device localization, interface localization, composite score |
2728
| Efficiency metrics | Diagnosis time, tool calls, input/output tokens |
2829

29-
For metric definitions, see [Benchmark Methodology](/docs/benchmark/methodology).
30+
For topology dimensions, fault type details, and scenario counts, see [Benchmark Methodology — Test suite](/docs/benchmark/methodology#benchmark-test-suite). For metric definitions, see [Benchmark Methodology — Scoring](/docs/benchmark/methodology#core-scoring-dimensions).
3031

3132
## Cross-scale snapshot
3233

3334
Read each cell as `XS -> Large`. Large absolute values are the strongest signal for deployment readiness.
3435

35-
| Vendor | Detection (%) | Device Loc (%) | Interface Loc (%) | Composite (%) | Avg Time (s) | Avg Tools | Input Tokens (K) |
36+
| Model | Detection (%) | Device Loc (%) | Interface Loc (%) | Composite (%) | Avg Time (s) | Avg Tools | Input Tokens (K) |
3637
|---|---:|---:|---:|---:|---:|---:|---:|
37-
| Kimi | 85.7 -> 61.5 | 83.3 -> 33.3 | 28.6 -> 14.3 | 77.1 -> 34.2 | 103.3 -> 224.3 | 22.6 -> 33.0 | 164.1 -> 517.0 |
38-
| DeepSeek | 100.0 -> 80.8 | 58.3 -> 37.5 | 85.7 -> 14.3 | 70.0 -> 30.0 | 161.5 -> 162.6 | 21.2 -> 16.7 | 153.7 -> 261.0 |
39-
| OpenAI | 100.0 -> 80.8 | 33.3 -> 45.8 | 42.9 -> 0.0 | 48.6 -> 32.3 | 97.0 -> 48.6 | 14.5 -> 12.4 | 105.1 -> 194.4 |
40-
| MiniMax | 25.0 -> 93.8 | 16.7 -> 31.2 | 14.3 -> 0.0 | 13.3 -> 23.8 | 68.1 -> 104.3 | 12.0 -> 13.9 | 87.0 -> 217.9 |
38+
| Kimi K2.6 | 85.7 -> 61.5 | 83.3 -> 33.3 | 28.6 -> 14.3 | 77.1 -> 34.2 | 103.3 -> 224.3 | 22.6 -> 33.0 | 164.1 -> 517.0 |
39+
| DeepSeek V4 Pro | 100.0 -> 80.8 | 58.3 -> 37.5 | 85.7 -> 14.3 | 70.0 -> 30.0 | 161.5 -> 162.6 | 21.2 -> 16.7 | 153.7 -> 261.0 |
40+
| OpenAI GPT-5.4 | 100.0 -> 80.8 | 33.3 -> 45.8 | 42.9 -> 0.0 | 48.6 -> 32.3 | 97.0 -> 48.6 | 14.5 -> 12.4 | 105.1 -> 194.4 |
41+
| MiniMax M2.7 | 25.0 -> 93.8 | 16.7 -> 31.2 | 14.3 -> 0.0 | 13.3 -> 23.8 | 68.1 -> 104.3 | 12.0 -> 13.9 | 87.0 -> 217.9 |
4142

4243
## Quality funnel: detection, device, interface
4344

@@ -47,53 +48,53 @@ Diagnosis quality is a funnel. An agent must detect the fault, then localize it
4748

4849
![Detection accuracy](/assets/benchmark/fig_detection.png)
4950

50-
Most models maintain high detection. OpenAI and DeepSeek hold 100% from XS through Medium and stay near 80% on Large. MiniMax climbs from 25% on XS to 94% on Large. Detection is a starting point, not a final RCA outcome.
51+
Most models maintain high detection. OpenAI GPT-5.4 and DeepSeek V4 Pro hold 100% from XS through Medium and stay near 80% on Large. MiniMax M2.7 climbs from 25% on XS to 94% on Large. Detection is a starting point, not a final RCA outcome.
5152

5253
### Device localization
5354

5455
![Device localization](/assets/benchmark/fig_device_loc.png)
5556

56-
Device localization is the first real bottleneck. Kimi reaches 91.7% on Small but drops to 33.3% on Large. DeepSeek peaks at 95.8% on Medium then falls to 37.5%. OpenAI is the strongest on Large at 45.8%, still below a production-grade RCA target.
57+
Device localization is the first real bottleneck. Kimi K2.6 reaches 91.7% on Small but drops to 33.3% on Large. DeepSeek V4 Pro peaks at 95.8% on Medium then falls to 37.5%. OpenAI GPT-5.4 is the strongest on Large at 45.8%, still below a production-grade RCA target.
5758

5859
### Interface localization
5960

6061
![Interface localization](/assets/benchmark/fig_intf_loc.png)
6162

62-
Interface precision is the weakest dimension. On Large, OpenAI and MiniMax fall to 0%; Kimi and DeepSeek reach only 14.3%. Many real fixes are interface-scoped, so this gap is the main barrier to operational automation.
63+
Interface precision is the weakest dimension. On Large, OpenAI GPT-5.4 and MiniMax M2.7 fall to 0%; Kimi K2.6 and DeepSeek V4 Pro reach only 14.3%. Many real fixes are interface-scoped, so this gap is the main barrier to operational automation.
6364

6465
### Composite score
6566

6667
![Composite score](/assets/benchmark/fig_avg_score.png)
6768

68-
The composite summarizes localization quality. All four models drop sharply at Large (Kimi 34.2, DeepSeek 30.0, OpenAI 32.3, MiniMax 23.8), confirming that current agents perform well on small topologies but lose actionable precision at scale.
69+
The composite summarizes localization quality. All four models drop sharply at Large (Kimi K2.6 34.2, DeepSeek V4 Pro 30.0, OpenAI GPT-5.4 32.3, MiniMax M2.7 23.8), confirming that current agents perform well on small topologies but lose actionable precision at scale.
6970

7071
## Cost: time, tools, tokens
7172

7273
More time, tool calls, or tokens do not reliably translate into better localization.
7374

7475
![Average diagnosis time](/assets/benchmark/fig_avg_time.png)
7576

76-
OpenAI is the fastest on Large at 48.6 s/case; Kimi is the slowest at 224.3 s. DeepSeek is roughly flat from XS to Large.
77+
OpenAI GPT-5.4 is the fastest on Large at 48.6 s/case; Kimi K2.6 is the slowest at 224.3 s. DeepSeek V4 Pro is roughly flat from XS to Large.
7778

7879
![Tool calls](/assets/benchmark/fig_tool_calls.png)
7980

80-
Kimi grows to 33 calls/case at Large; OpenAI stays at 12.4. Heavier tool use does not buy stronger interface localization.
81+
Kimi K2.6 grows to 33 calls/case at Large; OpenAI GPT-5.4 stays at 12.4. Heavier tool use does not buy stronger interface localization.
8182

8283
![Input tokens](/assets/benchmark/fig_input_tokens.png)
8384
![Output tokens](/assets/benchmark/fig_output_tokens.png)
8485

85-
Kimi spends the most tokens (517K input / 8K output per case at Large); OpenAI the least (194K / 3K). Cost should always be evaluated against device and interface localization, not against detection alone.
86+
Kimi K2.6 spends the most tokens (517K input / 8K output per case at Large); OpenAI GPT-5.4 the least (194K / 3K). Cost should always be evaluated against device and interface localization, not against detection alone.
8687

8788
## Model fit summary
8889

8990
The best choice depends on whether the priority is localization quality, fast triage, cost predictability, or scale readiness.
9091

9192
| Model | Strongest signal | Main risk | Best fit |
9293
|---|---|---|---|
93-
| Kimi | Strong Small-scale composite, early device localization | Large quality drops while time and token cost rise sharply | Deep-diagnosis runs with generous reasoning budget |
94-
| DeepSeek | Balanced quality on XS, Small, Medium | Large localization still degrades materially | Quality-oriented RCA evaluation |
95-
| OpenAI | Fast, low tool calls, strong detection | Interface localization collapses at Large | Fast triage, latency-sensitive flows |
96-
| MiniMax | Lower runtime/token footprint, improving detection | Lower absolute localization at Large | Budget-sensitive baselines and broad triage |
94+
| Kimi K2.6 | Strong Small-scale composite, early device localization | Large quality drops while time and token cost rise sharply | Deep-diagnosis runs with generous reasoning budget |
95+
| DeepSeek V4 Pro | Balanced quality on XS, Small, Medium | Large localization still degrades materially | Quality-oriented RCA evaluation |
96+
| OpenAI GPT-5.4 | Fast, low tool calls, strong detection | Interface localization collapses at Large | Fast triage, latency-sensitive flows |
97+
| MiniMax M2.7 | Lower runtime/token footprint, improving detection | Lower absolute localization at Large | Budget-sensitive baselines and broad triage |
9798

9899
## How to read these results
99100

docs/content/docs/contribute/fault-types.mdx

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ Every scenario references one fault type and provides:
3434
- `parameters` — fault-specific values (e.g. `latency_ms`, `iterations`)
3535
- `metadata` — descriptive benchmark annotations (not consumed by injection logic)
3636

37-
See [`scenarios/README.md`](https://github.com/netx-lab/NetOpsBench/blob/main/scenarios/README.md) for the full episode schema.
37+
See [`scenarios/README.md`](https://github.com/NetX-lab/NetOpsBench/blob/main/scenarios/README.md) for the full episode schema.
3838

3939
## Symmetric (link-like) faults
4040

docs/content/docs/getting-started/index.mdx

Lines changed: 20 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ NetOpsBench requires a **Linux host**. Windows and macOS are not supported becau
1414
- Python 3.11+
1515
- Docker
1616
- Containerlab
17-
- One supported LLM API key such as `OPENAI_API_KEY`
17+
- An API key for one of the supported LLM vendors (see table below)
1818

1919
Install Docker:
2020

@@ -33,6 +33,14 @@ containerlab version
3333

3434
Full installation guide: [containerlab.dev/install](https://containerlab.dev/install/).
3535

36+
Enable passwordless sudo for the commands used by the benchmark scripts:
37+
38+
```bash
39+
echo "$USER ALL=(ALL) NOPASSWD: /usr/bin/docker, /usr/bin/containerlab, /usr/bin/rm" \
40+
| sudo tee /etc/sudoers.d/netopsbench
41+
sudo chmod 440 /etc/sudoers.d/netopsbench
42+
```
43+
3644
## Install
3745

3846
```bash
@@ -45,9 +53,19 @@ pip install -e ".[agent]"
4553

4654
## Run one benchmark case
4755

56+
Supported vendors and their required environment variables:
57+
58+
| `--vendor` | Model | Environment variable |
59+
|------------|-------|----------------------|
60+
| `openai` | gpt-5.4 | `OPENAI_API_KEY` |
61+
| `minimax` | MiniMax-M2.7 | `MINIMAX_API_KEY` |
62+
| `deepseek` | deepseek-v4-pro | `DEEPSEEK_API_KEY` |
63+
| `zhipu` | glm-5.1 | `ZHIPU_API_KEY` |
64+
| `kimi` | kimi-k2.6 | `KIMI_API_KEY` |
65+
4866
```bash
4967
netopsbench benchmark prepare --scales xs
50-
export OPENAI_API_KEY=...
68+
export OPENAI_API_KEY=... # set the key for your chosen vendor
5169
PYTHONPATH=. python examples/01_run_scenario.py --vendor openai
5270
```
5371

docs/content/docs/operations/deployment.mdx

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -12,7 +12,8 @@ For the dashboard guide, see [Observability Guide](/docs/operations/observabilit
1212
NetOpsBench targets **Linux hosts only** with Docker, Containerlab, and non-interactive sudo:
1313

1414
```bash
15-
sudo bash -c 'echo "<your-username> ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/netopsbench'
15+
echo "$USER ALL=(ALL) NOPASSWD: /usr/bin/docker, /usr/bin/containerlab, /usr/bin/rm" \
16+
| sudo tee /etc/sudoers.d/netopsbench
1617
sudo chmod 440 /etc/sudoers.d/netopsbench
1718
```
1819

-2.11 MB
Loading

0 commit comments

Comments
 (0)