Skip to content

Commit b22ecf8

Browse files
Adarsh KumarappanAdarsh Kumarappan
authored andcommitted
add arxiv to repo
1 parent 52d7009 commit b22ecf8

1 file changed

Lines changed: 19 additions & 0 deletions

File tree

README.md

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
# DevBench
22

3+
[![arXiv](https://img.shields.io/badge/arXiv-2601.11895-b31b1b.svg)](https://arxiv.org/abs/2601.11895)
4+
35
A comprehensive framework for generating, evaluating, and comparing code completions across multiple programming languages and AI models on our synthetic, telemetry-driven benchmark.
46

57
## Overview
@@ -19,6 +21,23 @@ The benchmark supports execution-based evaluation across multiple programming la
1921
- **C++** - Compiled with g++/clang++ with automatic library detection and linking
2022
- **C#** - Built and executed with .NET SDK, automatic NuGet package detection and installation
2123

24+
## Citation
25+
26+
If you use DevBench in your research, please cite our paper:
27+
```bibtex
28+
@misc{devbench2026,
29+
author = {Golnari, Pareesa Ameneh and Kumarappan, Adarsh and Wen, Wen and Liu, Xiaoyu and Ryan, Gabriel and Sun, Yuting and Fu, Shengyu and Nallipogu, Elsie},
30+
title = {{DevBench}: A Realistic, Developer-Informed Benchmark for Code Generation Models},
31+
year = {2026},
32+
eprint = {2601.11895},
33+
archivePrefix = {arXiv},
34+
primaryClass = {cs.LG},
35+
url = {https://arxiv.org/abs/2601.11895}
36+
}
37+
```
38+
39+
**Paper:** [https://arxiv.org/abs/2601.11895](https://arxiv.org/abs/2601.11895)
40+
2241
## Repository Structure
2342

2443
- `benchmark/` - Generated benchmark test cases organized by language and category

0 commit comments

Comments
 (0)