Skip to content

feat(edge-agentic): add BFCL v4 edge-agentic workload to edge benchma…#331

Merged
mrmhodak merged 6 commits into
mlcommons:masterfrom
Palanivelg:feat/edge-agentic-rules
Jul 21, 2026
Merged

feat(edge-agentic): add BFCL v4 edge-agentic workload to edge benchma…#331
mrmhodak merged 6 commits into
mlcommons:masterfrom
Palanivelg:feat/edge-agentic-rules

Conversation

@Palanivelg

@Palanivelg Palanivelg commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds the Edge Agentic workload (BFCL v4 function-calling accuracy) to the
MLPerf Inference edge benchmarks and required-scenarios tables.

Changes

  • Edge benchmarks table: new row — Language | Agentic Function Calling | Qwen3.6-27B | gorilla-llm/gorilla-eval-set (BFCL v4 single-turn: non_live, live, hallucination) | QSL 995 | 97% of validated Q4_K_M reference, 3% one-sided band (single-turn overall=86.23% → threshold 83.64%, non_live-normalized=87.96% → threshold 85.32%)
  • Edge required-scenarios table: new row — Language | Agentic Function Calling | Offline (accuracy-only)

Related

@github-actions

github-actions Bot commented Jun 8, 2026

Copy link
Copy Markdown

MLCommons CLA bot All contributors have signed the MLCommons CLA ✍️ ✅

@Palanivelg
Palanivelg marked this pull request as ready for review July 14, 2026 18:17
@Palanivelg
Palanivelg requested a review from a team as a code owner July 14, 2026 18:17
@mrmhodak
mrmhodak merged commit 22092d8 into mlcommons:master Jul 21, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants