-
Notifications
You must be signed in to change notification settings - Fork 0
Expand file tree
/
Copy pathhy4-benchmarks.html
More file actions
107 lines (106 loc) · 28.5 KB
/
Copy pathhy4-benchmarks.html
File metadata and controls
107 lines (106 loc) · 28.5 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
<!DOCTYPE html><html lang="en"><head><meta charset="UTF-8"><meta name="viewport" content="width=device-width, initial-scale=1.0"><title>Hy4 Benchmarks — Scores & Architecture | Tencent Hy4 Preview</title><meta name="description" content="Hy4 Preview benchmark scores across coding, search, agent, STEM, and reasoning tasks, plus a 78-layer architecture comparison with DeepSeek V4."><link rel="canonical" href="https://hy4ai.com/hy4-benchmarks.html"><link rel="icon" href="/favicon.svg" type="image/svg+xml"><meta property="og:title" content="Hy4 Benchmarks — Scores & Architecture"><meta property="og:description" content="Benchmark results across coding, search, agent, STEM, and reasoning tasks, with an architecture-level comparison."><meta property="og:url" content="https://hy4ai.com/hy4-benchmarks.html"><meta property="og:type" content="article"><meta property="og:image" content="https://hy4ai.com/og-image.png"><meta name="twitter:card" content="summary_large_image"><script type="application/ld+json">{"@context":"https://schema.org","@type":"FAQPage","mainEntity":[{"@type":"Question","name":"What are the Hy4 benchmark scores?" ,"acceptedAnswer":{"@type":"Answer","text":"The supplied dataset reports Hy4 Preview scores across Agentic Coding, Agentic Search, Working Agent, STEM Agent, and Reasoning tasks. Values are shown in the tables, including 82.9 on SWE-bench Multilingual, 85.4 on Terminal-Bench 2.1, 83.7 on MCP-Atlas, 71.3 on BioMysteryBench, and 92.3 on GPQA Diamond."}},{"@type":"Question","name":"How does Hy4 compare to other models on benchmarks?" ,"acceptedAnswer":{"@type":"Answer","text":"Hy4 Preview leads the supplied model set on several listed tasks, while other models lead on others. Because harnesses and settings differ, compare models within the same benchmark rather than treating the dataset as one absolute ranking."}},{"@type":"Question","name":"How does Hy4's architecture compare to DeepSeek V4?" ,"acceptedAnswer":{"@type":"Answer","text":"Hy4 Preview uses 78 layers with 256 routed experts per layer and top-8 routing (770B total / 49B active). DeepSeek-V4-Pro uses 61 layers with 384 routed experts per layer and top-6 routing (1.6T total / 49B active). Both activate 49B parameters per token."}},{"@type":"Question","name":"When will Hy4 be officially released?" ,"acceptedAnswer":{"@type":"Answer","text":"Hy4 is currently in grey-testing and Tencent has not announced a specific release date."}},{"@type":"Question","name":"Which benchmarks matter most for Hy4?" ,"acceptedAnswer":{"@type":"Answer","text":"The most useful rows depend on your workload: SWE-bench and Terminal-Bench for coding, MCP-Atlas and Toolathlon for tool use, BioMysteryBench and HLE for STEM and research, and GPQA Diamond or MathArena for reasoning."}}]}</script><script type="application/ld+json">{"@context":"https://schema.org","@type":"Article","headline":"Hy4 Benchmarks — Scores & Architecture","description":"Hy4 Preview benchmark scores across coding, search, agent, STEM, and reasoning tasks, plus an architecture comparison.","datePublished":"2026-08-28","dateModified":"2026-08-28","author":{"@type":"Organization","name":"hy4ai.com"},"publisher":{"@type":"Organization","name":"hy4ai.com"},"mainEntityOfPage":{"@type":"WebPage","@id":"https://hy4ai.com/hy4-benchmarks.html"}}</script><script type="application/ld+json">{"@context":"https://schema.org","@type":"WebPage","name":"Hy4 Benchmarks — Scores & Architecture","url":"https://hy4ai.com/hy4-benchmarks.html","breadcrumb":{"@type":"BreadcrumbList","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https://hy4ai.com/"},{"@type":"ListItem","position":2,"name":"Hy4 Benchmarks","item":"https://hy4ai.com/hy4-benchmarks.html"}]}}</script><style>
:root {
--ink: #14213d;
--muted: #65728c;
--blue: #1769ff;
--blue-deep: #0b4fd7;
--pale: #f3f7ff;
--line: #e4ebf7;
--card: rgba(255, 255, 255, .88);
}
* { box-sizing: border-box; }
html { scroll-behavior: smooth; }
body {
font-family: "Avenir Next", "Segoe UI", sans-serif;
line-height: 1.7;
color: var(--ink);
margin: 0;
background: #fff;
font-size: 15px;
}
body::before {
content: "";
position: fixed;
inset: 0;
pointer-events: none;
opacity: .32;
background-image: radial-gradient(#cddafa .7px, transparent .7px);
background-size: 22px 22px;
mask-image: linear-gradient(to bottom, #000, transparent 40%);
z-index: -1;
}
.wrap { max-width: 1180px; margin: 0 auto; padding: 0 34px; }
header {
border-bottom: 1px solid rgba(220, 229, 245, .75);
padding: 15px 0;
background: rgba(255,255,255,.84);
backdrop-filter: blur(14px);
position: sticky;
top: 0;
z-index: 10;
}
nav { display: flex; align-items: center; justify-content: space-between; flex-wrap: wrap; gap: 14px; }
.logo { font-weight: 800; letter-spacing: -.04em; font-size: 20px; color: #14213d; text-decoration: none; }
.logo::before { content: ""; display: inline-block; width: 6px; height: 20px; background: var(--blue); border-radius: 5px; vertical-align: -3px; margin-right: 9px; }
nav .links { display: flex; align-items: center; gap: 22px; }
nav .links a { color: #59667d; text-decoration: none; margin-left: 0; font-size: 13px; font-weight: 600; }
nav .links a:hover { color: var(--blue); }
nav .cta { background: #14213d; color: #fff !important; padding: 9px 15px; border-radius: 7px; text-decoration: none; font-size: 13px; box-shadow: 0 5px 14px rgba(20,33,61,.14); }
.hero { position: relative; overflow: hidden; padding: 82px 0 72px; background: linear-gradient(112deg, #f7faff 0%, #edf4ff 53%, #fff 100%); }
.hero::after { content: "Hy4"; position: absolute; right: 9%; top: 38px; font-size: clamp(130px, 23vw, 280px); line-height: 1; font-weight: 900; letter-spacing: -.12em; color: rgba(23,105,255,.055); transform: rotate(-8deg); pointer-events: none; }
.hero .wrap { position: relative; z-index: 1; }
h1 { max-width: 680px; font-size: clamp(36px, 5vw, 62px); line-height: 1.08; letter-spacing: -.055em; margin: 0 0 22px; }
h1::first-line { color: var(--blue); }
.sub { font-size: 17px; color: #526079; max-width: 660px; margin: 0; }
.sub strong { color: var(--ink); }
.stats { display: flex; gap: 0; flex-wrap: wrap; margin: 38px 0 30px; max-width: 730px; }
.stat { min-width: 126px; padding: 0 24px; border-left: 1px solid #cad8f0; }
.stat:first-child { padding-left: 0; border-left: 0; }
.stat b { display: block; font-size: 27px; line-height: 1.1; letter-spacing: -.05em; }
.stat span { font-size: 11px; color: var(--muted); display: block; margin-top: 7px; }
.btn { display: inline-block; background: linear-gradient(135deg, #277bff, #0955df); color: #fff; padding: 13px 22px; border-radius: 8px; text-decoration: none; font-weight: 700; font-size: 14px; box-shadow: 0 10px 22px rgba(23,105,255,.24); transition: transform .2s, box-shadow .2s; }
.btn:hover { transform: translateY(-2px); box-shadow: 0 14px 28px rgba(23,105,255,.3); }
code { background: #eaf1ff; color: #155bd5; padding: 3px 7px; border-radius: 4px; font-size: 12px; }
a { color: var(--blue-deep); }
section { padding: 66px 0; border-top: 1px solid var(--line); }
section:nth-of-type(even) { background: rgba(247,250,255,.72); }
h2 { font-size: 25px; line-height: 1.2; letter-spacing: -.035em; margin: 0 0 18px; display: flex; align-items: center; gap: 12px; }
h2::before { content: ""; width: 4px; height: 25px; border-radius: 4px; background: var(--blue); flex: 0 0 auto; }
p { color: #5d6b83; }
section p, section ul { max-width: 830px; }
ul { color: #5d6b83; padding-left: 21px; }
li { padding: 3px 0; }
table { width: 100%; border-collapse: separate; border-spacing: 0; font-size: 13px; margin-top: 24px; overflow: hidden; border: 1px solid var(--line); border-radius: 10px; background: var(--card); box-shadow: 0 12px 35px rgba(37,78,145,.06); }
th, td { border: 0; border-bottom: 1px solid var(--line); padding: 13px 16px; text-align: left; }
tr:last-child td { border-bottom: 0; }
th { background: #eef4ff; color: #33415e; font-size: 12px; text-transform: uppercase; letter-spacing: .04em; }
td { color: #59667d; }
td:first-child { color: var(--ink); font-weight: 600; }
.faq-item { margin-bottom: 14px; padding: 20px 24px; border: 1px solid var(--line); border-radius: 10px; background: rgba(255,255,255,.86); box-shadow: 0 8px 24px rgba(37,78,145,.045); }
.faq-item h3 { font-size: 16px; margin: 0 0 6px; color: var(--ink); }
.faq-item p { margin: 0; font-size: 14px; }
footer { border-top: 1px solid var(--line); padding: 30px 0; margin-top: 0; font-size: 12px; color: #8994a8; background: #f6f9ff; }
footer p { color: #8994a8; }
.code-card { margin: 26px 0; border: 1px solid #d8e4fa; border-radius: 11px; overflow: hidden; background: #122344; box-shadow: 0 14px 35px rgba(16, 48, 102, .15); }
.code-card-top { display: flex; align-items: center; justify-content: space-between; gap: 14px; padding: 11px 15px; color: #c9d9f7; background: #1b315c; font-size: 12px; font-weight: 700; }
.copy-btn { border: 1px solid #6e9af1; border-radius: 5px; padding: 6px 10px; color: #fff; background: transparent; font: inherit; cursor: pointer; }
.copy-btn:hover, .copy-btn:focus-visible { background: #2d66c9; outline: 2px solid #a9c7ff; outline-offset: 2px; }
.code-card pre { margin: 0; padding: 20px; overflow-x: auto; }
.code-card pre code { padding: 0; color: #e4edff; background: transparent; font: 12px/1.8 Consolas, "SFMono-Regular", monospace; white-space: pre; }
.copy-status { display: block; min-height: 21px; padding: 0 15px 10px; color: #9fc2ff; font-size: 11px; }
@media (max-width: 700px) { .code-card pre { padding: 16px; } }
@media (max-width: 700px) {
.wrap { padding: 0 20px; }
header { position: static; }
nav .links { width: 100%; justify-content: space-between; gap: 9px; }
nav .links a { font-size: 11px; }
nav .cta { padding: 7px 9px !important; }
.hero { padding: 58px 0 52px; }
.stats { gap: 21px 0; }
.stat { min-width: 33%; padding: 0 13px; }
.stat:nth-child(4) { padding-left: 0; border-left: 0; }
section { padding: 46px 0; }
table { display: block; overflow-x: auto; white-space: nowrap; }
}
table { font-size: 12px; } .section-intro { margin: 0 0 20px; font-size: 13px; } .hy4-col { background: #edf5ff !important; color: #084fc4 !important; font-weight: 700; } .data-section table { overflow-x: auto; } .architecture table td:first-child { width: 30%; }
</style></head><body><header><div class="wrap"><nav><a class="logo" href="/">Hy4 Preview</a><div class="links"><a href="/hy4-api.html">API</a><a href="/hy4-benchmarks.html">Benchmarks</a><a href="/hy4-vs-deepseek.html">Compare</a><a href="/about.html">About</a><a class="cta" href="https://tokenra.io/sign-up" target="_blank" rel="noopener">Get API Access →</a></div></nav></div></header><div class="hero"><div class="wrap"><h1>Hy4 Benchmark — Scores & Architecture</h1><p class="sub">Benchmark results for <strong>Tencent Hy4 Preview</strong> across coding, search, working-agent, STEM, and reasoning tasks, plus an architecture-level comparison. Last updated: August 28, 2026.</p></div></div><main><section><div class="wrap"><h2>How to Read This Dataset</h2><p>The tables below are transcribed from <code>benchmark-data.md</code>, a dataset compiled from public reports, provider disclosures, and clearly labeled third-party or internal evaluations. They contain 46 benchmark rows across five task families and eight model columns. The Hy4 Preview column is highlighted in blue. An asterisk comes from the source party's own testing, a slash preserves multiple reported values, and a dash means no value was shown.</p><div class="note"><strong>Important:</strong> These results are provided for reference and are not universally verified official scores. Results use different harnesses, reasoning settings, judges, resource limits, and run counts. Compare models within the same benchmark and read the evaluation notes before drawing conclusions across task families.</div></div></section><section class="data-section"><div class="wrap"><h2>Agentic Coding</h2><p class="section-intro">Scores transcribed from <code>benchmark-data.md</code>; source markers are preserved.</p><table><thead><tr><th>Benchmark</th><th>Hy3</th><th class="hy4-col">Hy4 preview</th><th>DeepSeek V4 Pro 0813</th><th>Qwen 3.8 Max</th><th>GLM 5.3</th><th>Kimi K3</th><th>GPT 5.6 Sol</th><th>Claude Opus 5</th></tr></thead><tbody><tr><td>SWE-bench Multilingual</td><td>75.8</td><td class="hy4-col">82.9</td><td>77.3*</td><td>82.6*</td><td>81.3*</td><td>80.8*</td><td>74.1*</td><td>89.5/85.8*</td></tr><tr><td>SWE-bench Pro</td><td>57.9</td><td class="hy4-col">65.7</td><td>60.3*</td><td>67.7/61.6*</td><td>64.6*</td><td>63.3*</td><td>64.6/60.5*</td><td>79.2/79.9*</td></tr><tr><td>DeepSWE</td><td>28.0</td><td class="hy4-col">64.3</td><td>62.7/58.8*</td><td>56.6/55.6*</td><td>66.9/68.1*</td><td>67.5/74.0*</td><td>72.7/68.9*</td><td>68.8/74.7*</td></tr><tr><td>SWE Atlas - Codebase Q&A</td><td>30.8</td><td class="hy4-col">64.0</td><td>53.4*</td><td>55.4*</td><td>55.8*</td><td>35.2*</td><td>58.1*</td><td>57.5*</td></tr><tr><td>SWE Atlas - Test Writing</td><td>35.9</td><td class="hy4-col">57.8</td><td>45.6*</td><td>52.8*</td><td>49.6*</td><td>35.6*</td><td>49.6*</td><td>68.7*</td></tr><tr><td>SWE Atlas - Refactoring</td><td>32.9</td><td class="hy4-col">53.3</td><td>48.6*</td><td>51.0*</td><td>51.9*</td><td>37.4*</td><td>52.4*</td><td>60.0*</td></tr><tr><td>SWE-Marathon</td><td>5.0</td><td class="hy4-col">31.9</td><td>19.0*</td><td>31.0*</td><td>42.5/35.6*</td><td>42/44.4*</td><td>42.5/35.9*</td><td>50.0/48.0*</td></tr><tr><td>Terminal-Bench 2.1</td><td>70.8</td><td class="hy4-col">85.4</td><td>87.9/80.3*</td><td>86.6/85.8*</td><td>88.2/88.3*</td><td>88.3/85.7*</td><td>88.8/88.3*</td><td>86.7/85.4*</td></tr><tr><td>NL2Repo-Bench</td><td>45.6</td><td class="hy4-col">58.9</td><td>61.5/54.4*</td><td>55.9/58.0*</td><td>58.0/56.1*</td><td>58.0/58.3*</td><td>56.8*</td><td>75.3*</td></tr><tr><td>CyberGym</td><td>51.8</td><td class="hy4-col">78.4</td><td>83.3/80.3*</td><td>78.5/78.5*</td><td>84.5/83.0*</td><td>80.0</td><td>83.6</td><td>-</td></tr><tr><td>ProgramBench</td><td>3.0</td><td class="hy4-col">17.5</td><td>15.5*</td><td>17.5*</td><td>18.0*</td><td>24.5*</td><td>25.0*</td><td>39.5*</td></tr><tr><td>PostTrainBench V1.1</td><td>14.5</td><td class="hy4-col">35.6</td><td>24.5*</td><td>-</td><td>33.2*</td><td>32.0*</td><td>36.2</td><td>35.0</td></tr><tr><td>Harbor-Index</td><td>15.6</td><td class="hy4-col">39.6</td><td>36.9*</td><td>38.8*</td><td>42.5*</td><td>-</td><td>46.3*</td><td>56.9*</td></tr><tr><td>Hy-Backend 2.0 (Internal)</td><td>26.2</td><td class="hy4-col">35.2</td><td>33.8*</td><td>34.9*</td><td>41.9*</td><td>37.9*</td><td>49.6*</td><td>40.3*</td></tr><tr><td>Hy-SWE Max Verified (Internal)</td><td>49.0</td><td class="hy4-col">64.2</td><td>65.7*</td><td>65.2*</td><td>67.2*</td><td>65.6*</td><td>69.8*</td><td>70.1*</td></tr><tr><td>Hy-CompanyBench V2 (Internal)</td><td>29.8</td><td class="hy4-col">62.4</td><td>64.4*</td><td>63.3*</td><td>64.5*</td><td>63.3*</td><td>70.6*</td><td>72.7*</td></tr></tbody></table></div></section><section class="data-section"><div class="wrap"><h2>Agentic Search</h2><p class="section-intro">Scores transcribed from <code>benchmark-data.md</code>; source markers are preserved.</p><table><thead><tr><th>Benchmark</th><th>Hy3</th><th class="hy4-col">Hy4 preview</th><th>DeepSeek V4 Pro 0813</th><th>Qwen 3.8 Max</th><th>GLM 5.3</th><th>Kimi K3</th><th>GPT 5.6 Sol</th><th>Claude Opus 5</th></tr></thead><tbody><tr><td>WideSearch</td><td>81.9</td><td class="hy4-col">83.9</td><td>81.8*</td><td>81.9/81.1*</td><td>83.2*</td><td>81.0*</td><td>86.3*</td><td>84.0*</td></tr><tr><td>OneMillionBench (with tools)</td><td>51.5</td><td class="hy4-col">65.4</td><td>62.0*</td><td>63.1*</td><td>64.5*</td><td>63.5*</td><td>67.1*</td><td>68.1*</td></tr><tr><td>DRACO</td><td>65.2</td><td class="hy4-col">77.2</td><td>77.3*</td><td>76.4*</td><td>78.1*</td><td>77.5*</td><td>77.7*</td><td>88.6/79.2*</td></tr><tr><td>Hy-Live-Search (Internal)</td><td>38.9</td><td class="hy4-col">49.2</td><td>46.9*</td><td>47.5*</td><td>49.2*</td><td>45.8*</td><td>63.4*</td><td>56.1*</td></tr><tr><td>Hy-BrowseComp-Pro2 (Internal)</td><td>55.0</td><td class="hy4-col">56.1</td><td>46.5*</td><td>46.7*</td><td>48.4*</td><td>58.1*</td><td>56.4*</td><td>61.3*</td></tr></tbody></table></div></section><section class="data-section"><div class="wrap"><h2>Working Agent</h2><p class="section-intro">Scores transcribed from <code>benchmark-data.md</code>; source markers are preserved.</p><table><thead><tr><th>Benchmark</th><th>Hy3</th><th class="hy4-col">Hy4 preview</th><th>DeepSeek V4 Pro 0813</th><th>Qwen 3.8 Max</th><th>GLM 5.3</th><th>Kimi K3</th><th>GPT 5.6 Sol</th><th>Claude Opus 5</th></tr></thead><tbody><tr><td>OfficeQA Pro</td><td>54.1</td><td class="hy4-col">66.2</td><td>65.4*</td><td>65.4*</td><td>66.2*</td><td>65.4*</td><td>65.4*</td><td>66.9/66.9*</td></tr><tr><td>MCP-Atlas (public)</td><td>75.0</td><td class="hy4-col">83.7</td><td>82.5*</td><td>81.9*</td><td>81.9*</td><td>84.2/82.8*</td><td>82.5*</td><td>85.7*</td></tr><tr><td>Toolathlon-Verified</td><td>56.2</td><td class="hy4-col">74.1</td><td>74.1/70.1*</td><td>72.5/69.1*</td><td>73.0/73.8*</td><td>76.5/74.7*</td><td>73.2*</td><td>76.5*</td></tr><tr><td>APEX-Agents (pass@1)</td><td>24.4</td><td class="hy4-col">37.1</td><td>32.4*</td><td>34.0*</td><td>38.1*</td><td>41.0/37.2*</td><td>39.9/37.9*</td><td>41.8*</td></tr><tr><td>SkillsBench (79, text-only)</td><td>55.3</td><td class="hy4-col">62.9</td><td>65.0*</td><td>66.7*</td><td>63.3*</td><td>51.9*</td><td>62.5*</td><td>63.7*</td></tr><tr><td>JobBench</td><td>34.6</td><td class="hy4-col">61.7</td><td>54.1*</td><td>53.0/52.2*</td><td>58.2*</td><td>54.3/54.7*</td><td>45.4/46.8*</td><td>68.0*</td></tr><tr><td>WorkspaceBench</td><td>58.2</td><td class="hy4-col">60.2</td><td>65.4*</td><td>67.7/66.9*</td><td>68.2*</td><td>65.0*</td><td>65.3*</td><td>75.0*</td></tr><tr><td>Agents' Last Exam (ALE-CLI)</td><td>17.1</td><td class="hy4-col">22.8</td><td>21.9*</td><td>25.4*</td><td>23.8*</td><td>23.2*</td><td>27.6*</td><td>25.1*</td></tr><tr><td>GDPval-AA V2 (Elo, official)</td><td>1213</td><td class="hy4-col">1678</td><td>1580</td><td>1717</td><td>1763</td><td>1675</td><td>1711</td><td>1831</td></tr><tr><td>AutomationBench (v1.0.6)</td><td>16.1</td><td class="hy4-col">32.1</td><td>30.4*</td><td>39.8/41.1*</td><td>48.2/49.4*</td><td>46.7/45.5*</td><td>45.8/39.9*</td><td>48.7*</td></tr><tr><td>BankerToolBench</td><td>68.8</td><td class="hy4-col">78.6</td><td>73.1*</td><td>74.7*</td><td>77.8*</td><td>73.5*</td><td>79.0*</td><td>81.9*</td></tr><tr><td>E-Bench (Internal)</td><td>48.5</td><td class="hy4-col">77.1</td><td>61.3*</td><td>66.8*</td><td>71.4*</td><td>73.8*</td><td>80.6*</td><td>77.8*</td></tr><tr><td>E-Bench-Code (Internal)</td><td>64.4</td><td class="hy4-col">79.0</td><td>64.3*</td><td>67.1*</td><td>66.5*</td><td>77.6*</td><td>83.3*</td><td>82.7*</td></tr><tr><td>Hy-FinAgentBench (Internal)</td><td>69.5</td><td class="hy4-col">79.7</td><td>78.5*</td><td>77.2*</td><td>80.4*</td><td>78.5*</td><td>83.0*</td><td>82.0*</td></tr><tr><td>Hy-FinmodelBench v2 (Internal)</td><td>28.6</td><td class="hy4-col">57.0</td><td>51.3*</td><td>52.5*</td><td>57.8*</td><td>52.4*</td><td>65.3*</td><td>66.0*</td></tr></tbody></table></div></section><section class="data-section"><div class="wrap"><h2>STEM Agent</h2><p class="section-intro">Scores transcribed from <code>benchmark-data.md</code>; source markers are preserved.</p><table><thead><tr><th>Benchmark</th><th>Hy3</th><th class="hy4-col">Hy4 preview</th><th>DeepSeek V4 Pro 0813</th><th>Qwen 3.8 Max</th><th>GLM 5.3</th><th>Kimi K3</th><th>GPT 5.6 Sol</th><th>Claude Opus 5</th></tr></thead><tbody><tr><td>BioMysteryBench</td><td>54.9</td><td class="hy4-col">71.3</td><td>61.6*</td><td>58.9*</td><td>69.0*</td><td>61.3*</td><td>73.1*</td><td>72.1*</td></tr><tr><td>HLE (with tools, text-only)</td><td>51.9</td><td class="hy4-col">55.4</td><td>60.0/55.8*</td><td>56.2/54.1*</td><td>62.5/54.3*</td><td>57.0*</td><td>60.2*</td><td>60.9*</td></tr></tbody></table></div></section><section class="data-section"><div class="wrap"><h2>Reasoning</h2><p class="section-intro">Scores transcribed from <code>benchmark-data.md</code>; source markers are preserved.</p><table><thead><tr><th>Benchmark</th><th>Hy3</th><th class="hy4-col">Hy4 preview</th><th>DeepSeek V4 Pro 0813</th><th>Qwen 3.8 Max</th><th>GLM 5.3</th><th>Kimi K3</th><th>GPT 5.6 Sol</th><th>Claude Opus 5</th></tr></thead><tbody><tr><td>CritPt (official)</td><td>4.9</td><td class="hy4-col">16.9</td><td>18.0</td><td>20.0</td><td>19.1</td><td>23.4</td><td>32.3</td><td>29.1</td></tr><tr><td>GPQA Diamond</td><td>90.9</td><td class="hy4-col">92.3</td><td>92.8/91.7*</td><td>92.6/92.2*</td><td>91.7/91.4*</td><td>93.5/92.8*</td><td>94.1/94.7*</td><td>93.7/93.3*</td></tr><tr><td>HLE (no tools, text-only)</td><td>34.4</td><td class="hy4-col">43.4</td><td>42.7/40.5*</td><td>43.6/41.5*</td><td>42.3</td><td>46.9/46.6*</td><td>49.5/49.6*</td><td>54.9/53.2*</td></tr><tr><td>SUPERChem</td><td>52.6</td><td class="hy4-col">66.4</td><td>62.0*</td><td>61.9*</td><td>58.5*</td><td>66.9*</td><td>73.6*</td><td>76.7*</td></tr><tr><td>ArXivMath</td><td>51.7</td><td class="hy4-col">66.6</td><td>62.1*</td><td>67.1*</td><td>-</td><td>60.8*</td><td>79.5*</td><td>71.5*</td></tr><tr><td>HorizonMath (pass@4)</td><td>3.5</td><td class="hy4-col">8.8</td><td>4.42*</td><td>5.31*</td><td>-</td><td>7.08*</td><td>10.62*</td><td>5.3*</td></tr><tr><td>MathArena Apex 2025</td><td>38.7</td><td class="hy4-col">74.2</td><td>66.3*</td><td>72.8*</td><td>-</td><td>68.4*</td><td>90.0*</td><td>91.4*</td></tr><tr><td>BrokenArXiv</td><td>26.7</td><td class="hy4-col">54.6</td><td>43.1*</td><td>42.7*</td><td>-</td><td>56.3*</td><td>64.4*</td><td>77.7*</td></tr></tbody></table></div></section><section class="architecture"><div class="wrap"><h2>Architecture-Level Comparison</h2><p>Both models are described as activating 49B parameters per token, but their published designs use different depths and routing choices.</p><table><thead><tr><th>Architecture detail</th><th>Hy4 Preview</th><th>DeepSeek-V4-Pro</th></tr></thead><tbody><tr><td>Total / Active parameters</td><td>770B / 49B</td><td>1.6T / 49B</td></tr><tr><td>Activation ratio</td><td>≈6.4%</td><td>≈3.1%</td></tr><tr><td>Backbone layers</td><td>78</td><td>61</td></tr><tr><td>Layer structure</td><td>1 dense FFN + 77 MoE</td><td>0–1 HCA + 2–60 alternating CSA/HCA</td></tr><tr><td>Routed experts per MoE layer</td><td>256</td><td>384</td></tr><tr><td>Shared experts per layer</td><td>1</td><td>1</td></tr><tr><td>Experts activated per token</td><td>top-8 + shared</td><td>top-6 + shared</td></tr><tr><td>Speculative decoding</td><td>1 native MTP layer (10B / 0.7B active)</td><td>1 MTP layer; DSpark module (GA only)</td></tr><tr><td>Context window</td><td>1M</td><td>1M</td></tr><tr><td>Max output</td><td>64K</td><td>384K</td></tr><tr><td>Open weights</td><td>No (hosted API only)</td><td>Yes (MIT)</td></tr></tbody></table><div class="note"><strong>What this means:</strong> Hy4 spreads active parameters over fewer, larger experts, while DeepSeek V4 uses more routed experts and finer routing. Both designs should be evaluated against task-level results.</div></div></section><section><div class="wrap"><h2>What Each Benchmark Family Measures</h2><table><thead><tr><th>Family</th><th>Typical focus</th><th>Examples in this dataset</th></tr></thead><tbody><tr><td>Agentic Coding</td><td>Issue resolution, code generation, testing, terminal work</td><td>SWE-bench, DeepSWE, Terminal-Bench, CyberGym</td></tr><tr><td>Agentic Search</td><td>Research, retrieval, browsing, and tool-assisted discovery</td><td>WideSearch, DRACO, Hy-Live-Search</td></tr><tr><td>Working Agent</td><td>Office tasks, tools, automation, and applied workflows</td><td>MCP-Atlas, Toolathlon, GDPval-AA, E-Bench</td></tr><tr><td>STEM Agent</td><td>Scientific investigation and tool-supported knowledge work</td><td>BioMysteryBench, HLE with tools</td></tr><tr><td>Reasoning</td><td>Expert knowledge, mathematics, and abstract problem solving</td><td>GPQA, SUPERChem, ArXivMath, MathArena</td></tr></tbody></table></div></section><section><div class="wrap"><h2>Evaluation Notes</h2><ul><li>Each model was evaluated and reported at the highest available reasoning setting; results marked with `*` are from the source party's own testing.</li><li>Hy3 benchmark scores may differ from previously reported results because of updates to the evaluation harness, judge model, and anti-hacking mechanisms.</li><li>SWE-bench Multilingual and SWE-bench Pro use different evaluation methods.</li><li>For DeepSWE, the mini-swe-agent was used; each task ran in a sandbox with 8 CPU cores and 16 GB memory.</li><li>SWE-Marathon used the official evaluation setup, with tasks rerun until a valid score was obtained.</li><li>Terminal-Bench 2.1 used the Claude Code harness, with up to 500 turns and a 12-hour timeout per trial; resources were capped at 16 CPU cores and 32 GB RAM.</li><li>MCP-Atlas followed the April 2026 methodology, using a 500-task public set, a 100 tool-call budget per task, and Gemini 3.1 Pro Preview as the judge model.</li><li>SkillsBench results are averaged across three runs.</li><li>SWE-Atlas used the Claude Code harness with a 200-turn budget per task.</li><li>NL2Repo-Bench used a 100-turn budget with anti-hacking constraints and tool-call monitoring.</li><li>ProgramBench used a 2,000-turn budget; the DeepSeek-V4-Pro result may have been affected by repeated loops.</li><li>Hy-SWE Max Verified (Internal) used an internally built 300-task benchmark with WebSearch and WebFetch disabled.</li><li>Agents' Last Exam used 105 ALE-CLI tasks and the official evaluation protocol.</li><li>Apex-Agents used a 250-step trial with resources capped at 4 CPU cores and 16 GB RAM.</li><li>BioMysteryBench used an agent harness with a two-hour and 250-turn budget per task.</li><li>GPQA used each model's highest reasoning setting.</li><li>All results should be interpreted together with each benchmark's definition, evaluation environment, and number of runs; they should not be treated as an absolute ranking across benchmarks.</li></ul></div></section><section><div class="wrap"><h2>Frequently Asked Questions</h2><div class="faq-item"><h3>What are the Hy4 benchmark scores?</h3><p>The supplied dataset reports Hy4 Preview scores across Agentic Coding, Agentic Search, Working Agent, STEM Agent, and Reasoning tasks. Values are shown in the tables, including 82.9 on SWE-bench Multilingual, 85.4 on Terminal-Bench 2.1, 83.7 on MCP-Atlas, 71.3 on BioMysteryBench, and 92.3 on GPQA Diamond.</p></div><div class="faq-item"><h3>How does Hy4 compare to other models on benchmarks?</h3><p>Hy4 Preview leads the supplied model set on several listed tasks, while other models lead on others. Because harnesses and settings differ, compare models within the same benchmark rather than treating the dataset as one absolute ranking.</p></div><div class="faq-item"><h3>How does Hy4's architecture compare to DeepSeek V4?</h3><p>Hy4 Preview uses 78 layers with 256 routed experts per layer and top-8 routing (770B total / 49B active). DeepSeek-V4-Pro uses 61 layers with 384 routed experts per layer and top-6 routing (1.6T total / 49B active). Both activate 49B parameters per token.</p></div><div class="faq-item"><h3>When will Hy4 be officially released?</h3><p>Hy4 is currently in grey-testing and Tencent has not announced a specific release date.</p></div><div class="faq-item"><h3>Which benchmarks matter most for Hy4?</h3><p>The most useful rows depend on your workload: SWE-bench and Terminal-Bench for coding, MCP-Atlas and Toolathlon for tool use, BioMysteryBench and HLE for STEM and research, and GPQA Diamond or MathArena for reasoning.</p></div></div></section><section><div class="wrap"><p><a class="btn" href="https://tokenra.io/register" target="_blank" rel="noopener">Test Hy4 Preview Yourself →</a></p><p>Read the <a href="/">Hy4 Preview home page</a> or the <a href="/hy4-vs-deepseek.html">Hy4 vs DeepSeek comparison</a>.</p></div></section></main><footer><div class="wrap"><p><a href="/">← Back to Hy4 Preview home</a> · <a href="/hy4-api.html">Hy4 API</a> · <a href="/about.html">About</a></p><p>© 2026 hy4ai.com. Benchmark values are transcribed from a supplied dataset compiled from public reports, provider disclosures, and clearly labeled third-party or internal evaluations. Results are provided for reference and may change with future evaluations.</p></div></footer></body></html>