fix(judge): make ltp judge robust against log pollution and duplicate summaries - #5
Open
Freefor100 wants to merge 1 commit into
Open
fix(judge): make ltp judge robust against log pollution and duplicate summaries#5Freefor100 wants to merge 1 commit into
Freefor100 wants to merge 1 commit into
Conversation
judge_ltp-{musl,glibc}.py 三处防御性修复：
1. 解析前剥离 ANSI 色码。内核把带色日志（如 "\x1b[93m[WARN] ..."）混入
测试输出流时，原判分器在 int() 解析处崩溃
（ValueError: invalid literal for int() with base 10: '0\x1b[93m[WARN]'），
导致整场评测中断。
2. Summary 数值解析加防御：无法解析的行跳过而非崩溃（数值位置可能被
交错的内核日志污染）。
3. 每个用例只累计第一份 Summary。部分内核会打印两份相同的 Summary
（测试程序自身 + 内核 wrapper 合成），原判分器两份都累加，passed
翻倍。正常输出（每用例一份 Summary，即脚本自带模板的格式）不受影响。
验证：此前必崩的一例评测现已完整解析；其 ltp passed 回落至单份
Summary 口径（约减半），干净的单份 Summary 输入结果不变。
已知局限：判分器信任串口输出中的 RUN 用例名——内核若打印不同名的
假用例（如加序号前缀重复运行、或伪造自定义用例名），判分器无法
区分真实执行与伪造记录，此类问题需复测（隐藏测例）与评审把关。
Co-authored-by: DeepSeek V4 Flash <deepseek-v4-flash@deepseek.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
问题
两个参赛内核的评测在 LTP 组判分时暴露了 judge_ltp 的健壮性问题：
评测崩溃：内核把带 ANSI 色码的日志（如
\x1b[93m[WARN] ...）混入测试输出流时，判分器在ValueError: invalid literal for int() with base 10: '0\x1b[93m[WARN]'处崩溃，导致整场评测中断、无评分结果。
重复计分：部分内核为每个用例打印两份相同的 Summary 块（测试程序自身的 + 内核 wrapper 合成的），
判分器两份都累加，passed 计数翻倍。
修复（judge_ltp-{musl,glibc}.py）
\x1b\[[0-9;]*m）；验证
已知局限
判分器信任串口输出中的 RUN 用例名——若内核打印不同名的伪造用例（如加序号前缀重复运行、
或伪造自定义用例名），判分器无法区分真实执行与伪造记录。此类问题超出判分脚本能防御的范围，
需复测（隐藏测例）与评审环节把关，本 PR 仅修复上述解析层问题。