Bug: `parse_flag_from_text` returns only prefix (`flag`/`picoCTF`) and can inflate correctness/pass@k metrics

[sheng1feng]

Summary

parse_flag_from_text in rl_helpers.py currently returns only the flag prefix ("flag" or "picoCTF") instead of the full flag body (e.g., flag{...}), due to a regex capture-group + re.findall interaction.

This can make different flags look identical during correctness checks, and can inflate training reward/evaluation metrics in tool-use pipelines.

Affected Code

In rl_helpers.py (on origin/main, commit tested: 558d307):

FLAG_CONTENT_RE = re.compile(r"(flag|picoCTF)\{[^}]+\}", flags=re.IGNORECASE)

def parse_flag_from_text(text):
    flag = re.findall(FLAG_CONTENT_RE, text)
    if not flag:
        return None
    return flag[-1]

## Root Cause

re.findall returns captured groups when the pattern contains capturing parentheses.
So for:

- flag{abc} -> returns "flag"
- flag{xyz} -> returns "flag"

The parser is therefore not comparing full flags.

## Minimal Reproduction

import re

FLAG_CONTENT_RE = re.compile(r"(flag|picoCTF)\{[^}]+\}", flags=re.IGNORECASE)

def parse_flag_from_text(text):
    m = re.findall(FLAG_CONTENT_RE, text)
    return m[-1] if m else None

print(parse_flag_from_text("flag{correct_one}"))    # flag
print(parse_flag_from_text("flag{totally_wrong}"))  # flag
print(parse_flag_from_text("picoCTF{real}"))        # picoCTF
print(parse_flag_from_text("picoCTF{fake}"))        # picoCTF

## Expected vs Actual

Expected:

- parse_flag_from_text("flag{abc}") == "flag{abc}"

Actual:

- parse_flag_from_text("flag{abc}") == "flag"

## Impact

This affects any path that uses parse_flag_from_text for correctness:

- compute_correctness_reward(...)
- legacy pass@k / majority@k parsing path in tool-use evaluation

Different predicted flags with same prefix can be considered equal.

## Additional Consistency Note

eval_llm.py uses a different parser pattern that extracts full flag{...} strings, so behavior is inconsistent across evaluation paths.