Skip to content

fix(molrecbench_wild): give the converter its required track column - #1668

Open
Anai-Guo wants to merge 1 commit into
open-compass:mainfrom
Anai-Guo:fix/molrecbench-wild-track-column
Open

fix(molrecbench_wild): give the converter its required track column#1668
Anai-Guo wants to merge 1 commit into
open-compass:mainfrom
Anai-Guo:fix/molrecbench-wild-track-column

Conversation

@Anai-Guo

@Anai-Guo Anai-Guo commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

The bug

MolRecBenchWildDataset.evaluate() cannot run at all. It builds a two-column
frame and then calls the converter with a track= keyword:

selected = pd.DataFrame({
    'index': sample_ids,
    'prediction': [prediction_map[index] for index in selected_indices],
})

from .utils.molrecbench_wild import convert_dataframe, score_records

converted, _ = convert_dataframe(selected, track=self.track)

But convert_dataframe takes no track argument — it reads the track from a
required column:

def convert_dataframe(dataframe, *, sheet="Sheet1", id_suffix=""):
    required_columns = {"index", "track", "prediction"}

So every run of any of the three MolRecBench-Wild tracks raises:

TypeError: convert_dataframe() got an unexpected keyword argument 'track'

and simply deleting the keyword is not enough, because the frame that
evaluate() builds has no track column either:

WorkbookConversionError: sheet 'Sheet1' is missing columns: track

(score_records(ground_truth, converted, self.track, ...) a few lines below is
correct — track really is a positional parameter there, which is likely
where the mix-up came from.)

The fix

Supply the track as the column the converter asks for, and drop the keyword.

Verification

Ran the real vlmeval/dataset/utils/molrecbench_wild package at main
(5a91183) against the exact frame evaluate() constructs.

Before — both failure modes:

--- convert_dataframe(selected, track=self.track) ---
TypeError: convert_dataframe() got an unexpected keyword argument 'track'

--- keyword removed but column still absent ---
WorkbookConversionError: sheet 'Sheet1' is missing columns: track

After — records convert, no diagnostics:

records= 2 issues= 0
  id= mol_0001.png smiles= 'CCO'
  id= mol_0002.png smiles= 'c1ccccc1'

(second prediction was fenced ```json output, to check the normal
extraction path still runs rather than falling back to empty records).

pre-commit's flake8 (7.1.2, --max-line-length=120 --ignore=W503) and isort
6.0.1 are clean on the changed file, and yapf produces the same diff before and
after the change — the added lines are untouched by it.

🤖 Generated with Claude Code

MolRecBenchWildDataset.evaluate() calls
convert_dataframe(selected, track=self.track), but convert_dataframe's
signature is (dataframe, *, sheet, id_suffix) - it reads the track from a
required "track" column instead of a keyword argument. Every evaluation
run therefore dies with

    TypeError: convert_dataframe() got an unexpected keyword argument 'track'

and dropping the keyword alone is not enough, because the frame that
evaluate() builds only has "index" and "prediction":

    WorkbookConversionError: sheet 'Sheet1' is missing columns: track

Add the track column to the frame and drop the keyword.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant