Add multimodal C4 Bench evaluation

[sci-m-wang]

Motivation

Add C4 Bench, an image-text-to-text benchmark for cross-concept creative understanding through Chinese chengyu. The paper and public task rows are available at:

• Paper: https://arxiv.org/abs/2608.06501
• Dataset: https://huggingface.co/datasets/sci-m-wang/C4-Eval

Modification

• add the C4 dataset loader, official exact-recovery evaluator, primary and task-specific configs, dataset index entry, and complete BibTeX citation
• load structured image/text ChatML messages from Hugging Face
• forward multimodal ChatML messages without flattening them to strings
• normalize OpenCompass image items to OpenAI/LiteLLM image_url content
• allow ChatML inference to leave output length unset; LiteLLMAPI omits max_tokens when no value is configured
• update the English and Chinese custom-dataset documentation and add focused tests

The official C4 primary score uses exact recovery over H0, H1, H4, and E0 (884 rows). E1 is available separately and is excluded from the primary denominator.

BC-breaking (Optional)

No. Existing text-only ChatML behavior and explicitly configured output lengths are unchanged.

Use cases (Optional)

from opencompass.configs.datasets.c4_bench.c4_bench_gen import c4_bench_datasets

datasets = [*c4_bench_datasets]

Use an image-capable API backend such as LiteLLMAPI configured for a vision-language model. The C4 dataset config does not override model context or output length settings.

Testing

• flake8, isort, YAPF, YAML, compilation, and whitespace checks pass
• isolated end-to-end checks against the public 1,105-row file confirm 884 primary rows, structured image forwarding, official scoring, and omission of max_tokens when unset
• focused dataset, evaluator, ChatML, and LiteLLM tests are included; the local environment lacked torch and datasets, so the complete pytest run is left to CI

Checklist

Before PR:

• Pre-commit or other linting tools are used to fix the potential lint issues.
• Bug fixes are fully covered by unit tests, the case that causes the bug should be added in the unit tests.
• The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
• The documentation has been modified accordingly, like docstring or example tutorials.

After PR:

• If the modification has potential influence on downstream or other related projects, this PR should be tested with those projects.
• CLA has been signed and all committers have signed the CLA in this PR.

[ssiq]

Until now, OpenCompass is a pure text benchmark evaluation framework. Feel free to contribute this benchmark to our multi-model evaluation framework VLMEvalKit.

[sci-m-wang]

Thanks for the clarification. The C4 Bench integration has already been submitted to VLMEvalKit as open-compass/VLMEvalKit#1624:

open-compass/VLMEvalKit#1624

I will continue the integration and review there.