InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution
in Interactive Website Generation?

Qiyao Wang^1,2,*, Haoran Hu^3,*, Longze Chen^1,2, Hongbo Wang³, Hamid Alinejad-Rokny⁴, Yuan Lin^3,†, Min Yang^1,5,†

¹SIAT-NLP, ²UCAS, ³DUT-IR, ⁴UNSW Sydney and ⁵SUAT

^* Equal Contribution    ^† Corresponding Authors

🌐 Homepage | 🤗 Dataset | 📖 Paper | 🤗 HuggingFace Paper | GitHub

This repo contains the evaluation code for the paper "InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?"

🔔 News

• 😄 [2026-04-30] Releasing Code and Data.
• 😄 [2026-04-24] Releasing Website.
• 🔥 [2026-02-16] Research Begining.

📅 Timeline

• Code
• Dataset
• Software Demostration (Comming soon...)

📝 Introduction

InteractWeb-Bench is a multimodal interactive benchmark for evaluating website generation agents under real-world, non-expert user conditions.

It simulates ambiguous, noisy, and conflicting user instructions through persona-driven user agents, and introduces a dynamic action space (Clarify, Implement, Verify, Submit) to assess agents’ ability to escape “blind execution” and align with user intent.

This project provides a realistic environment for studying interactive code generation, intent clarification, and GUI-based verification.

🚀 Quick start

Follow the steps below to quickly set up and run InteractWeb-Bench.

1. Environment Setup

conda create -n InteractWeb-Bench python=3.10 -y
conda activate InteractWeb-Bench
pip install -r requirements.txt

playwright install chromium

Install Node.js:

cd scripts
bash install_node.sh

2. Configure Environment Variables

Create your .env file:

cp .env.example .env

Edit .env and fill in your API keys and model endpoints.

3. Configure Experiment Settings

Edit config.yaml in the root directory:

data_path: "path_to_your_dataset.jsonl"
output_dir: "/path_to_your_workspace/experiment_results"
models:
  builder_model: "your_builder_model"
  visual_copilot_model: "your_visual_model"
  webvoyager_model: "your_judge_model"
  user_model: "your_user_model"

4. (Optional) Launch Local Models

If using local models, start your vLLM services:

bash src/scripts/deploy_your_local_model.sh

You can configure multiple models and ports via LOCAL_MODELS_MAP.

5. Run the Benchmark

python src/experiment/run_simulation.py --config /your_config_path/

6. (Optional) Docker Deployment

docker load -i interactweb-bench_v1.0.tar
bash docker_run.sh

Then run:

python src/experiment/run_simulation.py --config /your_config_path/

Citation

When citing this work, please use the following BibTeX entry:

Contact

Feel free to contact the author with wangqiyao25@mails.ucas.ac.cn.