MCQ Test Data Generator for Item Analysis

Generate realistic Multiple Choice Question (MCQ) test data for item analysis, psychometric checks, and educational assessment pipelines.

Instead of uniform random A/B/C/D answers, both generators use a 2-Parameter Logistic (2PL) Item Response Theory (IRT) model. Candidate ability and question difficulty drive who gets which items right, so metrics such as the difficulty index ((p)-value), discrimination index, and distractor patterns behave more like a real exam.

---

Features

• MCQ only: one correct choice, four options (A, B, C, D)
• Configurable number of questions and number of candidates
• Two generators: classic (clear nested loops) and optimized (vectorized NumPy)
• Outputs always under output/, never in the project root
• Timestamped run folders so re-runs do not overwrite previous datasets
• Each run writes a matching pair: candidate_responses.csv + answer_key.csv

---

Project layout

.
├── generate_test_data.py              # Classic (nested-loop) generator
├── generate_test_data_optimized.py    # Optimized (vectorized) generator
├── output_utils.py                    # Shared timestamp / path helpers
├── README.md
├── Objective.md                       # Original requirements notes
└── output/
    ├── classic/                       # Runs from generate_test_data.py
    │   └── <YYYY-MM-DD_HHMMSS>/
    │       ├── candidate_responses.csv
    │       └── answer_key.csv
    ├── optimized/                     # Runs from generate_test_data_optimized.py
    │   └── <YYYY-MM-DD_HHMMSS>/
    │       ├── candidate_responses.csv
    │       └── answer_key.csv
    └── legacy/                        # Older one-off files (if present)

---

Two generators

Script	Output tag	Implementation	Best for
generate_test_data.py	classic	Nested Python loops	Readability, teaching the model, small/medium data
generate_test_data_optimized.py	optimized	Vectorized NumPy	Large datasets, many repeated runs

	Classic	Optimized
Statistical model	2PL IRT	2PL IRT (same idea)
Output files	Same CSV shapes	Same CSV shapes
Same seed, same cells?	—	No — different RNG order; stats still comparable
Code style	Easy to follow and change	Faster at scale; denser array code

Use classic when you care about clarity. Use optimized when generation volume or speed matters.

---

Output layout

Every run creates a new folder:

output/<classic|optimized>/<YYYY-MM-DD_HHMMSS>/
  candidate_responses.csv
  answer_key.csv

• Timestamp format: YYYY-MM-DD_HHMMSS (local time)
• If two runs start in the same second, a numeric suffix is added (…_1, …_2, …)
• Previous runs are left untouched

File formats

candidate_responses.csv

Rows	One per candidate (CAND_0001, …)
Columns	Q1, Q2, …
Values	Chosen option: A, B, C, or D

answer_key.csv

Column	Meaning
Question_ID	e.g. Q1
Correct_Answer	Ground-truth option for that item

---

Requirements

• Python 3.8+
• pandas
• numpy

pip install pandas numpy

On macOS, use python3 if python is not available.

---

Quick start

Classic generator:

python3 generate_test_data.py

Optimized generator:

python3 generate_test_data_optimized.py

Default CLI settings (in each script’s __main__ block):

• 100 questions
• 300 candidates

Example result path:

output/classic/2026-08-01_135636/candidate_responses.csv
output/classic/2026-08-01_135636/answer_key.csv

---

Programmatic usage

from generate_test_data import generate_item_analysis_data
# or:
# from generate_test_data_optimized import generate_item_analysis_data

df_responses, df_key, responses_path, key_path = generate_item_analysis_data(
    num_questions=20,
    num_candidates=250,
    seed=42,
)

print(responses_path)  # path to the new candidate_responses.csv
print(key_path)        # path to the new answer_key.csv
print(responses_path.parent)  # the timestamped run folder

Parameters

Parameter	Type	Default	Description
num_questions	int	20	Number of MCQ items
num_candidates	int	100	Number of candidates
seed	int	42	Random seed (reproducible within that script)
output_dir	path-like	output/<classic|optimized>/	Base folder for runs
timestamped	bool	True	If True, create a unique timestamp subfolder under the base
responses_filename	str	"candidate_responses.csv"	Responses file name inside the run folder
answer_key_filename	str	"answer_key.csv"	Answer-key file name inside the run folder

Return value

A 4-tuple:

1. df_responses — pandas DataFrame of choices
2. df_key — pandas DataFrame of the answer key
3. responses_path — Path to the written responses CSV
4. key_path — Path to the written answer-key CSV

Disable timestamps (overwrite mode)

# Write directly into the base folder (can overwrite previous files there)
generate_item_analysis_data(timestamped=False)

# Custom base path, still with a timestamp subfolder
generate_item_analysis_data(output_dir="my_runs", timestamped=True)

# Custom base path, no timestamp
generate_item_analysis_data(output_dir="my_runs/latest", timestamped=False)

---

How the model works

Correct/incorrect outcomes follow the 2PL logistic model:

[ P(Y_{ij} = 1) = \frac{1}{1 + e^{-a_j (\theta_i - b_j)}} ]

Symbol	Meaning	How it is sampled
(\theta_i)	Candidate ability	(\mathcal{N}(0, 1))
(b_j)	Item difficulty	Uniform on ([-2, 2]) (easy → hard)
(a_j)	Item discrimination	Uniform on ([0.5, 2.0])

Flow per answer:

1. Compute (P(\text{correct})) from ability, difficulty, and discrimination.
2. Draw whether the candidate is correct.
3. If correct → assign the key option.
4. If incorrect → pick one of the other three options at random (equal distractors).

That keeps response patterns ability-linked while still producing full A–D choice data for distractor analysis.

---

Choosing classic vs optimized

Situation	Prefer
Learning / demos / code review	Classic
Changing rules (blanks, fatigue, etc.)	Classic (easier to edit)
Thousands+ candidates or many Monte Carlo runs	Optimized
Need bit-identical files to an old classic run	Keep the classic CSV; do not regenerate with optimized

Shared path logic lives in output_utils.py (resolve_run_dir) so both scripts use the same folder rules.

---

License / notes

This project is a local utility for generating synthetic assessment data. It does not perform item analysis itself — it produces inputs you can feed into an analysis pipeline.