Classify multilingual (AZ/RU/EN) Apar user feedback into the handling department:
technical_support · customer_support · other. Metric: Macro F1.
pip install -r requirements.txt # Python 3.14, see DOCUMENTATION.md
python -m src.train_tfidf # Phase 0 baseline -> submissions/baseline_svc.csv
# or: make submissionconfig.py seeds, paths, hyperparameters (single source of truth)
data/ CONFIDENTIAL, gitignored — train/test/sample CSVs
src/
prep.py load, normalize (AZ folding for TF-IDF), features
validate.py validate_submission() — called before every save
train_tfidf.py Phase 0 baseline: 5-fold CV -> OOF -> submission
train_transformer.py Phase 1 (stub)
export_onnx.py Phase 2 CPU/ONNX INT8 (stub)
blend.py Phase 4 ensemble + threshold search (stub)
infer.py THE REVIEWED ARTIFACT — CPU-only, offline (finalized Phase 2)
oof/ out-of-fold predictions per model
submissions/ timestamped submissions + scores_log.csv
notebooks/ EDA only
≤600M params · ≤8 GB RAM · CPU-only inference · commercial-use licenses only ·
paid APIs allowed in training only · test.csv never used for training ·
dataset never leaves the machine.
| Stage | CV macro-F1 |
|---|---|
| TF-IDF + LinearSVC baseline | 0.8922 |
| xlm-roberta-base (5-fold) | 0.9089 |
| blend + per-class thresholds | 0.9186 |
Per-class (blend): technical 0.956 · customer 0.886 · other 0.916.
Deployed via src/infer.py → submissions/final_submission.csv (CPU, offline).
- Phase 0 — scaffolding + reproduced baseline (0.8922)
- Phase 1 — transformer fine-tune, local RTX 5070 (xlm-roberta-base; mdeberta-v3 dropped, NaN)
- Phase 2 — fp32 ONNX CPU inference + benchmark (INT8 rejected, −0.016)
- Phase 3 — synthetic customer_support generated + gated (+0.0004, dropped)
- Phase 4 — blend + per-class threshold tuning (0.9186)
- Phase 5 — time-ordered validation (no drift)
Reproduce: python -m src.train_tfidf · python -m src.train_transformer --model xlmr
· python -m src.blend · python -m src.train_transformer --deploy ·
python -m src.export_onnx · python -m src.infer