Skip to content

Latest commit

 

History

History
66 lines (44 loc) · 3.47 KB

File metadata and controls

66 lines (44 loc) · 3.47 KB

Reliability

Evalda's claim to being "battle-tested" isn't a vibe — it's two production events, both run against a live clock with no retry window, both finished with the leaderboard standing. This page is the receipts.


DataQuest 2026 (data science track, first deployment)

Evalda's first outing, judging the annual data science competition of the DataOverflow event.

Metric Value
Duration 7 continuous hours
Participants 164 across 41 teams
Total submissions 1,452 (435 accepted, 1,017 rejected)
Peak throughput 400 submissions in one hour
DDoS attacks absorbed 3
Total downtime < 1 minute (two config-only maintenance windows)

The lessons from this event (the preprocess() latency bypass, the trust-boundary redesign, the pre-event penetration audit) are written up honestly in architecture.md — Known Limitations and Lessons Learned.


AINS 4.0 (both tracks, overnight)

The second deployment ran both tracks simultaneously through an overnight event (21:00 → 05:00 UTC) — the data science track judging file submissions while the MLOps track provisioned and graded live cloud VMs.

Data science track

Metric Value
Submissions 349 (253 accepted, 96 rejected)
Teams 16 active of 27 registered, 61 users
Peak hour 67 submissions
Peak 10 minutes 18 submissions (at 4:50 AM, naturally)
Best accuracy reached 95.5%
Downtime 0

Submission pressure never let up — the hourly histogram is essentially flat from 1 AM to 5 AM (67, 64, 57, 66), meaning the judging pipeline ran at a sustained load all night rather than surviving one spike.

MLOps track

Metric Value
Grade runs 92 (64 graded, 28 rejected by the grader's gates)
VMs provisioned 15, across 2 AWS accounts (9 + 6)
Provisioning failures 0
Orphaned VMs after the event 0 (every VM reached terminated)
Challenges 3, opened sequentially (2h / 3h / 3h windows)
Teams 5 active of 7 registered

Every one of the three challenges was fully cracked — at least one team hit the maximum score of 1000 on each. The 28 non-graded runs were the grader's gates doing their job (VM unreachable, correctness gate failed, reset contract broken), not platform errors: each returned a precise verdict to the team instead of a silent failure.

Combined

Across both events: ~1,800 submissions, 48 active teams, 15 hours of live competition, zero unplanned downtime.


The 4 AM plot twist

One footnote for the readers who made it this far. At 4 AM the night before the event, a sanity call to Ossama Ferjani ended with a sentence you never want to hear at that hour: "You know free-trial AWS accounts cap your concurrent vCPUs, right?" With every team's VM running at once, the event needed more vCPUs than the account was allowed to have.

So the single-account provisioning layer became a multi-account load balancer overnight: a borrowed second account, a per-account vCPU budget in the atomic slot claim, per-account credentials and networks in Terraform, account-aware watchers and teardown. (The backend itself sat on an m6i.2xlarge in a separate paid account, unbothered.) Hours later it placed 15 VMs across the two accounts with zero failures.

Lesson: quotas are architecture — and the best pre-event engineering tool is a friend who's been burned before you.