[workshop-sim] Workshop Simulation Report — 2026-08-15 (Run #37, 1000×Monte Carlo)

[github-actions]

Overview

• Date: 2026-08-15
• Students simulated: 46 × 1000 Monte Carlo runs
• Workshop steps available: 29/29
• Overall success rate: 19.5% (95% Monte Carlo interval: 19.2%–19.9%)
• Highest-dropout step: 07-first-workflow (35.6% conditional dropout among 22,440 at-risk runs; 95% Monte Carlo interval: 35.0%–36.2%)
• Lowest curriculum quality step: 07d-confirm-model-access.md (overall score 7.19/10)
• Learning KPI index: 6.15/10 (active_learning 4.19 · checkpoint_quality 8.97 · scaffolding 5.00)
• Model: unknown / 2026-07-assumption-model-v2 (parameter hash n/a)
• Limitation: synthetic results reflect explicit model assumptions; intervals exclude model and population-assumption uncertainty

Part Summary

Part	Files	Mean Score	Std Dev
Part 1 — core path (lessons 00–14)	15	8.29 / 10.0	±0.97
Part 2 — advanced (lessons 15+)	14	8.11 / 10.0	±0.33
Overall corpus	29	8.20 / 10.0	±0.87

No pages are classified as other.

Critical Findings

1. Step 07 (Part 1) is the single largest dropout gate: 35.6% conditional failure rate driven by two compounding failure modes — workflow-authoring-friction (4,641 failures) and copilot-access-missing (3,343 failures). Copilot access errors silently block all subsequent steps regardless of skill level; the page's access-verification gate is correctly positioned but lacks inline diagnosis for the most common causes of a failed test prompt.
2. Conceptual funnel (Steps 04–05c, Part 1) sheds ~40% of beginners: concept-overload at Step 04 (5,271 failures) and agentic-concept-gap at Step 05 (5,784 failures) cascade. Both pages rely on read-and-reflect activities with no hands-on verification, and neither provides a recovery resource for learners who cannot pass the self-check. Active-learning scores (4.6 and 2.4/10 respectively) confirm the structural gap.
3. Learning quality health is adequate but uneven: learners who reach the hands-on steps (07, 09, 24) find strong active-learning scaffolding (AL ≥ 6.8). However, the conceptual-introduction cluster (Steps 04–05b) has AL scores of 2.4–4.6/10, meaning learners who do not drop out at these pages are progressing without adequate skill verification. The overall Learning KPI of 6.15/10 masks this gap.
4. The most urgent repair is in Part 1 (Step 07): adding inline Copilot-access diagnosis (for example, the three most common failure causes before the IMPORTANT callout) is a targeted access-barrier fix that does not reduce cognitive demand or degrade the learning KPI index.

Top Repairs to Prioritize

Note: some student dropout is expected and acceptable. Repairs must maintain or improve the learning KPI index — do not lower the cognitive bar or remove practice to chase headline completion numbers.

1. Add inline Copilot-access diagnosis to Step 07 before the IMPORTANT callout, listing the three most common failure causes (no Copilot subscription, org policy, missing seat). (completion impact: ↑ · learning KPI impact: ↔)
2. Add recovery resources and a verification loop to Step 05 (agentic-intro): after Activity 3, include a <details> block linking to the deep-dive side quest and a one-sentence confidence question so learners can self-identify a concept gap before continuing. (completion impact: ↑ · learning KPI impact: ↑)
3. Add a scaffolded worked-example to Step 04 (actions-intro): after the labeling-reveal, add a second worked example with a different trigger/job pattern and a new labeling prompt — giving learners who answered incorrectly a second attempt with corrective feedback rather than just the answer. (completion impact: ↑ · learning KPI impact: ↑)

Dropout by step

Step	At-risk runs	Conditional dropout	95% CI	Failure mode	Top reason
07-first-workflow	22,440	35.6%	35.0%–36.2%	Access barrier	Copilot access not verified / workflow authoring blocked
05-agentic-intro	38,862	20.5%	20.1%–20.9%	Learning barrier	Learner cannot distinguish agentic from standard workflows
05c-agentic-practice	30,898	13.4%	13.0%–13.8%	Learning barrier	Learner cannot correctly classify agentic vs standard tasks
04-actions-intro	44,133	11.9%	11.6%–12.2%	Learning barrier	Concept overload: five new terms introduced without recovery path
05b-agentic-security	26,750	10.2%	9.8%–10.6%	Learning barrier	Learner cannot articulate the security model for agentic workflows
06-install-gh-aw	24,025	6.6%	6.3%–6.9%	Access barrier	CLI extension installation friction
17-add-mcp-tools	12,293	4.7%	4.3%–5.1%	Access barrier	MCP tooling setup friction
15-conditional-logic	13,072	4.3%	4.0%–4.6%	Learning barrier	Conditional logic authoring friction
18-share-and-reuse	11,715	4.3%	3.9%–4.6%	Learning barrier	Workflow reuse pattern friction
02-setup	46,000	4.1%	3.9%–4.2%	Access barrier	Codespace setup friction
19-research-driven	11,217	3.9%	3.6%–4.3%	Learning barrier	Research node pattern friction
14b-pr-reviewer	13,505	3.2%	2.9%–3.5%	Learning barrier	Event trigger pattern friction
09-agentic-editing	13,947	3.2%	2.9%–3.5%	Learning barrier	Workflow editing pattern friction
24-self-hosted-runners	9,920	2.8%	2.5%–3.1%	Access barrier	Self-hosted runner config friction
08-run-your-workflow	14,456	1.6%	1.4%–1.8%	Access barrier	UI run guidance gap

Curriculum quality and learning KPIs

File	Overall	Active Learning	Checkpoint Quality	Scaffolding	Learning KPI	Weakest dim	Repair priority
07d-confirm-model-access.md	7.19	3.5	10.0	5.0	6.27	AL	High
04-github-actions-intro.md	7.42	4.6	10.0	5.0	6.67	AL	High
05-agentic-workflows-intro.md	7.45	2.4	10.0	5.0	5.87	AL	High
15-conditional-logic.md	7.53	3.8	10.0	5.0	6.38	AL	Medium
08-run-your-workflow.md	7.67	3.0	10.0	5.0	6.09	AL	Medium
14b-pr-reviewer-workflow.md	7.67	4.8	10.0	5.0	6.75	AL	Medium
16-connect-data-source.md	7.71	3.8	10.0	5.0	6.38	AL	Medium
05b-agentic-workflows-security.md	7.75	2.5	10.0	5.0	5.91	AL	High
17-add-mcp-tools.md	7.75	3.4	10.0	5.0	6.24	AL	Medium
20-persistent-memory.md	7.81	3.6	10.0	5.0	6.31	AL	Low
26-manage-costs-and-budgets.md	7.85	4.0	10.0	5.0	6.45	AL	Low
14-next-steps.md	7.91	3.3	10.0	5.0	6.20	AL	Low
08b-interpret-your-run.md	7.99	3.7	10.0	5.0	6.35	AL	Low
21-inline-sub-agents.md	7.99	4.1	10.0	5.0	6.49	AL	Low
09-agentic-editing.md	8.03	4.8	10.0	5.0	6.75	AL	Low
02a-setup-codespace.md	8.09	5.0	10.0	5.0	6.82	AL	Low
18-share-and-reuse.md	8.15	4.5	10.0	5.0	6.64	AL	Low
28-orchestrate-workflows.md	8.23	4.9	10.0	5.0	6.78	AL	Low
25-audit-and-observability.md	8.25	5.0	10.0	5.0	6.82	AL	Low
22-error-handling-and-resilience.md	8.29	5.2	10.0	5.0	6.89	SC	Low
27-evaluate-workflow-quality.md	8.37	5.6	10.0	5.0	7.04	SC	Low
19-research-driven-training-node.md	8.39	5.7	10.0	5.0	7.07	SC	Low
23-ab-experiments.md	8.53	6.4	10.0	5.0	7.33	SC	Low
05c-agentic-workflows-practice.md	8.59	6.7	10.0	5.0	7.44	SC	Low
07-your-first-workflow.md	8.61	6.8	10.0	5.0	7.47	SC	Low
24-self-hosted-runners.md	8.63	6.9	10.0	5.0	7.51	SC	Low
00-welcome.md	10.0	0.0	0.0	5.0	1.36	AL	n/a (non-learning)
01-prerequisites.md	10.0	0.0	0.0	5.0	1.36	AL	n/a (non-learning)
06-install-gh-aw.md	10.0	3.6	0.0	5.0	2.67	CQ	n/a (setup)
Cohort mean	8.20	4.19	8.97	5.00	6.15

Segment breakdowns

By technical level

Level	Students	Mean success rate
advanced	5	41.5%
actions-user	11	39.5%
github-basic	19	13.3%
beginner	11	0.4%

By personality

Personality	Students	Mean success rate
impatient	6	22.0%
methodical	12	21.3%
skeptical	7	19.6%
confused	6	18.4%
curious	15	17.5%

By UI preference

UI preference	Students	Mean success rate
CLI-preferred (ui_preferred: false)	24	28.0%
UI-preferred (ui_preferred: true)	22	10.3%

UI-preferred students are disproportionately affected by the required Codespace terminal at Steps 06–07. This is expected in the model assumptions (not a measured outcome).

Notable student journeys (3)

Surprising success — Learner 026 (advanced · confused · cli · ui_preferred: false · 50.9% success rate)
Despite a confused personality, this advanced DevOps engineer using the CLI achieved the highest-tier success rate. Prior runs (18,413 successes accumulated) and CLI fluency mean the terminal-heavy Steps 06–07 present no additional friction. The confused personality slows progress at conceptual steps but doesn't cause dropout when prior experience fills the gap. Lesson: high technical level compensates for personality disadvantage at access-barrier steps.

Unexpected dropout — Learner 018 (beginner · curious · CCA · ui_preferred: true · 0.2% success rate)
Despite a curious personality (which predicts persistence) and prior runs (131 successes), this learner achieves near-zero success. The CCA tool preference means the core-step Codespace terminal path is unfamiliar. The conceptual cluster (Steps 04–05) generates agentic-concept-gap failures before the learner reaches the hands-on payoff. Curiosity is not sufficient without a viable terminal path and conceptual scaffolding.

Content-gap case — Learner 037 (beginner · methodical · cli · ui_preferred: false · 3.7% success rate)
A methodical personality and CLI preference are favourable signals, yet this learner's most common failure is 05-agentic-intro (agentic-concept-gap). The page's three activities are reflective-only; a methodical learner who cannot verify understanding via a hands-on check will stall at the self-assessment questions. This case illustrates that the active-learning gap in Step 05 (AL = 2.4/10) affects even motivated, CLI-comfortable learners.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔬 Workshop Student Simulator · 124.6 AIC · ⌖ 7.89 AIC · ⊞ 10.4K · ◷

• expires on Aug 16, 2026, 2:52 AM UTC