🚀 AI Product Manager Toolkit

A comprehensive collection of prompts, templates, tools, and frameworks for Product Managers working with AI/ML products.

Pull from a real system into the shape every script reads, then analyze — no hand-built CSV.
Runs against the bundled fixture, so it reproduces from a clean clone with no credentials. Regenerate with vhs docs/demo.tape.

📚 What's Inside

Section	Description
prompts/	AI-assisted prompts for core PM tasks, AI/ML work, and developer communities
templates/	PRD, postmortem, OKRs, RICE, technical specs, prioritization, prompt library management
scripts/	118 Python utilities by category: experiments, cost/ROI, delivery & velocity, adoption & health, incidents/SLO, feedback & support, risk & governance, launch, evals, strategy, agentic AI & orchestration, AI safety & red-teaming — plus shared modules for pricing, CSV headers, result envelope, and a smoke test that runs all 114 scripts against real sample data. See scripts/README.md for the full index and sample CSVs.
connectors/	Pull data from Jira (API or CSV export) and AI gateways (LiteLLM, OpenRouter) into the CSV shapes scripts/ already reads — canonical dataset contracts, a one-command CLI, and a self-test that verifies each connector's output actually feeds its consuming scripts. See connectors/README.md.
frameworks/	Prioritization, ML product lifecycle, build vs. buy, AI feature deprecation, SPACE (team health)
mcps/	MCP servers for Jira, Confluence, GitHub, Slack, Notion, Braintrust, LangSmith, product analytics, and Calendar/meetings (mcps/README.md)
agents/	System prompts, rules, and patterns for AI agents
evals/	Evaluation frameworks, scripts, and metrics for AI products
learning/	AI/ML fundamentals, AWS Bedrock for PMs, glossary, resources
strategy/	AI product strategy, pricing, go-to-market (see strategy/README.md)
governance/	Responsible AI, ethics, bias detection, risk assessment
communication/	Stakeholder management and executive communication
launch/	GTM playbooks for AI features
incidents/	Incident response and rollback strategies
career/	AI PM skills roadmap and interview prep
workflows/	Daily productivity and tool recommendations

🧭 Finding things

• By goal (recommended) → docs/tool-picker.md: what to open for planning, delivery, experiments, AI/ML work, and cost.
• Scripts → scripts/README.md: categorized index, one-line descriptions, and which sample CSV to use with each script.
• Prompts → prompts/README.md: index by category; files are prompts/*/*.prompt.md.
• Templates → templates/: PRD, OKR, RICE, technical spec, DX assessment, etc.
• MCPs → mcps/README.md: Jira, Confluence, GitHub, Slack, Notion, Braintrust, LangSmith, Product Analytics. Task-based routing → docs/tool-picker.md.
• Connectors → connectors/README.md: fetch.py --list for sources, fetch.py --describe issues for a column contract.
• Evals → evals/scripts/README.md: eval harness, regression runner, cost calculator.

🎯 Quick Start

Using Prompts

All prompt files end in .prompt.md. Copy the prompt content and use it with your preferred AI assistant (Claude, ChatGPT, Copilot, etc.).

# Example: Generate a PRD
cat prompts/core-pm/prd-generator.prompt.md

Using MCP Servers

MCP servers require Node.js 18+. See mcps/README.md for setup instructions.

cd mcps/servers/jira-pm-assistant
npm install
npm run build

Using Scripts

Python scripts require Python 3.10+. Sample data lives in scripts/samples/; see scripts/README.md for the full index and which sample file goes with each script.

pip install -r scripts/requirements.txt
python scripts/ab-test-calculator.py
python scripts/experiment-duration-calculator.py --baseline 0.05 --mde 0.10 --daily-visitors 5000
python scripts/ai-unit-economics-calculator.py --cost-per-request 0.002 --requests-per-month 1e6 --revenue-per-user 5 --mau 200000
python scripts/bedrock-cost-calculator.py --input-tokens 1000 --output-tokens 500 --model claude
python scripts/multi-model-cost-comparator.py --input-tokens 1000 --output-tokens 500
python scripts/model-selection-scorecard.py -s scripts/samples/sample-model-selection-scores.csv -w scripts/samples/sample-model-selection-weights.csv
python scripts/meeting-load-optimizer.py --csv scripts/samples/sample-meetings.csv
python scripts/feature-rollout-calculator.py --daily-volume 100000
python scripts/ai-initiative-roi-calculator.py --dev-cost 50000 --monthly-ai-cost 2000 --monthly-benefit 10000
python scripts/confidence-interval-calculator.py --n 500 --proportion 0.32
python scripts/nps-csat-summary.py nps --promoters 40 --passives 30 --detractors 30
python scripts/survey-sample-size.py --margin 0.05 --confidence 0.95
python scripts/latency-slo-calculator.py --availability 99.9 --requests-per-month 10e6
python scripts/churn-risk-calculator.py --cohort "New Users" --usage-drop 40 --adoption 25 --tickets 12
python scripts/prompt-cost-optimizer.py --file prompt.txt --model gpt-4o --requests-per-month 500000
python scripts/data-drift-detector.py --baseline baseline.csv --current current.csv
python scripts/adoption-funnel-analyzer.py --steps "Visit:10000" "Signup:4000" "Activate:2500" "Repeat:800"
python scripts/sla-uptime-calculator.py --sla 99.9 --incidents 45 120 15 --forecast-days 90
python scripts/velocity-trend-analyzer.py --sprints 38 42 35 45 40 48 --window 3 --target 45
python scripts/capacity-planning-calculator.py --team 6 --sprint-days 10 --pto 2 --meetings 0.2 --points-per-day 4
python scripts/cycle-lead-time-analyzer.py --csv tickets.csv --group-by type
python scripts/sprint-burndown-checker.py --csv burndown.csv --chart
python scripts/sprint-mix-report.py --csv sprint.csv --group-by type
python scripts/commitment-predictability-index.py --csv velocity.csv
python scripts/status-duration-analyzer.py --csv transitions.csv --chart
python scripts/experiment-result-interpreter.py --baseline 5.0 --variant 5.6 --n 8000
python scripts/backlog-aging-report.py --csv backlog.csv --oldest 15
python scripts/sprint-goal-checker.py --goals goals.csv --completed done.csv
python scripts/eval-label-economics.py --margin 0.05 --confidence 0.95 --proportion 0.5 --cost-per-label 2.50
python scripts/eval-score-trend.py --csv eval-runs.csv --chart
python scripts/incident-rate-trend.py --csv incidents.csv --chart
python scripts/risk-register-summary.py --csv risks.csv --top 5
python scripts/release-impact-summary.py --csv shipped.csv --version "v2.1.0" --bullets
python scripts/prompt-version-diff.py --old prompt_v1.txt --new prompt_v2.txt
python scripts/hallucination-safety-trend.py --csv evals.csv --metric-type hallucination --chart
python scripts/roadmap-timeline-summary.py --csv roadmap.csv --overlaps --by-quarter
python scripts/launch-readiness-score.py --csv checklist.csv --go-threshold 95
python scripts/feedback-theme-counter.py --csv feedback.csv --themes "pricing,reliability,ux,support"
python scripts/support-escalation-trend.py --csv tickets.csv --chart --group-by severity
python scripts/audit-checklist-summary.py --csv controls.csv --group-by domain --open
python scripts/budget-burn-summary.py --csv budget.csv --group-by category
python scripts/win-loss-summary.py --csv deals.csv --top 5 --group-by segment
python scripts/inference-latency-trend.py --csv latency-runs.csv --metric p99 --chart
python scripts/feature-adoption-trend.py --csv adoption.csv --chart --group-by segment
python scripts/stakeholder-signoff-tracker.py --csv signoffs.csv --pending --group-by deliverable
python scripts/dependency-blocked-summary.py --csv deps.csv --blocking
python scripts/beta-conversion-report.py --csv beta.csv --chart
python scripts/customer-health-score-trend.py --csv health.csv --at-risk-below 50 --chart --group-by segment
python scripts/release-cadence-report.py --csv releases.csv --period month --chart --group-by product
python scripts/agentic-cost-simulator.py --steps-per-task 6 --retry-rate 0.15 --tasks-per-month 20000
python scripts/agent-task-success-tracker.py --csv agent-runs.csv --target-success-rate 0.85
python scripts/guardrail-effectiveness-analyzer.py --csv guardrail-labels.csv --max-fn-rate 0.05
python scripts/red-team-coverage-tracker.py --csv redteam-results.csv
python scripts/model-deprecation-watch.py --csv model-portfolio.csv --warn-within-days 90
python scripts/model_pricing.py --check
python scripts/launch-checklist.py --name "Agent Copilot" --type backend --csv gate.csv

Trusting the numbers

• Pricing comes from one table (scripts/model_pricing.py), and every model carries a last_verified date. Anthropic rates are verified; Bedrock, OpenAI, and Google entries are inherited from earlier versions of this toolkit and are labeled UNVERIFIED until checked. Run python scripts/model_pricing.py --check to see what needs attention, and expect any comparison including an unverified model to say so.

• CSV headers don't have to match exactly. Duration (Minutes), duration_minutes, and DURATION MINUTES all resolve to the same column, so a raw Jira or analytics export works without hand-editing headers first.

• --output JSON carries the tool name, timestamp, and schema version alongside the result, so a number in a deck can be traced back to what produced it: python scripts/toolkit_io.py results.json.

• Every script runs, not just --helps. scripts/samples/ has real sample data for all 114 scripts, and scripts/smoke-test.py runs each one against it and checks it actually produces output: python scripts/smoke-test.py. 113 of 114 pass on a clean checkout with no pip install at all; the one exception is sentiment-analysis.py, which genuinely needs an NLP library (pip install textblob pandas) and now exits non-zero rather than reporting success without it.

• Connectors are checked the same way, but end to end: python connectors/self-test.py fetches from a fixture and then runs every script the dataset contract claims to feed, failing if any of them reads zero rows.

• Where the data came from is recorded too. connectors/fetch.py writes a .meta.json sidecar beside every CSV it produces — source, column mapping, row count, timestamp — so a number traces back past the script to the query that fed it.

🗂️ Directory Structure

pm-toolkit/
├── README.md
├── .gitignore
├── prompts/
│   ├── core-pm/              # PRDs, user stories, stakeholder updates, pricing page, API changelog
│   ├── ai-ml/                # ML system design, model cards, MLOps, migration playbooks
│   └── developer-community/  # AI accelerator resources
├── templates/                # RICE, OKRs, technical specs, prompt library management
├── scripts/                  # See scripts/README.md for categorized index and sample CSVs
├── connectors/
│   ├── datasets.py           # Canonical dataset contracts (the column shapes scripts read)
│   ├── fetch.py              # CLI: fetch.py <source> <dataset> --out FILE
│   ├── base.py               # Shared config, retries, normalization, provenance sidecar
│   ├── sources/              # One module per system (csvfile, jira, gateway)
│   ├── profiles/             # Per-instance field mappings (e.g. Jira custom field IDs)
│   └── fixtures/             # Recorded responses for offline runs and self-test
├── frameworks/               # Prioritization, ML lifecycle, build vs. buy, deprecation playbook, SPACE
├── mcps/
│   ├── guides/               # Setup and use case documentation
│   ├── TOOLS.md              # Server → MCP tool name reference
│   └── servers/              # Jira, Confluence, GitHub, Slack, Notion, Calendar, etc.
├── agents/
│   ├── system-prompts/       # Ready-to-use agent personas
│   ├── rules/                # Cursor rules, Claude instructions
│   ├── evaluation/           # Agent evaluation frameworks
│   └── patterns/             # Agent design patterns
├── evals/
│   ├── frameworks/           # LLM evaluation methodology
│   ├── scripts/              # Eval harness, eval summary report generator
│   ├── templates/            # Eval planning docs
│   └── metrics/              # AI product metrics guide
├── learning/                 # AI/ML fundamentals, glossary, resources
├── strategy/                 # AI product strategy frameworks
├── governance/               # Responsible AI checklist
├── communication/            # Executive communication prompts
├── launch/                   # AI feature launch checklist
├── incidents/                # Incident response playbook
├── career/                   # AI PM skills roadmap
└── workflows/                # Daily AI PM workflow

☁️ Building on AWS Bedrock

If you use Amazon Bedrock, see:

• AWS Bedrock for PMs – concepts, model families, data/compliance
• scripts/bedrock-cost-calculator.py – estimate inference cost
• prompts/ai-ml/bedrock-model-selection.prompt.md – choose a model
• evals/frameworks/bedrock-eval-guide.md – run evals with Bedrock

---

🤝 Contributing

This is primarily a personal toolkit, shared in case it's useful to others. It isn't actively looking for outside contributions, and PRs aren't reviewed on any schedule (or at all). If you want to change or extend something here, fork it — see CONTRIBUTING.md for how the repo is organized.

📄 License

MIT License - Use freely, and attribution appreciated.

---

Built for Product Managers navigating the AI era. 🧠