Michelle Chala | Agentic Ops Internship Take-Home Stack: Python 3.13, GPT-4o-mini, python-dotenv
The naive system is one prompt doing four jobs at once:
def extract_action_items(meeting_notes):
prompt = f"Summarize the meeting and list all action items with owners.\nNotes:\n{meeting_notes}"
return llm(prompt)This is the same problem as John Sweller's cognitive load theory — you can't ask a system (or a person) to simultaneously summarize, classify, extract, and assign ownership without something getting dropped. The specific failure modes:
It treats certainty as binary. "Sarah might slip by a week" and "Alex will check with finance" get the same treatment. One is a heads-up, one is a commitment. The output doesn't distinguish them.
It fabricates or drops unowned tasks. "Someone needs to follow up with the vendor" has no owner. The model either invents one ("Team to follow up") or quietly drops the item. Neither is acceptable — the right move is to flag it and surface it for a human to resolve.
Zero auditability. Free text output means you can't verify where any item came from, how confident the model was, or which items still need attention. If it's wrong, you have no way to know.
No recovery path. Single pass, no feedback. If the model misreads something, that error ships.
raw notes
│
▼
[normalize_input] ← GPT cleans abbreviations, fixes casing, joins wrapped lines
│
▼
[Stage 1: Preprocess] ← pure Python — split statements, filter by signal words
│
▼
[Stage 2: Extract] ← 1 GPT call per candidate sentence
│
▼
[Stage 3: Memory] ← track committed items, build domain map
│
▼
[Stage 4: Feedback] ← second pass ONLY on low-confidence / unowned items
│
▼
[post-filter] ← null-task items go to open_questions, not action_items
│
▼
structured JSON
The core idea: decompose the problem so each step has exactly one job. Stage 1 never touches the LLM. Stage 2 processes one sentence at a time with a focused prompt. Stage 3 builds context from what's already been confirmed. Stage 4 only retries the problem cases — not everything.
Why GPT-4o-mini over Claude here: for hedged informal language like "sounds like maybe a week?" and "i honestly don't remember who" — GPT handles semantic ambiguity in colloquial text more naturally without needing as detailed a system prompt.
Why not fine-tuning or agent builder frameworks: this is a lightweight prototype, not a production system. Fine-tuning requires labeled data at scale. The Assistants API adds managed state I don't need — I built the memory myself because that's the interesting part to show.
Key ownership decision: owner is never inferred from topic context. If "Alex" isn't explicitly named as doing a task, owner stays null. A soft guess goes to possible_owner with an inference_reason so it's clear it's a suggestion, not a fact. Collapsing that uncertainty is exactly what the naive system does wrong.
{
"summary": "...",
"action_items": [
{
"task": "check with finance on budget approval",
"owner": "Alex",
"possible_owner": null,
"inference_reason": null,
"type": "commitment",
"confidence": 0.95,
"needs_clarification": false,
"evidence": "Alex will check with finance on budget approval"
},
{
"task": "follow up with vendor about pricing",
"owner": null,
"possible_owner": null,
"inference_reason": null,
"type": "vague",
"confidence": 0.55,
"needs_clarification": true,
"evidence": "Someone needs to follow up with the vendor about pricing"
}
],
"open_questions": [
"Someone needs to follow up with the vendor about pricing",
"Not sure who is owning the landing page yet"
]
}The evidence field is there so a human can audit any extraction and trace it back to the exact sentence. That's a deliberate design choice — the output should be trustworthy enough that someone doesn't have to re-read the notes.
First pass: ownership was inferred by proximity. The model was assigning Alex as owner of the vendor follow-up because Alex appeared earlier in the same notes. Fixed by making the ownership rule explicit in the prompt: "DO NOT guess ownership based on topic."
Second pass: Input B was fragmenting mid-sentence.
Messy prose with short wrapped lines was getting split into incomplete fragments like "alex said he's got the budget thing, he's gonna check with" — nothing the model could extract from. Added normalize_input() as a GPT pre-processing step that joins wrapped lines and expands abbreviations before any regex runs.
Third pass: Stage 4 was inferring Alex as possible owner for everything. Because Alex owned finance/budget in the domain map, Stage 4 was suggesting him for landing page, design delays, vendor follow-up — everything. Rewrote Stage 2 and Stage 4 prompts to explicitly prohibit topic-based ownership inference.
Fourth pass: null-task items were showing up in action_items.
Discussion entries with no extractable task were cluttering the output. Added a post-filter: null-task items go to open_questions only.
git clone https://github.com/submarinejuice/Nascent-Model.git
cd Nascent-Model
python3 -m venv venv
source venv/bin/activate # Windows: venv\Scripts\activate
pip install openai python-dotenvcp .env.example .envOpen .env and fill in:
OPENAI_API_KEY=sk-proj-...
You need a key with All permissions (not Restricted). Create one at platform.openai.com/api-keys. GPT-4o-mini costs fractions of a cent per run — the entire test suite is less than $0.05.
python3 test_cases.py25 tests covering all rule-based logic. Should show 25 passed | 0 failed. If this fails, it's an install issue, not an API issue.
python3 pipeline.py --saveRuns Input A (clean bullets) and Input B (messy prose) through both naive and pipeline, prints side-by-side results, and writes to outputs/:
output_a_naive.json— what the broken single-prompt system producesoutput_a_pipeline.json— what the redesigned pipeline producesoutput_b_naive.jsonoutput_b_pipeline.json
python3 pipeline.py --interactiveGives you a >> prompt. Available commands:
| Command | What it runs |
|---|---|
a |
Input A — clean bullet notes |
b |
Input B — messy informal prose |
d / d2 |
Startup engineering standup — clean / messy |
e / e2 |
Corporate finance meeting — clean / messy |
f / f2 |
Research lab — clean / messy |
g / g2 |
Product/UX team — clean / messy |
h / h2 |
DevOps/infra — clean / messy |
custom |
Paste your own notes, type END when done |
file |
Load any .txt file by path |
quit |
Exit |
Trying custom input:
>> custom
Paste your meeting notes below.
Type END on its own line when done:
- Jordan will send the contract by Thursday
- Someone needs to review staging before launch
- Maya mentioned the API might need another day
END
Label (e.g. 'standup', 'kickoff'): standup
Your name (optional, press Enter to skip): Michelle
If you enter your name, any "I'll do X" in the notes maps to you as owner instead of the literal string "I". All runs auto-save to outputs/.
SUMMARY: The team discussed Q2 launch timing, a potential design slip,
and unresolved ownership of the vendor follow-up and landing page.
ACTION ITEMS:
1. [COMMITMENT 95%] check with finance on budget approval
owner: Alex
evidence: "Alex will check with finance on budget approval"
2. [VAGUE 55%] follow up with vendor about pricing
owner: ? (possible: unassigned) ← NEEDS CLARIFICATION
evidence: "Someone needs to follow up with the vendor about pricing"
OPEN QUESTIONS (unowned or unclear — need human decision):
→ Someone needs to follow up with the vendor about pricing
→ Not sure who is owning the landing page yet
├── pipeline.py ← orchestrator, naive comparison, interactive mode
├── stage1_preprocess.py ← normalize + split + filter (1 GPT call + regex)
├── stage2_extract.py ← per-sentence extraction (1 GPT call each)
├── stage3_memory.py ← domain map, committed/unresolved tracking
├── stage4_feedback.py ← retry pass for low-confidence items only
├── test_cases.py ← 25 unit tests, zero API calls
├── inputs/ ← 12 test inputs (A, B + 5 domains × clean/messy)
├── outputs/ ← generated JSON on --save
├── .env.example
└── requirements.txt
Transcript-mode input where speaker labels ([00:15] Alex:) auto-resolve all "I" references. Streaming output so long meetings show results as they process. Configurable confidence threshold instead of hardcoded 0.5. The feedback loop architecture maps directly to how I'd build this for a real operational workflow — same pattern as what I built in Neurelle for querying a 13TB neuroimaging dataset on a federal HPC cluster, just different domain data.