Skip to content

Commit a2a5b6f

Browse files
Bordaclaude[bot]
andcommitted
refine(research:kaggle): win+teach goals + bare-# spacer check
- add dual equal-weight mandate to kaggle SKILL/description: win (leakage-safe CV, metric-aligned loss/model) and teach (reads top-to-bottom like a lecture); every decision motivated - require small single-purpose cells (one action per cell) and a one-line why per cell; extensive structured section markdown framing each plot before and after - add Step 4 verify check that greps the generated file for bare #/##/... heading-spacer lines and auto-fixes them to true blank lines — prose-only compliance proved insufficient - update research README kaggle section and style-rules accordingly - bump research 0.10.6 -> 0.10.7 --- Co-authored-by: claude[bot] <209825114+claude[bot]@users.noreply.github.com>
1 parent a859615 commit a2a5b6f

4 files changed

Lines changed: 53 additions & 16 deletions

File tree

plugins/research/.claude-plugin/plugin.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,5 +7,5 @@
77
"license": "Apache-2.0",
88
"name": "research",
99
"repository": "https://github.com/Borda/AI-Rig",
10-
"version": "0.10.7"
10+
"version": "0.10.8"
1111
}

plugins/research/README.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -516,7 +516,7 @@ ______________________________________________________________________
516516

517517
### `/research:kaggle` — Kaggle competition notebook
518518

519-
Generates Kaggle competition notebook as Jupytext `# %%` Python script (compatible with VS Code Jupyter, JupyterLab, `jupytext --to notebook`). Distills competition context from Kaggle API or URL, asks for missing facts via grounding protocol, generates structured notebook via `foundry:sw-engineer`.
519+
Generates Kaggle competition notebook as Jupytext `# %%` Python script (compatible with VS Code Jupyter, JupyterLab, `jupytext --to notebook`). Distills competition context from Kaggle API or URL, asks for missing facts via grounding protocol, generates structured notebook via `foundry:sw-engineer`. Tuned to win — leakage-safe CV, metric-aligned modeling — as much as to teach: small single-purpose cells, each carrying a why.
520520

521521
**Invocation**:
522522

@@ -543,11 +543,17 @@ Full mode (`<name>.py`): Header + Setup, Imports + Constants, EDA, Dataset + Dat
543543
- All bash commands as `! cmd` — never `subprocess`, never `get_ipython().system()`
544544
- PyTorch Lightning (`pl`) + `torchmetrics` for all DNN training, even simple baselines
545545
- `timm.create_model(...)` for image classification; SMP for segmentation; XGBoost for tabular
546+
- Small, single-purpose cells — one load/transform/plot/check/train call per cell, never bundled
547+
- Every cell carries a why — inline comment or preceding markdown sentence names the specific reason (metric, leakage risk, memory limit), never restates the code
548+
- Long comments (multi-line, paragraph-length) never sit inside a code cell — extracted to a markdown cell; mid-cell → split into code → markdown → code
549+
- Section markdown is extensive, structured (tables/lists/blockquotes, not dense prose), and reads like a lecture — every plot framed by a "what to expect" cell before and a "finding + implication" cell after
546550
- Commented-out hyperparameter alternatives for every tunable value
547551
- `del` + `gc.collect()` + `time.sleep(9)` for GPU memory management
548552

549553
**Grounding protocol**: all competition-specific facts (input modality, eval metric, submission format) must come from fetched URL, user answer, or past notebook. Skill asks via `AskUserQuestion` for ungroundable required facts — never hallucinates competition details.
550554

555+
**Bare-`#` heading-spacer check**: Step 4 verify mechanically greps the generated file for bare `#`/`##`/... lines used as spacers inside markdown cells (renders as an empty heading + oversized margin in Jupyter/Kaggle, not whitespace) and auto-fixes them to true blank lines — prose-only compliance with the style rule proved insufficient in practice.
556+
551557
**Competitor context**: `resources/competitors/` contains `.ipynb` or `.py` files → skill reads each, summarises approach (model choice, preprocessing, augmentation strategy) before profiling problem. Findings inform detection method + domain-specific preprocessing.
552558

553559
**Package distillation gate**: after notebook verified, skill offers to extract reusable helpers (data loading, submission builder, metric utilities) into `src/<package>/` with Google-style docstrings + tests. Refactored notebook written as new file (`notebooks/01_<name>_pkg.py`) — validated baseline never modified.
@@ -839,7 +845,7 @@ Plugin part of Borda-AI-Rig project. Skills + agents in `plugins/research/` in r
839845

840846
Skill files (`plugins/research/skills/*/SKILL.md`) and agent files (`plugins/research/agents/*.md`) = canonical source of truth — README must stay in sync. Any skill behavior change (flags, NOT-for scope, trigger conditions) requires update here.
841847

842-
Version bumps per project policy: new capability → minor bump; fixes, wording, refactors → patch bump. Current version: `0.10.6`.
848+
Version bumps per project policy: new capability → minor bump; fixes, wording, refactors → patch bump.
843849

844850
**Mode-dispatch layout**: large conditional sections externalised under `skills/<skill>/modes/*.md`, loaded on demand. Run's hypothesis pipeline, team, report modes under `skills/run/modes/`. ML-concepts reference for `research:scientist` under `agents/scientist/ml-concepts.md` — loaded only for ML-domain tasks.
845851

plugins/research/skills/kaggle/SKILL.md

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
name: kaggle
3-
description: "Generate a Kaggle competition notebook as a Jupytext `# %%` Python script following the user's established ML research style: PTL for DNN training, best-fit tool selection, EDA→Baseline→Train→Inference pipeline with per-stage lens cells. Writes output to .experiments/kaggle/<name>.py."
3+
description: "Generate a Kaggle competition notebook as a Jupytext `# %%` Python script following the user's established ML research style: PTL for DNN training, best-fit tool selection, EDA→Baseline→Train→Inference pipeline with per-stage lens cells, small single-purpose cells each carrying a why. Tuned to win (leakage-safe CV, metric-aligned modeling) as much as to teach. Writes output to .experiments/kaggle/<name>.py."
44
argument-hint: "<competition-name> [<url-or-description>] [--type classification|regression|segmentation|detection|tabular] [--eda-only] [--inference-only] [--offline-setup] [--resume <existing.py>] [--keep \"<items>\"]"
55
allowed-tools: Read, Write, Edit, Bash, Grep, Glob, Agent, WebFetch, WebSearch, AskUserQuestion, TaskCreate, TaskUpdate, TaskList
66
disable-model-invocation: true
@@ -11,10 +11,17 @@ effort: high
1111

1212
Generate Kaggle competition notebook script, Jupytext `# %%` format.
1313

14+
Two goals, equal weight — neither traded for other:
15+
- **Win** — leaderboard-competitive: leakage-safe CV, metric-aligned loss/model choice, tuning/ensembling when it moves the score, not style theater
16+
- **Teach** — read top to bottom like a university/seminar lecture on solving this competition: reader new to it follows the full reasoning chain, every decision motivated, nothing left as unexplained code
17+
1418
Follows user's ML research style distilled from past notebooks:
1519
- **PTL always for DNN training** (PyTorch Lightning + torchmetrics) — even simple baselines
1620
- **Tool agnostic** — best-fit library for problem; PTL when training loop needed
1721
- **Stages with lenses** — each major stage: quick sanity check cell (show one batch, print shapes, verify submission format)
22+
- **Small, single-purpose cells** — one action per cell (load, one transform, one plot, one check); never bundle setup + run + verify to save cell count
23+
- **Every cell earns its place** — one-line why (comment or markdown sentence) before/in each cell: the specific reason this step happens now — never a restatement of what the code does
24+
- **Section markdown is extensive and structured** — full explanation of what/why/how-it-advances-the-goal per section, formatted as tables/lists/blockquotes over dense prose paragraphs; markdown before a plot sets up the question, markdown after states the finding and its implication — plot and prose flow as one beat, never an orphaned chart
1825
- **`# !` bash over subprocess** — package installs, `nvidia-smi`, `ls -lh`, `# ! head submission.csv`
1926
- **EDA is visual** — distribution plots, sample grids, dimension scatters before any model
2027
- **Inference included** — model save pattern + separate load-and-infer cells
@@ -273,6 +280,7 @@ After agent completes:
273280
1. Read first 30 lines of generated file to verify `# %%` structure
274281
2. Count cell markers: `grep -c "^# %%" .experiments/kaggle/<name>.py`
275282
3. Resolve the current row from `composition.md`; verify every listed section is present and no unlisted section was generated
283+
4. Mechanically check for bare `#` heading-spacer lines (style-rules.md rule 13) — prose compliance alone proved insufficient in practice; auto-fix rather than trust the generating pass
276284
277285
```bash
278286
# Re-derive OUTFILE from flags persisted in Step 1 (bash state lost between steps)
@@ -285,6 +293,23 @@ echo "=== Composition ==="; echo "$MODE"
285293
echo "=== Cell count ==="; grep -c "^# %%" "$OUTFILE" # timeout: 5000
286294
echo "=== Sections ==="; grep "^# %% \[markdown\]" "$OUTFILE" # timeout: 5000
287295
echo "=== File size ==="; wc -l "$OUTFILE" # timeout: 5000
296+
297+
echo "=== Bare '#' heading-spacer check (rule 13) ==="
298+
BARE_HASH_COUNT=$(grep -cE '^#+[[:space:]]*$' "$OUTFILE" 2>/dev/null || echo 0)
299+
echo "Found: $BARE_HASH_COUNT"
300+
if [ "$BARE_HASH_COUNT" -gt 0 ]; then
301+
grep -nE '^#+[[:space:]]*$' "$OUTFILE" # timeout: 5000
302+
python3 -c "
303+
import re
304+
path = '$OUTFILE'
305+
with open(path) as f:
306+
text = f.read()
307+
fixed = re.sub(r'(?m)^#+[ \t]*$', '', text)
308+
with open(path, 'w') as f:
309+
f.write(fixed)
310+
" # timeout: 5000
311+
echo "Auto-fixed: $BARE_HASH_COUNT bare '#' spacer line(s) converted to true blank lines"
312+
fi
288313
```
289314
290315
Print to terminal:
@@ -293,6 +318,7 @@ Print to terminal:
293318
- Problem type + recommended model
294319
- Cell count and section list
295320
- Missing required sections flagged with ``
321+
- Bare `#` heading-spacer count found/auto-fixed (`0` when clean)
296322
297323
Invoke `AskUserQuestion` as follow-up gate:
298324
- (a) Open in editor — `! code $OUTFILE`

plugins/research/skills/kaggle/modes/style-rules.md

Lines changed: 17 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,15 +2,20 @@
22

33
Apply ALL of these in generated script:
44

5-
1. **Every section `# %% [markdown]` header: explain what, why, how it advances goal**: after `## Section Name` write 2–4 sentences covering (a) what stage does, (b) why this approach chosen over alternatives, (c) how it contributes to competition objective (metric, leaderboard, submission quality). Notebook is public educational resource — write for reader seeing competition first time. Bare `## Title` headings with no explanation forbidden.
6-
2. **Shell commands — `# ! cmd` (Python comment)**: write every shell command as `# ! cmd`; valid Python syntax, visible as comment in Jupyter. `%matplotlib inline` verbatim — never `get_ipython().run_line_magic(...)`; linter rejects `%` magic → fix linter config
7-
3. `# ==============================` between logical blocks within cell (not every line — only major breaks)
8-
4. `_=` to suppress matplotlib/pandas return values: `_= df["col"].plot(...)`
9-
5. **Every plot: axis labels + grid + legend when multiple series**: always call `plt.xlabel("...")`, `plt.ylabel("...")`, `plt.grid(True)` after any plot; chart with multiple lines/bars/hues → add `plt.legend()` or pass `legend=True`; seaborn facets use `g.set_axis_labels("x label", "y label")`
10-
6. No `if __name__ == '__main__':` guards
11-
7. No argparse, no dataclasses for config
12-
8. **Markdown blank lines — empty lines only**: inside `# %% [markdown]` cells, use a truly empty line between paragraphs; never write bare `#` or `# ` lines because Kaggle renders them as headings
13-
9. **`display()` over `print()` for pandas objects**: use `display(df.head())`, `display(df.dtypes)`, `display(metrics.dropna(axis=1, how="all").head())`; use `print()` only for scalars and status strings
14-
10. **No doctests in ipy scripts**: doctests belong in package modules, not notebook scripts — `# %% [markdown]` cell above function cell IS explanation; don't duplicate as doctest
15-
11. **Compact docstrings — never omit**: always include one-line docstring; never omit — narrative lives in `# %% [markdown]` cell immediately above function cell; full Google-style docstrings with `Args:`, `Returns:`, `Example:` blocks apply only after distillation to `src/` utils package
16-
12. **No forward references in headers**: describe only what the cell contains now; keep future refactoring or package-distillation plans out of notebook headings
5+
1. **Every section `# %% [markdown]` header: extensive narrative, not caption**: after `## Section Name`, write a full explanation, not a 2-sentence blurb — cover (a) what this stage does, (b) why this specific approach over named alternatives (trade-offs stated), (c) how it advances the competition objective (metric, leaderboard placement, submission quality), (d) how it builds on the previous section's finding and sets up the next. Read the whole notebook top to bottom like a university/seminar lecture on solving this competition — reader new to it follows the full reasoning chain, not just code output. Applies to every heading level generated (`##`, `###`, `####`), not only top-level section openers: a subsection heading (e.g. `### Dataset overview`) still needs at least one full sentence directly beneath it before any list, table, blockquote, or code cell follows. Bare heading with nothing but a list/table/code underneath — no sentence at all — forbidden at any level.
6+
2. **Structured markdown over prose blocks**: whenever a section's content has more than one comparable item — options considered, config values, metrics, schema fields, decisions — render it as a bulleted/numbered list or a markdown table, not a wall of paragraph text. Plain unbroken paragraphs are boring and easy to skim past; structure (tables, lists, short bolded lead-ins, blockquote takeaways) is scannable and keeps the reader oriented. Prose paragraphs still carry the connecting narrative between structured blocks — this rule trims filler paragraphs, it does not replace narrative with bullet fragments. Separate stacked blocks (table→list, list→table) with a truly blank line only — never a bare `#` spacer line (rule 13): a lone `#` renders as an empty H1, not whitespace.
7+
3. **Markdown ↔ plot cells flow together**: the markdown cell immediately before a plotting cell states what the plot will show and the question it answers — never drop a chart on the reader cold. The markdown (or short comment) immediately after states the observed pattern and its design implication (what changes because of what was just seen). Plot → interpretation → decision is one continuous beat, never an orphaned chart with no before/after framing.
8+
4. **Small, single-purpose code cells**: one action per `# %%` cell — one load, one transform, one plot, one check, one train call. Never bundle load+display+validate, or setup+run+verify, into a single cell to save cell count; split immediately when a cell does more than one of those. Exception: a just-in-time config cell (constants only) may precede its action cell without being split further.
9+
5. **Every code cell carries a why, not a what**: one short line — inline `#` comment (procedural cells) or the preceding markdown sentence (section-opening cells) — states the *specific reason* this step happens here: a metric choice, a leakage risk avoided, a memory limit, a competition-specific quirk. Never restate what the code already shows (`# load the data` forbidden); no why → no cell.
10+
6. **Long comments move to a markdown cell, never sit in code**: a comment longer than one short why-line — a multi-line `#` block, or a paragraph explaining rationale/trade-offs — does not belong inside a code cell. Extract it into its own `# %% [markdown]` cell (using rule 2's structured formatting, not a wall of `#`-prefixed lines). If the long comment sits mid-cell **at top level, between statements**, split there: `code cell` (up to the comment) → `# %% [markdown]` cell (the extracted explanation) → `code cell` (continuing after it). **Not feasible mid-function or mid-class body**: a markdown cell cannot interrupt a `def`/`class` block — the code before it would be an incomplete, unparsable block on its own. When the long comment sits inside a function/class, either move the whole rationale into the markdown cell immediately *before* the `def`/`class` (function-level docstring covers the *what*; the preceding markdown covers the *why* at length) or compress it to a single why-line (rule 5) that stays inline — never split the function/class itself.
11+
7. **Shell commands — `# ! cmd` (Python comment)**: write every shell command as `# ! cmd`; valid Python syntax, visible as comment in Jupyter. `%matplotlib inline` verbatim — never `get_ipython().run_line_magic(...)`; linter rejects `%` magic → fix linter config
12+
8. `# ==============================` between logical blocks within cell (not every line — only major breaks)
13+
9. `_=` to suppress matplotlib/pandas return values: `_= df["col"].plot(...)`
14+
10. **Every plot: axis labels + grid + legend when multiple series**: always call `plt.xlabel("...")`, `plt.ylabel("...")`, `plt.grid(True)` after any plot; chart with multiple lines/bars/hues → add `plt.legend()` or pass `legend=True`; seaborn facets use `g.set_axis_labels("x label", "y label")`
15+
11. No `if __name__ == '__main__':` guards
16+
12. No argparse, no dataclasses for config
17+
13. **Markdown blank lines — empty lines only, never bare `#`**: inside `# %% [markdown]` cells, use a truly empty line to separate paragraphs, list, table, and blockquote blocks. Never write a bare `#` or `# ` line as a spacer — Jupyter/Kaggle renders it as an empty level-1 heading, not whitespace: source looks like a harmless blank line but output shows a stray heading plus its outsized margin, blowing a large empty gap into the rendered cell. Risk rises with rule 2's heavier list/table/blockquote stacking — check every inter-block gap is an actual empty line before finishing a markdown cell.
18+
14. **`display()` over `print()` for pandas objects**: use `display(df.head())`, `display(df.dtypes)`, `display(metrics.dropna(axis=1, how="all").head())`; use `print()` only for scalars and status strings
19+
15. **No doctests in ipy scripts**: doctests belong in package modules, not notebook scripts — `# %% [markdown]` cell above function cell IS explanation; don't duplicate as doctest
20+
16. **Compact docstrings — never omit**: always include one-line docstring; never omit — narrative lives in `# %% [markdown]` cell immediately above function cell; full Google-style docstrings with `Args:`, `Returns:`, `Example:` blocks apply only after distillation to `src/` utils package
21+
17. **No forward references in headers**: describe only what the cell contains now; keep future refactoring or package-distillation plans out of notebook headings

0 commit comments

Comments
 (0)