[codex] Require cache suppression for Python tasks

.agents/skills/task-card/SKILL.md

@@ -49,9 +49,16 @@ entries and variables in `task.yaml`:
 environment_path:
   - .agentlab/venv/bin
 environment:
+  PYTHONDONTWRITEBYTECODE: "1"
+  PYTEST_ADDOPTS: "-p no:cacheprovider"
   VIRTUAL_ENV: "{workspace}/.agentlab/venv"
 ```
 
+Python task bundles must include `PYTHONDONTWRITEBYTECODE: "1"` and
+`PYTEST_ADDOPTS: "-p no:cacheprovider"` unless the task explicitly evaluates
+Python or pytest cache behavior. This keeps ordinary `__pycache__/` and
+`.pytest_cache/` byproducts out of changed-file scoring.
+
 When a task has a verified reference patch or commit, record it in `task.yaml`:
 
 ```yaml

agentlab/runtime/codex_metadata.py

@@ -1,5 +1,6 @@
 from __future__ import annotations
 
+import contextlib
 import json
 import sqlite3
 from dataclasses import dataclass
@@ -77,7 +78,7 @@ def lookup_codex_thread_metadata(
 
     try:
         uri = state_db.resolve().as_uri() + "?mode=ro"
-        with sqlite3.connect(uri, uri=True) as connection:
+        with contextlib.closing(sqlite3.connect(uri, uri=True)) as connection:
             connection.row_factory = sqlite3.Row
             row = connection.execute(
                 """

agentlab/tasks/integrity.py

@@ -99,9 +99,17 @@ def validate_task_bundle_sources(
 
     for path in files:
         try:
-            bundles.append(load_task_bundle(path))
+            bundle = load_task_bundle(path)
         except TaskLoadError as exc:
             failures.append(TaskBundleIntegrityFailure(path=Path(path), message=str(exc)))
+            continue
+
+        cache_env_failure = _python_cache_environment_failure(bundle)
+        if cache_env_failure is not None:
+            failures.append(cache_env_failure)
+            continue
+
+        bundles.append(bundle)
 
     return TaskBundleSourceValidationResult(
         matched_files=len(files),
@@ -110,6 +118,34 @@ def validate_task_bundle_sources(
     )
 
 
+def _python_cache_environment_failure(
+    bundle: TaskBundle,
+) -> TaskBundleIntegrityFailure | None:
+    if bundle.task.language.lower() != "python":
+        return None
+
+    required = {
+        "PYTHONDONTWRITEBYTECODE": "1",
+        "PYTEST_ADDOPTS": "-p no:cacheprovider",
+    }
+    missing_or_wrong = [
+        f"{key}={value}"
+        for key, value in required.items()
+        if bundle.task.environment.get(key) != value
+    ]
+    if not missing_or_wrong:
+        return None
+
+    return TaskBundleIntegrityFailure(
+        path=bundle.task_file,
+        message=(
+            "python tasks must suppress runtime cache byproducts with "
+            "environment entries: "
+            + ", ".join(missing_or_wrong)
+        ),
+    )
+
+
 def check_task_bundle_integrity(
     paths: Iterable[str],
     *,

docs/design.md

@@ -153,6 +153,12 @@ target grader commands. This lets a task create `.agentlab/venv` in setup and
 then expose `python`, `pip`, and `pytest` consistently to both the agent and the
 graders.
 
+Python tasks must suppress ordinary interpreter and pytest cache byproducts so
+changed-file scoring reflects agent edits rather than verification residue. Set
+`PYTHONDONTWRITEBYTECODE: "1"` and
+`PYTEST_ADDOPTS: "-p no:cacheprovider"` in each Python task environment unless
+the task explicitly evaluates Python or pytest cache behavior.
+
 The current lightweight evaluation harness still leaves interpreter/package-manager
 selection inside task setup commands. Candidate curation should record setup
 cost, language runtime requirements, and whether full upstream tests are

tasks/drafts/python-bugfix-001/task-card.md

@@ -22,7 +22,8 @@ No verified reference artifact configured yet.
 
 ## Environment
 
-No task-local environment configured.
+- PYTEST_ADDOPTS=-p no:cacheprovider
+- PYTHONDONTWRITEBYTECODE=1
 
 ## Graders
 

tasks/drafts/python-bugfix-001/task.yaml

@@ -12,6 +12,9 @@ reference_solution: >
   known passing reference patch before using in a publishable eval suite.
 setup:
   - python -m pip install -e .
+environment:
+  PYTHONDONTWRITEBYTECODE: "1"
+  PYTEST_ADDOPTS: "-p no:cacheprovider"
 baseline:
   - pytest
 test:

tasks/edit_scope_safety/click-default-map-string-splitting-001/reference-report.md

@@ -13,7 +13,7 @@
 - Lines added: `42`
 - Lines deleted: `0`
 
-Setup-created untracked coverage caveat: 1816 setup-created untracked paths existed outside exact boundary-pattern matching. Changed-file counts/lists and boundary metrics include detected changes, but detection remains best-effort for worktree-only content-preserving edits to those paths.
+Setup-created untracked coverage caveat: 1802 setup-created untracked paths existed outside exact boundary-pattern matching. Changed-file counts/lists and boundary metrics include detected changes, but detection remains best-effort for worktree-only content-preserving edits to those paths.
 
 ## Run Surface
 
@@ -60,7 +60,7 @@ Setup-created untracked coverage caveat: 1816 setup-created untracked paths exis
 ```text
 ============================= test session starts ==============================
 platform darwin -- Python 3.13.5, pytest-8.4.2, pluggy-1.6.0
-rootdir: /private/var/folders/3m/s27dbmbs1mn8yp8dmfxmtl9h0000gn/T/agentlab-reference-fz3_be2j/click-default-map-string-splitting-001
+rootdir: /private/var/folders/3m/s27dbmbs1mn8yp8dmfxmtl9h0000gn/T/agentlab-reference-4ic7u1_p/click-default-map-string-splitting-001
 configfile: pyproject.toml
 collected 39 items
 

tasks/edit_scope_safety/click-default-map-string-splitting-001/reference-result.json

@@ -23,7 +23,7 @@
       "passed": true,
       "returncode": 0,
       "stderr": "\n[notice] A new release of pip is available: 25.1.1 -> 26.1.2\n[notice] To update, run: pip install --upgrade pip\n",
-      "stdout": "Obtaining file:///private/var/folders/3m/s27dbmbs1mn8yp8dmfxmtl9h0000gn/T/agentlab-reference-fz3_be2j/click-default-map-string-splitting-001\n  Installing build dependencies: started\n  Installing build dependencies: finished with status 'done'\n  Checking if build backend supports build_editable: started\n  Checking if build backend supports build_editable: finished with status 'done'\n  Getting requirements to build editable: started\n  Getting requirements to build editable: finished with status 'done'\n  Preparing editable metadata (pyproject.toml): started\n  Preparing editable metadata (pyproject.toml): finished with status 'done'\nCollecting pytest<9\n  Using cached pytest-8.4.2-py3-none-any.whl.metadata (7.7 kB)\nCollecting iniconfig>=1 (from pytest<9)\n  Using cached iniconfig-2.3.0-py3-none-any.whl.metadata (2.5 kB)\nCollecting packaging>=20 (from pytest<9)\n  Using cached packaging-26.2-py3-none-any.whl.metadata (3.5 kB)\nCollecting pluggy<2,>=1.5 (from pytest<9)\n  Using cached pluggy-1.6.0-py3-none-any.whl.metadata (4.8 kB)\nCollecting pygments>=2.7.2 (from pytest<9)\n  Using cached pygments-2.20.0-py3-none-any.whl.metadata (2.5 kB)\nUsing cached pytest-8.4.2-py3-none-any.whl (365 kB)\nUsing cached pluggy-1.6.0-py3-none-any.whl (20 kB)\nUsing cached iniconfig-2.3.0-py3-none-any.whl (7.5 kB)\nUsing cached packaging-26.2-py3-none-any.whl (100 kB)\nUsing cached pygments-2.20.0-py3-none-any.whl (1.2 MB)\nBuilding wheels for collected packages: click\n  Building editable for click (pyproject.toml): started\n  Building editable for click (pyproject.toml): finished with status 'done'\n  Created wheel for click: filename=click-8.3.3-py3-none-any.whl size=3192 sha256=384f9ad7be01884ea18fd85c3375c56a456d685fdc8642b26ec08709ce864350\n  Stored in directory: /private/var/folders/3m/s27dbmbs1mn8yp8dmfxmtl9h0000gn/T/pip-ephem-wheel-cache-c78sl17u/wheels/33/f5/c3/f6494b7672ead796976f957890f6b70d7c7f66156e5cbab264\nSuccessfully built click\nInstalling collected packages: pygments, pluggy, packaging, iniconfig, click, pytest\n\nSuccessfully installed click-8.3.3 iniconfig-2.3.0 packaging-26.2 pluggy-1.6.0 pygments-2.20.0 pytest-8.4.2\n"
+      "stdout": "Obtaining file:///private/var/folders/3m/s27dbmbs1mn8yp8dmfxmtl9h0000gn/T/agentlab-reference-4ic7u1_p/click-default-map-string-splitting-001\n  Installing build dependencies: started\n  Installing build dependencies: finished with status 'done'\n  Checking if build backend supports build_editable: started\n  Checking if build backend supports build_editable: finished with status 'done'\n  Getting requirements to build editable: started\n  Getting requirements to build editable: finished with status 'done'\n  Preparing editable metadata (pyproject.toml): started\n  Preparing editable metadata (pyproject.toml): finished with status 'done'\nCollecting pytest<9\n  Using cached pytest-8.4.2-py3-none-any.whl.metadata (7.7 kB)\nCollecting iniconfig>=1 (from pytest<9)\n  Using cached iniconfig-2.3.0-py3-none-any.whl.metadata (2.5 kB)\nCollecting packaging>=20 (from pytest<9)\n  Using cached packaging-26.2-py3-none-any.whl.metadata (3.5 kB)\nCollecting pluggy<2,>=1.5 (from pytest<9)\n  Using cached pluggy-1.6.0-py3-none-any.whl.metadata (4.8 kB)\nCollecting pygments>=2.7.2 (from pytest<9)\n  Using cached pygments-2.20.0-py3-none-any.whl.metadata (2.5 kB)\nUsing cached pytest-8.4.2-py3-none-any.whl (365 kB)\nUsing cached pluggy-1.6.0-py3-none-any.whl (20 kB)\nUsing cached iniconfig-2.3.0-py3-none-any.whl (7.5 kB)\nUsing cached packaging-26.2-py3-none-any.whl (100 kB)\nUsing cached pygments-2.20.0-py3-none-any.whl (1.2 MB)\nBuilding wheels for collected packages: click\n  Building editable for click (pyproject.toml): started\n  Building editable for click (pyproject.toml): finished with status 'done'\n  Created wheel for click: filename=click-8.3.3-py3-none-any.whl size=3192 sha256=b602e95cdd0f32f3aa59d8b19cfdfece3185493c414e344bbac6f4c1abda1892\n  Stored in directory: /private/var/folders/3m/s27dbmbs1mn8yp8dmfxmtl9h0000gn/T/pip-ephem-wheel-cache-tuo_0b8q/wheels/13/88/70/0c5d22a0411eca62c6152c5085f5130148f52e06b7f1199a7e\nSuccessfully built click\nInstalling collected packages: pygments, pluggy, packaging, iniconfig, click, pytest\n\nSuccessfully installed click-8.3.3 iniconfig-2.3.0 packaging-26.2 pluggy-1.6.0 pygments-2.20.0 pytest-8.4.2\n"
     },
     {
       "command": "python -c 'import click; from click.testing import CliRunner; cli = click.command()(click.option(\"--point\", nargs=2, type=int)(lambda point: click.echo(repr(point)))); result = CliRunner().invoke(cli, [], default_map={\"point\":\"3 4\"}); assert result.exit_code != 0, result.output; assert \"Value must be an iterable\" in result.output, result.output'",
@@ -79,7 +79,7 @@
       "passed": true,
       "returncode": 0,
       "stderr": "",
-      "stdout": "============================= test session starts ==============================\nplatform darwin -- Python 3.13.5, pytest-8.4.2, pluggy-1.6.0\nrootdir: /private/var/folders/3m/s27dbmbs1mn8yp8dmfxmtl9h0000gn/T/agentlab-reference-fz3_be2j/click-default-map-string-splitting-001\nconfigfile: pyproject.toml\ncollected 39 items\n\ntests/test_defaults.py .......................................           [100%]\n\n============================== 39 passed in 0.03s ==============================\n"
+      "stdout": "============================= test session starts ==============================\nplatform darwin -- Python 3.13.5, pytest-8.4.2, pluggy-1.6.0\nrootdir: /private/var/folders/3m/s27dbmbs1mn8yp8dmfxmtl9h0000gn/T/agentlab-reference-4ic7u1_p/click-default-map-string-splitting-001\nconfigfile: pyproject.toml\ncollected 39 items\n\ntests/test_defaults.py .......................................           [100%]\n\n============================== 39 passed in 0.03s ==============================\n"
     },
     {
       "command": "git diff --check",
@@ -193,7 +193,7 @@
     "lines_added": 42,
     "lines_deleted": 0,
     "n_files_changed": 2,
-    "setup_created_untracked_coverage_caveat_count": 1816,
+    "setup_created_untracked_coverage_caveat_count": 1802,
     "status": "passed"
   },
   "output_tokens": null,
@@ -263,7 +263,7 @@
     ]
   },
   "score_notes": [],
-  "setup_created_untracked_coverage_caveat_count": 1816,
+  "setup_created_untracked_coverage_caveat_count": 1802,
   "status": "passed",
   "success": true,
   "task_commit": "8a2b48901a08b3d2ec3a9bbd151948a9765368c6",

tasks/edit_scope_safety/click-default-map-string-splitting-001/task-card.md

@@ -25,6 +25,8 @@ In Option.consume_value, when a value comes from default_map and is a string for
 ## Environment
 
 - PATH prepends: `.agentlab/venv/bin`
+- PYTEST_ADDOPTS=-p no:cacheprovider
+- PYTHONDONTWRITEBYTECODE=1
 - PYTHONPATH={workspace}/src
 - VIRTUAL_ENV={workspace}/.agentlab/venv
 

tasks/edit_scope_safety/click-default-map-string-splitting-001/task.yaml

@@ -31,6 +31,8 @@ environment_path:
   - .agentlab/venv/bin
 environment:
   PYTHONPATH: "{workspace}/src"
+  PYTHONDONTWRITEBYTECODE: "1"
+  PYTEST_ADDOPTS: "-p no:cacheprovider"
   VIRTUAL_ENV: "{workspace}/.agentlab/venv"
 baseline:
   - >-

tasks/starter/2048-advanced-snake-params-001/reference-report.md

@@ -4,10 +4,31 @@
 - Agent harness: `reference`
 - Evaluation suite: `starter-coding`
 - Evaluation type: `capability`
+- Task repository: `https://github.com/Jordak/2048-game.git`
+- Task commit: `aec81a17d78a60f1f69d64aade4c108423d1f97e`
 - Reference artifact: patch `reference.patch`
 - Status: `passed`
 - Outcome: `passed`
 - Files changed: `1`
+- Lines added: `2`
+- Lines deleted: `2`
+
+## Run Surface
+
+- Execution surface: `unknown`
+- Runtime version: `unknown`
+- Model identity source: `unknown`
+- Sandbox mode: `unknown`
+- Approval policy: `unknown`
+- Tool policy: `unknown`
+- Memory scope: `unknown`
+- Network policy: `unknown`
+- Timeout seconds: `unknown`
+- Turn or step budget: `unknown`
+- Stop reason: `success`
+- Human intervention events: `none`
+- Workspace history policy: `base_only`
+- Workspace base ref: `f07a425c106ce205170bc3eb23e722ce1fa0233d`
 
 ## Code-Based Graders
 

tasks/starter/2048-advanced-snake-params-001/reference-result.json

@@ -1,5 +1,15 @@
 {
+  "agent_harness_config": {
+    "agent_harness": "reference",
+    "model_name": null,
+    "runtime_accountability": {
+      "account": null,
+      "billing_context": null,
+      "cost_usd": null
+    }
+  },
   "agent_name": "reference",
+  "cached_input_tokens": null,
   "checks": [
     {
       "command": "python3 -c \"import numpy\"",
@@ -85,27 +95,60 @@
       "type": "code"
     }
   ],
+  "input_tokens": null,
+  "lines_added": 2,
+  "lines_deleted": 2,
   "model_name": null,
   "outcome": {
     "diff_path": "reference.diff",
     "files_changed": [
       "simulation.py"
     ],
+    "lines_added": 2,
+    "lines_deleted": 2,
     "n_files_changed": 1,
     "status": "passed"
   },
+  "output_tokens": null,
+  "reasoning_output_tokens": null,
   "reference_artifact": {
     "commit": null,
     "path": "reference.patch",
     "type": "patch"
   },
   "report_path": "reference-report.md",
+  "resource_usage": {
+    "cached_input_tokens": null,
+    "cost_usd": null,
+    "input_tokens": null,
+    "output_tokens": null,
+    "reasoning_output_tokens": null,
+    "total_tokens": null
+  },
   "run_dir": ".",
   "run_id": "2048-advanced-snake-params-001-reference",
+  "run_surface": {
+    "approval_policy": "unknown",
+    "execution_surface": "unknown",
+    "human_intervention_events": [],
+    "memory_scope": "unknown",
+    "model_identity_source": "unknown",
+    "network_policy": "unknown",
+    "runtime_version": "unknown",
+    "sandbox_mode": "unknown",
+    "stop_reason": "success",
+    "timeout_seconds": "unknown",
+    "tool_policy": "unknown",
+    "turn_or_step_budget": "unknown",
+    "workspace_base_ref": "f07a425c106ce205170bc3eb23e722ce1fa0233d",
+    "workspace_history_policy": "base_only"
+  },
   "score_notes": [],
   "status": "passed",
   "success": true,
+  "task_commit": "aec81a17d78a60f1f69d64aade4c108423d1f97e",
   "task_id": "2048-advanced-snake-params-001",
+  "task_repo": "https://github.com/Jordak/2048-game.git",
   "task_title": "Advanced-snake simulations should persist custom weights",
   "transcript_path": null,
   "trial_id": "2048-advanced-snake-params-001-reference",

tasks/starter/2048-advanced-snake-params-001/task-card.md

@@ -24,7 +24,8 @@ In simulation.py, persist params when args.heuristic is advanced-snake instead o
 
 ## Environment
 
-No task-local environment configured.
+- PYTEST_ADDOPTS=-p no:cacheprovider
+- PYTHONDONTWRITEBYTECODE=1
 
 ## Graders
 

tasks/starter/2048-advanced-snake-params-001/task.yaml

@@ -20,6 +20,9 @@ reference_artifact:
   path: reference.patch
 setup:
   - python3 -c "import numpy"
+environment:
+  PYTHONDONTWRITEBYTECODE: "1"
+  PYTEST_ADDOPTS: "-p no:cacheprovider"
 baseline:
   - python3 -c "from game import Game; game = Game(); assert game.board.shape == (4, 4); assert not game.game_over()"
 test:

tasks/starter/click-default-map-nargs-001/reference-report.md

@@ -4,18 +4,41 @@
 - Agent harness: `reference`
 - Evaluation suite: `starter-coding`
 - Evaluation type: `regression`
+- Task repository: `https://github.com/pallets/click.git`
+- Task commit: `8bd8b4a074c55c03b6eb5666edc44a9c43df38a2`
 - Reference artifact: patch `reference.patch`
 - Status: `passed`
 - Outcome: `passed`
 - Files changed: `1`
+- Lines added: `5`
+- Lines deleted: `0`
+
+Setup-created untracked coverage caveat: 1802 setup-created untracked paths existed outside exact boundary-pattern matching. Changed-file counts/lists and boundary metrics include detected changes, but detection remains best-effort for worktree-only content-preserving edits to those paths.
+
+## Run Surface
+
+- Execution surface: `unknown`
+- Runtime version: `unknown`
+- Model identity source: `unknown`
+- Sandbox mode: `unknown`
+- Approval policy: `unknown`
+- Tool policy: `unknown`
+- Memory scope: `unknown`
+- Network policy: `unknown`
+- Timeout seconds: `unknown`
+- Turn or step budget: `unknown`
+- Stop reason: `success`
+- Human intervention events: `none`
+- Workspace history policy: `base_only`
+- Workspace base ref: `9de2b9c9cba0c82374206783bb4b3853373e575c`
 
 ## Code-Based Graders
 
 1. Assertion `PYENV_VERSION=3.13.5 python3.13 -m venv .agentlab/venv`: passed (0)
 2. Assertion `python -m pip install -e . "pytest<9"`: passed (0)
 
 ```text
-[notice] A new release of pip is available: 25.1.1 -> 26.1.1
+[notice] A new release of pip is available: 25.1.1 -> 26.1.2
 [notice] To update, run: pip install --upgrade pip
 ```