Relocate agent configs and workflow library under COPILOTJ_HOME

.gitignore

@@ -7,6 +7,7 @@ knowledge_bank/index/registry.jsonl
 cache/
 data/
 temp/
+workflows/
 uploads/
 config.json
 *.local

.plans/0005-knowledge-bank-upgrade-parity.md

@@ -0,0 +1,8 @@
+# Give knowledge_bank the same backup-on-upgrade treatment as agents
+
+- **What:** Apply the dpkg-style `.bak.YYYYMMDD` backup-on-upgrade logic that `agents/` now has (per-file baseline in `copilotj/multiagent/agent_loader.py::sync_agent_configs`) to `knowledge_bank/`, so user edits to shipped `macro/`/`research/` TOMLs survive plugin upgrades instead of being silently overwritten.
+- **Why:** `DefaultCopilotJBridgeService.extractPythonSources` re-extracts `knowledge_bank/*` into `$COPILOTJ_HOME` on every plugin upgrade — `jarFingerprint` (JAR path + `lastModified`) changes on every release, `resolveCopilotJSource` triggers re-extraction on the mismatch, and `Files.copy(..., REPLACE_EXISTING)` (`DefaultCopilotJBridgeService.java:614`) overwrites shipped files with no "was this user-modified?" check. Only user-*added* filenames survive; edits to shipped files are lost. After the agents relocation, `agents/` has a safer upgrade story than `knowledge_bank` — an inconsistency, and a latent user-data-loss path in the more user-facing store.
+- **Pros:** Eliminates silent overwriting of user KB edits on upgrade; unifies the two user-editable TOML stores under one upgrade model.
+- **Cons:** The cleanest fix lives in the Java extraction layer (backup-before-overwrite), or requires a Python kb sync that must coordinate with Java's extraction — either way crosses the Python/Java boundary and is its own design + test effort (roughly doubling this PR's scope if bundled).
+- **Context:** Surfaced in the `/plan-eng-review` for the agents/workflow relocation (2026-07-02); deferred (TODO proposal 2 → "Add to TODOS"). The agents reference implementation is in `copilotj/multiagent/agent_loader.py` (`sync_agent_configs`, `_load_seed_baseline`, `_save_seed_baseline`, `_seed_files`). knowledge_bank bootstrap lives in `copilotj/multiagent/kb_tools.py` (`_bootstrap_kb`, now using `bootstrap_dir_if_empty`); extraction in `plugin/src/main/java/copilotj/DefaultCopilotJBridgeService.java` (`extractPythonSources`, `resolveCopilotJSource`).
+- **Depends on / blocked by:** Nothing — independent follow-up.

.plans/0006-agents-sync-cross-process-lock.md

@@ -0,0 +1,8 @@
+# Add cross-process mutual exclusion to sync_agent_configs
+
+- **What:** Add a file lock (e.g. `fcntl.flock`) and an atomic baseline swap (`os.replace` from a temp name) around the refresh + baseline-write critical section in `copilotj/multiagent/agent_loader.py::sync_agent_configs`.
+- **Why:** The seed-baseline read → per-file backup/copy → baseline-write sequence is not atomic, and the module-level `_synced` flag only guards within one process. Two processes sharing one `$COPILOTJ_HOME` (not the default single-bridge deployment, but plausible under multi-process or test setups) could both refresh concurrently. The backup logic itself is data-safe under concurrent syncers (the collision-avoiding `.bak.YYYYMMDD(.N)` naming + `filecmp` short-circuit preserve every distinct state), so this is hardening, not a known data-loss path.
+- **Pros:** Eliminates the one identified concurrency fragility in the agents refresh; makes multi-instance `$COPILOTJ_HOME` safe if ever introduced.
+- **Cons:** Adds locking complexity for a scenario (multi-process sharing) that the single-bridge deployment doesn't hit; risk of deadlocks if not careful, and Windows `fcntl` portability needs a fallback.
+- **Context:** Surfaced by the `/review` adversarial pass (Claude F1, 2026-07-02); deferred as hardening. Single bridge server per `$COPILOTJ_HOME` is the supported deployment, so real-world concurrency is unlikely. Reference: `sync_agent_configs`, `_save_seed_baseline` in `copilotj/multiagent/agent_loader.py`.
+- **Depends on / blocked by:** Nothing.

AGENTS.md

@@ -30,8 +30,8 @@ The core agent framework follows a layered architecture:
 - `LeaderDriven` (Pattern) — Main multi-agent pattern. Creates a `LeaderAgent` and loads specialized `Executor` agents from TOML configs.
 - `LeaderAgent` (ChatAgent) — ReAct-style leader that reasons, calls tools (ImageJ perception, macro execution, Python scripts, knowledge bank), and delegates to specialized agents. Manages dialog-level conversation history with summarization.
 - `Executor` (ChatAgent) — Generic specialized agent with its own system prompt and tools, loaded from TOML. Runs ReAct loops with tool retry and error recovery.
-- `agent_configs/` — TOML files defining specialized agents. Active configs: `tool_agent.toml`, `research_agent.toml`, `success_case.toml`, `imagej_macro_help.toml`. Disabled configs have `.disabled.toml` suffix.
-- `agent_loader.py` — Dynamically loads agent classes and tool functions from TOML configs using `importlib`.
+- `agents/` — Repo-root dir of TOML seed files defining specialized agents (sibling of `knowledge_bank/`). At runtime the loader reads from `$COPILOTJ_HOME/agents/` (synced from the seed; see `sync_agent_configs`). Active configs: `tool_agent.toml`, `research_agent.toml`. Disabled templates use the `.disabled.toml` suffix; only `*_agent.toml` files are loaded.
+- `agent_loader.py` — Dynamically loads agent classes and tool functions from TOML configs using `importlib`. Seeds/syncs `$COPILOTJ_HOME/agents/` dpkg-style (customized files are backed up as `<name>.bak.YYYYMMDD` on upgrade).
 
 **`copilotj/server/`** — HTTP layer:
 

DEVELOPMENT.md

@@ -274,15 +274,24 @@ COPILOTJ_KB_AUTOSAVE=0
 
 ### Agent configuration (advanced)
 
-CopilotJ uses a configurable multi-agent architecture. Agent configuration files are located in
-`copilotj/multiagent/agent_configs/`. Each configuration file defines an agent's system prompt, role description,
-and optional constraints.
-
-**Customizing existing agents:** Modify prompt files in `agent_configs/` to adjust reasoning style, constrain
-responsibilities, or tune domain-specific behavior. Changes take effect after restarting the server.
-
-**Adding new agents:** Copy an existing configuration file, define a unique agent name and role, write a system prompt,
-and register any custom tools.
+CopilotJ uses a configurable multi-agent architecture. The shipped defaults live in the repo-root
+`agents/` directory (sibling of `knowledge_bank/`). At runtime the active configs are read from
+`$COPILOTJ_HOME/agents/`. In dev mode (`COPILOTJ_HOME` unset → repo root) the two paths coincide,
+so editing `agents/*.toml` directly takes effect after restarting the server. In production the
+defaults are synced into `$COPILOTJ_HOME/agents/` from the bundled seed.
+
+**Customizing existing agents:** Edit the TOML in `$COPILOTJ_HOME/agents/` to adjust reasoning
+style, constrain responsibilities, or tune domain-specific behavior. Changes take effect after
+restarting the server.
+
+**Adding new agents:** Drop a new `*_agent.toml` file into `$COPILOTJ_HOME/agents/` (a unique
+agent name, role, system prompt, and any custom tools). User-added configs are never touched by the
+sync logic.
+
+**Upgrades:** When a release ships changed default configs, each file you have customized is first
+backed up as `<name>.bak.YYYYMMDD` inside `$COPILOTJ_HOME/agents/`, then overwritten with the
+new default (so your edits are recoverable, not lost). Re-apply your edits from the `.bak` if needed.
+Do not modify or delete those `.bak` files by hand.
 
 ## Testing
 

copilotj/core/config.py

@@ -26,6 +26,7 @@
     "is_single_client",
     "load_managed_config",
     "save_managed_config",
+    "bootstrap_dir_if_empty",
     "bootstrap_assets",
     "resolve_vision_config",
 ]
@@ -237,19 +238,35 @@ def save_managed_config(data: dict) -> None:
     path.write_text(json.dumps(data, indent=2), "utf-8")
 
 
+def bootstrap_dir_if_empty(src: Path, dst: Path) -> bool:
+    """Copy a seed directory tree into ``dst`` only when ``dst`` is missing or empty.
+
+    Used to seed user-data dirs (``assets`` and ``knowledge_bank``) from the bundled
+    source on first run. Note: the ``agents`` dir does NOT use this — it is user-editable
+    and uses the richer dpkg-style refresh in ``copilotj.multiagent.agent_loader``.
+    No-op (returns False) when:
+
+    - ``src`` does not exist (no seed available),
+    - ``src`` and ``dst`` resolve to the same path (dev mode, where the home dir
+      IS the source tree, so copying would recurse into itself),
+    - ``dst`` already exists and is non-empty (user data present; never clobber).
+
+    Otherwise copies ``src`` onto ``dst`` (``dirs_exist_ok=True``) and returns True.
+    """
+    if not src.exists():
+        return False
+    if src.resolve() == dst.resolve():
+        return False
+    if dst.exists() and any(dst.iterdir()):
+        return False
+    shutil.copytree(src, dst, dirs_exist_ok=True)
+    return True
+
+
 def bootstrap_assets() -> None:
     """Copy assets/ from project source to COPILOTJ_HOME if missing."""
-    home = get_home()
     source_root = Path(__file__).resolve().parent.parent.parent
-    source_assets = source_root / "assets"
-    target_assets = home / "assets"
-
-    if not source_assets.exists():
-        return
-    if target_assets.exists() and any(target_assets.iterdir()):
-        return
-
-    shutil.copytree(source_assets, target_assets, dirs_exist_ok=True)
+    bootstrap_dir_if_empty(source_root / "assets", get_home() / "assets")
 
 
 def _is_dev(cfg: Config) -> bool: