Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Data and configs
*.toml
#*.toml
examples/slac_fel/data/
# DS_Store files
.DS_Store
.idea

double_check_data.ipynb
uv.lock

# Logging files
*.db
Expand Down
10 changes: 10 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,16 @@ Note: `EnsembleDetector` is recognized by the loader but raises `NotImplementedE
| `kfac_online` | KFAC approximation | kfac_lambda, kfac_ema_decay |
| `none` | No-op (skip CL) | (none) |

Replay of historical data is handled in `BaseUpdater.fwd_bwd()` and gated by
`[continual_learning] mix_historic_data` (default `false`). When enabled and the
harness supplies historical dataloaders, half of each stream is combined into a
single forward/backward pass, so the sample count per step stays at
`train.batch_size` whether or not mixing is on (an undersized historical batch is
topped up from the current one). `base`, `ewc_online`, and `kfac_online` inherit this. `jvp_reg` ignores the flag: it
overrides `fwd_bwd()` and mixes the two streams itself (it needs a current-only
gradient as the JVP tangent direction), so it calls `super().fwd_bwd(batch)` without
the historical batch. `none` skips training entirely.

### Coding Conventions
- Python 3.13+, type hints everywhere
- Formatting: ruff format, ruff check, mypy
Expand Down
20 changes: 14 additions & 6 deletions examples/mnist/model.py
Original file line number Diff line number Diff line change
Expand Up @@ -142,21 +142,29 @@ def get_hist_dataloaders(
self,
) -> Tuple[Optional[DataLoader], Optional[DataLoader]]:
"""
If no history yet: add current drift to history and return (None, None).
Else: return loaders over ConcatDataset of prior drifts, then append current drift to history.
Effective dataset length = len(aug_history) * len(full_split).
If no history yet: return (None, None).
Else: return loaders over a replay buffer spanning EVERY prior cumulative
regime (0 … k-1), not just the immediately-previous one. Each past regime
`i` is the composition ``FixedAffine(aug_history[:i])``, matching the data
that was actually streamed at window i-1. Concatenating all of them keeps
the model anchored to the earliest regimes so CL adaptation does not
specialise to the most recent drift alone.
Effective dataset length = (len(aug_history) - 1) * len(full_split).
"""
if self.task_counter == 1:
return None, None

# Concatenate FULL train/val views for each historical drift
# One FULL train/val view per prior cumulative regime: regimes 0 … k-1.
# aug_history[-1] is the current regime, so we stop before it.
train_views = [
TransformedView(
self.ds_train, x_transform=FixedAffine(self.aug_history[:-1])
self.ds_train, x_transform=FixedAffine(self.aug_history[:i])
)
for i in range(1, len(self.aug_history))
]
val_views = [
TransformedView(self.ds_val, x_transform=FixedAffine(self.aug_history[:-1]))
TransformedView(self.ds_val, x_transform=FixedAffine(self.aug_history[:i]))
for i in range(1, len(self.aug_history))
]

ds_hist_train: ConcatDataset[Any] = ConcatDataset(train_views)
Expand Down
Empty file added examples/slac_fel/__init__.py
Empty file.
Loading
Loading