Skip to content

Repository files navigation

Abstract: The ability to selectively remove the influence of specific training experiences from a learned model has become increasingly important in deployed machine learning systems due to concerns related to data privacy, corrupted data, and regulatory compliance. In reinforcement learning, this problem is more challenging than in supervised settings because training data consist of temporally correlated trajectories whose effects propagate through iterative policy and value updates. In multi-agent reinforcement learning (MARL), the challenge is further amplified by the shared, non-stationary environment, where the behavior of one agent influences the learning dynamics of others, complicating targeted data removal. Existing approaches typically rely on full retraining after discarding undesired data, which is computationally expensive and impractical for large-scale or time-sensitive applications. In this work, we investigate selective unlearning within the Multi-Agent Deep Deterministic Policy Gradient (MADDPG) framework, a centralized-training decentralized-execution algorithm for cooperative continuous-action tasks. We partition replay experience into a forget set $\mathcal{D}_f$ and a retain set $\mathcal{D}_r$, and propose three unlearning variants: critic-only, actor-only, and combined actor-critic unlearning. Each variant is followed by a short recovery phase using only $\mathcal{D}_r$ to restore performance stability. Experiments on a cooperative multi-agent coverage task across five random seeds, with statistical validation via paired $t$-tests, compare the proposed methods against full retain-only retraining. Results show that selective unlearning achieves utility retention comparable to full retraining while providing a computational speedup of approximately $352\times$. Among the variants, actor-only unlearning demonstrates the most stable forgetting behavior, with a consistent forget drop of $82.56\pm1.51$ and minimal parameter perturbation. These findings suggest that targeted actor updates provide an efficient and stable approach for selective unlearning in cooperative MARL systems without requiring full retraining.

About

selective unlearning in MADDPG

Resources

Stars

3 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages