Project page · v0.5.3 · CI · MIT
MATLAB reference implementations and reproducible experiments for adaptive dynamic programming control.
中文说明 · Quick start · Methods · Validation · Roadmap
Learn from a small integral policy-iteration example, then explore Koopman models, neural HJB solvers, mean-field control and recent control-journal methods. Each implementation connects its equations, MATLAB functions, assumptions and recorded outcomes.
v0.5.3: 8 runnable entries · 7 paper packages · 107/107 local tests passed · 6 research skills. Tested on 25.2.0.3312555 (R2025b) Update 6 (MACA64).
The figure is generated from an actual run of the double-integrator example. Its CSV data and settings accompany the source.
Clone this repository or extract a source ZIP, then open its root folder in MATLAB. The baseline needs base MATLAB with the standard JVM.
git clone https://github.com/tanjunkai2001/adp-matlab.git
cd adp-matlabcheck_environment();
demo_reproductions();
[result, runDir] = demo_reproductions('baseline');
result.learning.K % Reference: [1, sqrt(3)]
run_tests;To inspect or execute all local tests:
run_all_tests('list');
results = run_all_tests;The full suite uses Control System Toolbox and Deep Learning Toolbox. One PINN replay test performs two single Adam updates; Safe PINN tests include zero-update evaluation and three controlled failures on the second attempted training update. Complete neural training is invoked separately. See installation and examples for the full workflow.
| Method / source | Model & learning | Tests | Implemented scope |
|---|---|---|---|
| Integral policy iteration · Reference baseline | CT linear quadratic · known B | 46 | Analytic LQR comparator; two-state example. |
| Koopman generator + PI · L4DC 2025 | CT nonlinear · identified generator | 8 | Normalized pendulum variant; not every paper table. |
| Mean-field LQG · Automatica 2025 | Stochastic CT · two-gain PI | 8 | Finite-sample social optimization; low-sample failures retained. |
| Off-policy Q-learning · IEEE TAC 2023 | DT LQR · matrix Bellman equation | 7 | Data-based LQR; explicit MIMO initialization variant. |
| Infinite-horizon HJB PINN · IJRNC 2025 | Neural HJB · horizon continuation | 5 | Reduced LQR/pendulum training; corrected quartic cost identified. |
| Safe epigraph PINN · ICML 2025 | Epigraph HJB · neural value | 13 | Boat example; collision and budget violations remain. |
| Robust Koopman PI · Preprint 2026 | Lifted bilinear · robust PI | 8 | Held-out error bound fails on 25.55% of points. |
| Bias-policy iteration · Automatica 2026 | Unknown CT nonlinear · fixed data | 12 | Pendulum/arm variants; arm cost is 8.84% above local LQR. |
The method names link to their MATLAB packages. The source papers are credited individually. A passing test is a code-level result; numerical reproduction scope appears in the final column and in each paper card.
| Part | Use it for |
|---|---|
src/+adp |
Small shared MATLAB functions for models, features, integral PI, simulation and result storage. |
reproductions |
Direct functions for each paper, with its native mathematical and data conventions. |
docs/fulltext |
Source versions, equation maps, assumptions and differences from the papers. |
registry |
The method, paper, dependency and implementation-status inventory. |
templates |
Starting a method intake, equation map and experiment record. |
.agents/skills |
Six optional research workflows for coding, reproduction, theory, safety, benchmarks and Simulink. |
flowchart LR
A[Problem and assumptions] --> B[Data and behavior policy]
B --> C[Value or policy learning]
C --> D[Closed-loop evaluation]
D --> E[Results and independent checks]
E -. refine the experiment .-> A
The paper packages retain direct functions. Shared code is extracted when multiple methods use the same mathematical contract. New runs are saved separately from historical records.
- The baseline is checked against an analytic LQR solution.
- Paper packages include independent equation, derivative, integration and numerical checks.
- Safe-PINN collision/budget violations, robust-Koopman bound failures and Bias-PI performance limitations are retained in the validation record.
- Detailed historical runs belong in versioned artifact bundles, keeping the source checkout small.
The three original upstream sources—Frank Lewis’s software, FxT-CL-ACI, and Adaptive-dynamic-programming-algorithms—are acknowledged in THIRD_PARTY. Their complete legacy simulations have not yet been migrated here.
Good first contributions include an independent cross-version run, a clearer equation map, a reproducible bug report, or one complete method with an independent comparator. Read CONTRIBUTING before adding an implementation.
The next priorities are FxT-CL-ACI, one constrained execution example, and one MATLAB/Simulink robot example. See the roadmap and acceptance criteria.
Use CITATION.cff for the software metadata and cite the original paper for each method you use. Record the software version or commit in an experiment report. No software DOI has been assigned.
The independently written software and project documentation are released under the MIT License. Paper PDFs, upstream source archives and author checkpoints are not bundled and retain their own terms. See THIRD_PARTY for the source and adapter attributions.
Maintained by Junkai Tan.