Add documentation page describing PEtab-level training abstractions#64
Add documentation page describing PEtab-level training abstractions#64sebapersson wants to merge 8 commits into
Conversation
dilpath
left a comment
dilpath
left a comment
There was a problem hiding this comment.
Looks good! Some list formatting issues.
Also some questions out of curiosity but no need to address them now.
| value. | ||
| 3. For each PEtab experiment with ID ``expId`` in the MS PEtab problem: | ||
| 1. Create ``nWindows`` new PEtab experiments with IDs ``WINDOW{i}_{expId}`` | ||
| and set the initial time to ``t0_i`` for window ``i = 1..nWindows``. |
There was a problem hiding this comment.
This is rendered as a new paragraph, probably fixed by adding the newline for the previous comment
There was a problem hiding this comment.
I really need to improve my rst-skills, thanks for catching the formatting issues!
| (i.e., at least one subsequent window contains measurements), assign | ||
| initial window values and a continuity penalty: | ||
| 1. In the parameter table, create parameters | ||
| ``WINDOW{i}_{expId}_init_stateId{j}`` for each model state |
There was a problem hiding this comment.
Also rendered as a new paragraph
| 3. The final stage corresponds to the original PEtab problem. Use the parameter | ||
| estimate from stage ``nStages-1`` to initialize optimization for the final | ||
| stage. |
There was a problem hiding this comment.
Does it provide nice training properties to go from duplicating most measurements in the second-last stage, to having no duplicates in the last stage? This seems like it could make training worse, rather than constructing the windows at each stage such that there is no overlap (or weighting the duplicate measurements such that there is effectively no overlap from the perspective of the objective function...).
There was a problem hiding this comment.
This does really improve training, so even though a bit strange, it does help!
There was a problem hiding this comment.
Alright now I'm very excited for your paper. I would be surprised if it can be justified, but I won't argue with results 😀
BSnelling
left a comment
BSnelling
left a comment
There was a problem hiding this comment.
This looks really helpful!
On the whole I'm not clear yet on where the line is between helper functions we would implement in this repo and helpers that importers should implement to support these training strategies. The last section implies importers would implement splitting for curriculum learning and automatic window construction but could those not also make sense as petab_sciml helpers?
|
|
||
| - ``observableFormula = sqrt(lambda) * (stateId{j} - WINDOW{i}_{expId}_init_stateId{j})`` | ||
| - ``noiseFormula = 1.0`` | ||
| - ``noiseDistribution = normal`` |
There was a problem hiding this comment.
You could render this as columns in a table, it might be clearer.
There was a problem hiding this comment.
Actually, render as table did not make it much better
| since a single penalty weight may be impossible to tune. In this case, a | ||
| log-scale penalty such as | ||
|
|
||
| ``sqrt(lambda) * (log(abs(stateId{j})) - log(WINDOW{i}_{expId}_init_stateId{j}))`` |
There was a problem hiding this comment.
Does this definition go somewhere in the PEtab problem or is it up to the implementation?
There was a problem hiding this comment.
This is something Python library util functions could support
Would definitely make sense, I think this is something to discuss a bit down the line which helpers would be needed :) |
Co-authored-by: Dilan Pathirana <59329744+dilpath@users.noreply.github.com> Co-authored-by: BSnelling <branwen.snelling@crick.ac.uk>
…tab_sciml into training_approaches
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #64 +/- ##
=======================================
Coverage 94.01% 94.01%
=======================================
Files 6 6
Lines 301 301
=======================================
Hits 283 283
Misses 18 18 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
|
The PR has now been updated to match the reference implementations (that are in another branch). Ready to be merged once the training strategies are suitable to release. |
4 participants
Training SciML problems can be challenging, and we are currently evaluating several
strategies (curriculum learning, multiple shooting, and curriculum multiple shooting).
This PR adds a docs page that describes how these strategies can be represented at the
PEtab level, which can also be useful for non-SciML PEtab problems.
Longer-term, the goal is for the PEtab SciML library to provide a reference implementation.
This page will help with that, but also to support tool developers across
ecosystems (e.g., Julia) without requiring a Python dependency.