You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This study explores how perturbation initial values affects the quality of linear fits for models and for species.
4
-
5
-
In @data/evaluate_monomial_models-0.01, we see that about 15% of the models have a deg1_min of at least 0.9 on their training data. This study compares these results with perturbations of initial values. In the studies, the training data are unperturbed time courses. Separately, time courses are constructed where initial values have been changed by $\pm 5\%$, $\pm 10\%$, $\pm 20\%$, $\pm 50\%$ with a perturbation species fraction of 1.0. A separate ``Timecourse`` is constructed for each perturbation (at total of 8 plus one for 0%), and then the model trained on unperturbed data is used to predict perturbed data. Please use and/or add capabilities to the class method ``SystemDiscovery.analyzePerturbations`` (formerly ``perturbationAnalysis``) to do this analysis. The result is a CSV file with the columns: model_name, threshold, r2_0, r2_-05, r2_-10, r2_-20, r2_-50, r2_+05, r2_+10, r2_+20, r2_+50. The output file path is new argument to ``analyzePerturbations``.
6
-
7
-
* The arguments to ``analyzePerturbations``
8
-
9
-
* model: Model
10
-
* training_df: pd.DataFrame
11
-
* threshold: float
12
-
* perturbations: list[float]
13
-
* perturbation_species_fraction: float = 1.0
14
-
* figsize: tuple[float, float] | None = None
15
-
* poly_degree: int = 1
16
-
* frac_keep: float = 0.2
17
-
* is_plot: bool = True
18
-
*``analyzePerturbations`` returns a pandas.Series
19
-
*@scripts/perturbation_study has a default path for CSV of ``perturbation_study.csv``
20
-
* Calculate $R^2$ using the "derivative" method.
21
-
* Revise and/or rewrite @perturbation_study.py.
22
-
* As needed, make use of existing classes and functions in @src/. Use code in @scripts/ as a guide for implementation, but do not import these modules.
23
-
* The $R_2$ value for a model is the minimum $R^2$ value for all species in the model.
24
-
* The poly_degree is 1.
25
-
* Do $R^2$ clamping so that $0 \leq R^2 \leq 1$.
3
+
This study explores how perturbing initial species values affects the quality of linear fits for models and for individual species.
4
+
5
+
In `data/evaluate_monomial_models-0.01`, about 15% of the models have a `deg1_min` of at least 0.9 on their training data. This study compares those unperturbed results with perturbations applied to initial values. In each run, the training timecourse is simulated unperturbed. Separately, new timecourses are constructed where all species' initial values are changed by a signed fractional amount: ±5%, ±10%, ±20%, ±50% (plus an unperturbed 0% reference), with `perturbation_species_fraction=1.0`. For each perturbation level a separate ``Timecourse`` is simulated, and then the SINDy model trained on the unperturbed training data is used to predict each perturbed timecourse. Use the class method ``SystemDiscovery.analyzePerturbations`` (formerly ``perturbationAnalysis``) for this analysis. The per-model results are written as CSV files in `data/`.
6
+
7
+
## Parameters of ``analyzePerturbations``
8
+
9
+
| Parameter | Type | Default | Description |
10
+
|---|---|---|---|
11
+
|``model``| Model \| int | — | Simulates ground-truth timecourses for each perturbation. Accepts a model object or an integer BioModel number. |
12
+
|``training_df``| pd.DataFrame | NULL_DF | Unperturbed timecourse used to fit the SINDy model. |
|``poly_degree``| int | 1 | Degree of the polynomial library (default linear). |
15
+
|``perturbations``| list[float]|[-0.5, -0.2, -0.1, 0.0, 0.1, 0.2, 0.5]| Signed fractional perturbation values applied to initial species concentrations. |
16
+
|``col_percentile``| str | "p10" | Column of the accuracy DataFrame used for per-perturbation summaries in plots. |
17
+
|``perturbation_species_fraction``| float | 1.0 | Fraction of non-zero species whose initial values are perturbed when simulating each perturbed timecourse. |
Values are clipped to `[0, 1]`. Timepoints where `actual` is zero or non-finite receive a sentinel accuracy of `-1` and are excluded from aggregation. For each species × timepoint pair the Accuracy score aggregates across time via `StatisticCalculator`, producing percentile columns (`mean`, `min`, `max`, `count`, `invalid_count`, `p05`, `p10`, `p20`, `p25`, `p30`, `p50`, `p80`, `p90`, `p95`, `p99`).
35
+
36
+
## Output structure
37
+
38
+
``analyzePerturbations`` returns an ``AnalyzePerturbationsResult`` named tuple containing:
39
+
40
+
-``.df`` — a DataFrame with one row per perturbation value at each requested aggregation level.
41
+
-``.fig`` — the trajectory comparison figure (or `None` if `is_plot=False`).
42
+
43
+
### Rows in the DataFrame
44
+
45
+
| Column | Source | Notes |
46
+
|---|---|---|
47
+
|``perturbation``| set from input list | The signed fractional perturbation value for this row; preserved exactly per-row. |
48
+
|``fraction_species_perturbable``|`perturbation_species_fraction` argument | Same for every row in the result. |
49
+
|``system_id``|`model.model_name`| Model name, repeated on each row. |
50
+
|``aggregation_type``| computed |`'model'` for aggregated rows; species name string for per-species rows. |
51
+
|``mean``, ``min``, ``max``, ``count``, ``invalid_count``, ``p05``–``p99``|`StatisticCalculator` on the Accuracy score | Aggregated across valid (non-sentinel) timepoints within that aggregation level. |
52
+
53
+
### Model-level vs species-level rows
54
+
55
+
- With **both** flags True (the default), the result contains both:
56
+
- One model-level row per perturbation (`aggregation_type='model'`); statistics are aggregated across all non-zero species at each timepoint.
57
+
- One per-species row per perturbation (`aggregation_type=<species name>`).
58
+
- Set `is_analyze_model=False` or `is_analyze_species=False` to drop the corresponding level from the output entirely.
59
+
60
+
### CSV outputs in ``scripts/perturbation_study.py``
61
+
62
+
The script writes three variant CSVs, chosen by which aggregation levels are requested:
63
+
64
+
| Mode | Path pattern |
65
+
|---|---|
66
+
| model + species (default) |`data/perturbation_study-model_species{THRESHOLD}.csv`|
67
+
| model only |`data/perturbation_study-model{THRESHOLD}.csv`|
68
+
| species only |`data/perturbation_study-species{THRESHOLD}.csv`|
69
+
70
+
Each row in a CSV file is one perturbation × aggregation-level combination (i.e. not one row per model). The script skips models listed in its `EXCLUDES` set and resumes from an existing CSV on subsequent runs by checking `system_id`.
0 commit comments