Investigate convergence and scaling differences between Ecosim and FIMS

[Bai-Li-NOAA]

When creating a demo that samples observations from an Ecopath with Ecosim operating model to fit in Fisheries Integrated Modeling System (FIMS), the FIMS model fails to converge. In addition, the estimated biomass, recruitment, and fishing mortality trajectories differ in scale from the OM “truth.” See the vignette here. Help is needed to investigate the source of these discrepancies and improve convergence of the FIMS model.

Potential causes of the mismatches may include, but are not limited to:

• The proportion of females in FIMS is fixed at 0.5, which differs from the assumption used in Ecosim.
• Differences in fishing fleet double-logistic selectivity parameterization between FIMS and the OM: FIMS selectivity is not constrained to a maximum of 1, whereas the OM selectivity is scaled to a maximum of 1.

[Bai-Li-NOAA]

Update: I joined a FIMS code club meeting and got amazing help from the FIMS team debugging the issue. Special thanks to @nathanvaughan-NOAA for the incredibly helpful debugging guidance!

FIMS is fitting to unrepresentative age-composition data in this case. Early years had low catches but were assigned a high sample size (N=200), causing the model to treat those age compositions as highly certain and distort biomass, recruitment, and selectivity estimates to fit them. Reducing the sample size to 1 removed much of their influence, allowing catch and survey indices to drive the fit and better align with the OM.
The next step is to examine the age-composition data more carefully and implement time-varying selectivity. In the current FIMS configuration, a single static double-logistic selectivity curve is applied across all years, forcing the model to fit one average pattern to both early and late periods. If the early age data are atypical, this can bias biomass and recruitment estimates because the static selectivity cannot capture temporal changes in selectivity patterns.

[Bai-Li-NOAA]

I've tried many debugging scenarios that I'm starting to lose track of which results correspond to which configuration. 🤦‍♀️ Maybe I need to include {target} code to better track the methods and their associated results.