Add partition spec and index layout infrastructure for sex strata

[szu-yun-ko]

What is the feature?

This PR adds the first layer of partition infrastructure for population dynamics. It addresses parts 1–2 of issue #1557 only (representation of sex and indexing layout). Model dynamics, partitioned derived quantities, and likelihood wiring are intentionally deferred to follow-up PRs.

How have you implemented the solution?

• Flesh out subpopulation.hpp with Axis, GroupSelector, PartitionSpec, IndexLayout, and MakeDefaultSexPartitionSpec() (female/male sex axis).
• Add partition_spec and index_layout members to Population.
• Initialize the default sex partition and index layout for each population in CatchAtAge::Initialize().
• Add gtests for partition/indexing utilities and for partition initialization via CatchAtAge::Initialize().

Design Notes

• Partition types live in subpopulation.hpp, matching existing documentation that subpopulations represent generic partitions.
• IndexLayout::i_age_year() preserves the existing pooled (year, age) layout; i_stratum_age_year() adds stratum as the leading dimension for future partitioned containers (n_strata × n_years × n_ages).
• Sex is the first (and only) axis for now; GroupSelector supports wildcards so the same API can extend to additional axes (e.g., area) later.

Does the PR impact any other area of the project, maybe another repo?

• No.

---

Instructions for code reviewer

👋Hello reviewer👋, thank you for taking the time to review this PR!

• Please use this checklist during your review, checking off items that you have verified are complete but feel free to skip over items that are not relevant!
• See the GitHub documentation for how to comment on a PR to indicate where you have questions or changes are needed before approving the PR.
• Please use standard conventional messages for both commit messages and comments
• PR reviews are a great way to learn so feel free to share your tips and tricks. However, when suggesting changes to the PR that are optional please include nit: (for nitpicking) as the comment type. For example, nit: I prefer using a data.frame() instead of a matrix because ...
• Engage with the developer. Make it clear when the PR is approved by selecting the approved status, and potentially commenting on the PR with something like This PR is now ready to be merged.

Checklist

• The code is well-designed
• The code is designed well for both users and developers
• Code coverage remains high- [ ] Comments are clear, useful, and explain why instead of what
• Code is appropriately documented (doxygen and roxygen)

[github-actions]

🎨 Chore: code formatting workflow

Our automated workflows cannot run on forks because of permission issues, and thus, we ask that you run the following code locally and push any changes that are created to your feature branch. You will only be reminded of this once per PR. Thank you!

Format C++ code

1. Install clang-format version 18.0.0
2. Run the following command from the repository root:

   clang-format -i --style="{BasedOnStyle: Google, SortIncludes: false}" $(find ./inst/include ./src ./tests/gtest -name "*.hpp" -o -name "*.cpp")

Format R code

1. Install {styler} and {roxygen2}
2. Run the following commands in R from the repository root:

styler::style_pkg() # Style R code
roxygen2::roxygenise() # Update documentation
styler::style_pkg() # Style R code again
roxygen2::roxygenise() # Update documentation again
usethis::use_tidy_description() # Style DESCRIPTION file

Push changes

1. Commit the formatting with a commit message of "Chore: format feature branch"
2. Push to your fork

[codecov]

Codecov Report

❌ Patch coverage is 95.86957% with 19 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.21%. Comparing base (b7783d7) to head (882cca9).
⚠️ Report is 10 commits behind head on main.

Files with missing lines	Patch %	Lines
...e/population_dynamics/population/subpopulation.hpp	89.14%	10 Missing and 9 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1581      +/-   ##
==========================================
+ Coverage   84.88%   86.21%   +1.33%     
==========================================
  Files         105       59      -46     
  Lines        9473     2684    -6789     
  Branches      536      628      +92     
==========================================
- Hits         8041     2314    -5727     
+ Misses       1395      258    -1137     
- Partials       37      112      +75     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[szu-yun-ko]

@awilnoaa @nathanvaughan-NOAA @Andrea-Havron-NOAA This PR should be ready for an initial review. This PR addresses the tasks mentioned in issue #1557, after the review comments are addressed, I will continue tackling tasks in issue #1587 in this same PR.

Partition representation:

• Axis: a partition dimension (e.g. sex) with a vector of named levels. The size of an Axis is inferred from the length of its level vector.
• GroupSelector: a partial selection pattern that can be mapped to one stratum or many strata, where each stratum is a combination of a level from each axis.
• PartitionSpec: the schema of the partition grid, it includes a vector of axes (each having its own name and levels). Member functions include n_strata(), stratum_id(), levels_from_stratum(), and expand_group_to_strata(). These member functions are primarily used for bridging user-facing partition concepts and flat storage indices.
  • A stratum is the atomic storage unit (one level chosen on every axis).
  • A group (GroupSelector) is a looser selection that may match one or many strata.
  • expand_group_to_strata() resolves a group to the strata that should be read or written.
  • stratum_id() and levels_from_stratum() convert between those strata and the integer indices that appear in partitioned vectors.
  • n_strata() sizes those containers (e.g. n_strata × n_years × n_ages for fleet at-age DQs).
• IndexLayout: provides folded index helpers for pooled (year, age) and partitioned (stratum, year, age) storage.
• MakeDefaultSexPartitionSpec(): a function for setting up the current (default) partition, which is to include one axis (sex) with two levels (female and male).

Wiring into population:

• Population gains partition_spec and index_layout members (population.hpp).
• CatchAtAge::Initialize() sets the default sex partition and populates index_layout from n_years, n_ages, and partition_spec.n_strata() (catch_at_age.hpp).

Partitioned derived-quantity storage:

• Registers four fleet at-age *_by_partition DQs in rcpp_models.hpp:
  • landings_numbers_at_age_by_partition
  • landings_weight_at_age_by_partition
  • index_numbers_at_age_by_partition
  • index_weight_at_age_by_partition
• Each is sized n_strata × n_years × n_ages with DimensionInfo {"n_strata", "n_years", "n_ages"}
• Note that in this PR we implement the storage only. These containers are not filled during Calculate* yet (vectors are allocated and zeroed on Prepare()). This will be addressed later on (ref: issue Partition demand configuration #1587) along with the demand-based calculation routing logic.

Tests added:

GTest

• test_population_Subpopulation.cpp: default sex specifications, stratum encoding and decoding, stratum group conversion (include wildcard and single-stratum cases), folded indices.
• test_population_CatchAtAge_InitializePartition.cpp: Initialize() sets partition state on Population.
• test_fleet_Fleet_InitializePrepare.cpp: Check that for each fleet, the four *_by_partition vectors have size n_strata × n_years × n_ages, while pooled at-age DQs remain n_years × n_ages. Also check that after Prepare(), the same partitioned vectors are all zeros. This confirms storage is allocated and reset even though Calculate* does not fill partitioned values yet.

testthat

• Column names: expected_colnames now includes stratum_i, the R-side name for the n_strata dimension after reshaping JSON model output.
• Regenerated because registering partitioned fleet DQs adds rows to get_estimates() on the integration fixtures (deterministic and estimation runs).

[Copilot]

Pull request overview

Adds the first layer of partition/indexing infrastructure to support sex strata in population dynamics (as groundwork for future partitioned state/derived quantities), and exposes the new stratum dimension through the R output reshaping and related tests.

Changes:

• Introduces partition primitives (Axis, GroupSelector, PartitionSpec) and indexing helpers (IndexLayout) plus a default sex partition spec.
• Stores partition_spec and index_layout on Population and initializes default sex partition/index layout in CatchAtAge::Initialize().
• Adds partitioned fleet derived-quantity containers (storage only), updates R output dimension naming (stratum_i), and updates/extends gtests + testthat snapshots.

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
tests/testthat/test-get_estimates.R	Updates expected estimate columns to include stratum_i.
tests/testthat/_snaps/get_estimates.md	Updates snapshot output for new column and row/col counts.
tests/gtest/test_population_test_fixture.hpp	Initializes partitioned fleet age-based derived quantity vectors in fixtures.
tests/gtest/test_population_Subpopulation.cpp	Adds unit tests for PartitionSpec and IndexLayout.
tests/gtest/test_population_CatchAtAge_InitializePartition.cpp	Adds fixture-based test for default partition/index layout initialization in CatchAtAge::Initialize().
tests/gtest/test_fleet_Fleet_InitializePrepare.cpp	Extends fleet DQ size/reset expectations to include partitioned vectors.
tests/gtest/CMakeLists.txt	Registers the two new gtest executables.
R/reshape_output.R	Maps n_strata dimension header to stratum_i for tidy output.
inst/include/population_dynamics/population/subpopulation.hpp	Implements partition spec structures, indexing layout, and default sex partition helper.
inst/include/population_dynamics/population/population.hpp	Adds partition_spec and index_layout members to Population.
inst/include/models/functors/catch_at_age.hpp	Initializes default sex partition spec and index layout during model initialization.
inst/include/interface/rcpp/rcpp_objects/rcpp_models.hpp	Adds partitioned fleet DQ vectors + dimension metadata for R interface output.

Comments suppressed due to low confidence (1)

inst/include/interface/rcpp/rcpp_objects/rcpp_models.hpp:1276

• index_weight_at_age (and its DimensionInfo) are initialized a second time immediately after index_weight_at_age_by_partition. This duplicate block is redundant and likely accidental; it should be removed to avoid confusion and to ensure future edits don’t update only one copy.

      derived_quantities["index_weight_at_age"] = fims::Vector<Type>(
          fleet_interface->n_years.get() * fleet_interface->n_ages.get());
      derived_quantities_dim_info["index_weight_at_age"] =
          fims_popdy::DimensionInfo(
              "index_weight_at_age",

[szu-yun-ko]

This PR already adds partition structure (PartitionSpec, IndexLayout) and storage (fleet *_by_partition DQs). I've addressed the Copilot comments and added the split math we’ll need when writing into those partitioned containers (but Calculate* does not call any of this yet). The wiring will be addressed after the infrastructure is reviewed so that I can make sure I'm not building on unsolid grounds. The two helpers added in the latest commits include:

SexStratumSplitFactors(spec, proportion_female):

• For the default sex-only partition only (MakeDefaultSexPartitionSpec(), 2 strata). It returns {p_female, 1 - p_female}. proportion_female comes from Population::proportion_female.get_force_scalar(age) at evaluation time and it is not stored on PartitionSpec. It's not wired at the moment but see below for a sketch of how it will work later on.

PartitionSpec::stratum_split_factor(stratum, split_factors):

• This is a generic lookup for split_factors[stratum] with validation. split_factors is passed in by the caller and it is not a field on PartitionSpec. The spec defines how many strata exist, the caller supplies how much goes to each.

// Pooled write (unchanged)
const Type pooled = (Fmort * f_multiplier * selectivity) / Z
                    * N_aa * (1 - exp(-Z));
fdq_["landings_numbers_at_age"][i_age_year] += pooled;

// Partitioned write (future; sex-only Model 1)
const auto split_factors = SexStratumSplitFactors(
    population->partition_spec,
    population->proportion_female.get_force_scalar(age));

for (size_t s = 0; s < population->partition_spec.n_strata(); ++s) {
  const size_t i_part =
      population->index_layout.i_stratum_age_year(s, year, age);
  fdq_["landings_numbers_at_age_by_partition"][i_part] +=
      pooled * population->partition_spec.stratum_split_factor(s, split_factors);
}

The changes in this PR so far addresses all tasks mentioned in issue #1557 and I'll proceed to work on #1587 after the current work is reviewed.

[nathanvaughan-NOAA]

It doesn't look like any of these partitioned derived quantities are being pushed back to info->variable_map. In order to do that I think they will also need to be initialized as VariableVectors in the fleet and population .hpp files. You could look at how their respective non-partitioned quantities are handled to figure out which files are appropriate.

[nathanvaughan-NOAA]

The code here looks to be limited to a single new axis layer with hard coded years/ages axes. Do you have a plan for extending this to nested axes such as sex and area?

[szu-yun-ko]

Great question! Multi-axis nesting (e.g. sex × area) is handled by PartitionSpec, not by nesting more dimensions into IndexLayout. PartitionSpec holds a vector of axes, and n_strata() is the product of each axis’s level count (ref. subpopulation.hpp around line 47). Adding area (or season) increases n_strata and it does not require nested helpers like i_sex_area_year_age. The current wiring pulls n_strata from the spec in CatchAtAge::Initialize() (ref. catch_at_age.hpp around line 160):

      this->populations[p]->partition_spec = MakeDefaultSexPartitionSpec();
      this->populations[p]->index_layout.n_years = this->populations[p]->n_years;
      this->populations[p]->index_layout.n_ages = this->populations[p]->n_ages;
      this->populations[p]->index_layout.n_strata =
          this->populations[p]->partition_spec.n_strata();

[Andrea-Havron-NOAA]

@szu-yun-ko, @nathanvaughan-NOAA will be out for three weeks so I will be taking over primary review responsibilities while he is gone. I noticed you recently pushed changes to this PR. Is this ready for review now, or are you planning to add more commits first?

[szu-yun-ko]

@Andrea-Havron-NOAA No problem, thanks for letting me know. The current pushed commits are mainly for addressing the previous review comments. I’ve made more changes locally that addresses tasks mentioned in issue #1587, but I’m still making some final tests and fixes. It will be pushed for review in a few days! Thanks for checking in!

[szu-yun-ko]

@Andrea-Havron-NOAA @awilnoaa I've pushed recent changes that mainly address issue #1587. The only part left would be the R-interface, but I thought we could make sure the backend behaves as we intended before adding the interface. As the PR is getting larger, below is a brief description of every commit, mainly why we need the changes and how the changes were implemented. Please let me know if anything else is needed during the review process.

---

1. e35456eb — Demand as a named-axis request

Need a way to say “I want female only / male / both / nothing” that can grow to multi-axis later, without a sex-only enum.

How it works: The demand can be given as a C++ list (e.g. list(sex = "female")), which maps to stratum ids (e.g. {0}). An empty demand corresponds to no strata, therefore no partitioned results would be written.

struct AxisLevelSelection {
  std::string axis_name;                 /*!< axis name (e.g. "sex") */
  std::vector<std::string> level_names; /*!< requested level labels */
};

/**
 * @brief User demand for which partition strata to materialize in output.
 *
 * @details Empty selections means pooled output only (no partitioned write) —
 * the backward-compatible default. ...
 */
struct PartitionDemand {
  std::vector<AxisLevelSelection> selections; /*!< empty => pooled */

  /** @brief True when no partitioned strata are requested. */
  bool is_pooled() const { return selections.empty(); }
};

• Considering extendability, we need to reframe from using sex-specific enums for declaring demand (e.g. kFemale, kMale…etc.). We need a way for the user to specify axis + level as a list, so in the future the user can be specific about what level they want on each axis (e.g. list(sex = “female”, area = “north”)).
• In order to do that we declare a struct called AxisLevelSelection, which is the combination of an axis + a level on that axis. Then the PartitionDemand struct will simply consist of a vector of AxisLevelSelection (i.e. a vector of axis + level combinations).
  • PartitionDemand also includes a boolean flag to note whether the demand is pooled or not. When no partitioned strata are required, is_pooled() will be flagged as true.

inline std::vector<size_t> RequestedStrata(const PartitionSpec &spec,
                                           const PartitionDemand &demand) {
  if (demand.is_pooled()) {
    return {};
  }
  return spec.expand_group_to_strata(MakeGroupSelectorFromDemand(spec, demand));
}

• Given a PartitionSpec and a PartitionDemand, we can resolve which strata is requested via the RequestedStrata function, which first creates a group from the demand and then expand the group to strata IDs.

---

2. acd83586 — Hang demand on the population

Evaluation must read demand from the population Population::partition_demand. Default must stay backward-compatible (pooled): Initialize() sets MakePooledPartitionDemand().

How it works: Every run starts pooled unless something (later: R interface; today: tests) replaces demand after init.

  // partition infrastructure (see subpopulation.hpp)
  PartitionSpec partition_spec; /*!< axes and strata for partitioned structure */
  IndexLayout index_layout;     /*!< folded indices for partitioned quantities */
  PartitionDemand partition_demand =
      MakePooledPartitionDemand(); /*!< which strata to materialize; empty = pooled */

      this->populations[p]->partition_spec = MakeDefaultSexPartitionSpec();
      this->populations[p]->partition_demand = MakePooledPartitionDemand();
      this->populations[p]->index_layout.n_years = this->populations[p]->n_years;
      this->populations[p]->index_layout.n_ages = this->populations[p]->n_ages;
      this->populations[p]->index_layout.n_strata =
          this->populations[p]->partition_spec.n_strata();

---

3. b6217b9a — One write helper for split-on-write

Landings and index should share the same demand + sex-ratio logic, not duplicate it four times. So we need one place for splitting pooled cell onto sex-specific containers/planes: WritePartitionedQuantityAtAge(...).

How it works:

1. RequestedStrata: if empty, return.
2. Else build {p, 1−p} via SexStratumSplitFactors.
3. For each requested stratum only:partitioned[i_stratum_age_year] += pooled_value × factor.

Scope note: This helper is for the sex-ratio at age model only: split pooled value × proportion_female onto demanded sex strata. Explicit two-sex later should use the same demand/storage, but compute per sex in Calculate* rather than extending this splitter. Later on when the explicit two-sex model is also implemented, we can branch at evaluation to decide which path to use. Today, the code roughly looks like this (pseudo-code):

Evaluate()
  for each year, age:
    CalculateLandingsNumbersAA  → pooled landings + WritePartitioned… (Model 1 split)
    CalculateIndexNumbersAA     → pooled index   + WritePartitioned… (Model 1 split)
    ...

Later on the branching might look like:

Evaluate() / Calculate*
  if model_type == sex_ratio_at_age:     // Model 1
      compute pooled catch
      write pooled DQ
      WritePartitionedQuantityAtAge(...)   // split by proportion_female

  else if model_type == explicit_two_sex: // Model 3
      compute female catch from N^f
      compute male catch from N^m
      write into *_by_partition per stratum
      optionally pooled = female + male

---

4. 882cca93 — Plug the helper into evaluation

After each pooled landings/index at-age update, call the helper with the same cell value so that partitioned DQs actually gets filled when the demand asks for them.

How it works:

• Pooled path unchanged (landings_*_at_age, index_*_at_age).
• Same increment/weight also offered to the helper.
• With default pooled demand → _by_partition stays zero.
• If demand were female → female slot = pooled × p, male stays 0.

      // Baranov Catch Equation
      Type landings_naa =
          (population->fleets[fleet_]->Fmort[year] *
           ...
      fdq_["landings_numbers_at_age"][i_age_year] += landings_naa;
      fims_popdy::WritePartitionedQuantityAtAge(
          fdq_["landings_numbers_at_age_by_partition"], landings_naa,
          population->partition_spec, population->index_layout,
          population->partition_demand,
          population->proportion_female.get_force_scalar(age), year, age);

Testing: Tests are added to make sure the demand-based computations work.

    TEST_F(CAAEvaluateTestFixture, FemaleDemandWritesLandingsByPartition)
    {
        ...
        population->partition_demand =
            fims_popdy::MakeSexPartitionDemand({"female"});
        ...
        catch_at_age_model->CalculateLandingsNumbersAA(...);
        ...
            EXPECT_DOUBLE_EQ(
                dq_fleet["landings_numbers_at_age_by_partition"][female_idx],
                dq_fleet["landings_numbers_at_age"][i_age_year] * p_female);
            EXPECT_DOUBLE_EQ(
                dq_fleet["landings_numbers_at_age_by_partition"][male_idx], 0.0);

[awilnoaa]

This looks great. When you get to the user-facing choice, it may be helpful to include a test showing that a user request for partitioned output is applied during model setup and is not accidentally reset to pooled during initialization.

[awilnoaa]

I know the design mentions sex is the only axis for now. This may be outside the scope of this PR, but I am curious, @Andrea-Havron-NOAA: would the DQ storage eventually need to be sized from the active PartitionSpec so it stays in sync with n_strata changes?

[Andrea-Havron-NOAA]

Yes, I think this will need to be made more generic for keeping dimensions in sync. @szu-yun-ko, for the longer term design, will we need a PartitionSpec defined for each partitioning scheme (e.g. AreaPartitionSpec, SeasonPartitionSpec, etc.), or can this be a single generic PartitionSpec build from user-provided axes and levels?

[awilnoaa]

This looks good to me. When the R interface is added, it may be helpful to make sure users cannot accidentally give two different requests for the same axis, like sex = female and sex = male as separate entries. That way the backend always gets one clear request and does not have to silently choose one.

[awilnoaa]

Looks good overall! I had a few comments/questions but most can be addressed when the user-facing side is added.

[Andrea-Havron-NOAA]

Nice progress on partitioned DQ wiring. One concern: WritePartitionedQuantityAtAge still hard-codes sex behavior via proportion_female_at_age and SexStratumSplitFactors. Could we make the writer generic by using a split policy (or per-stratum weights) and keep sex as one policy implementation in a future PR? That would let area/season/etc. reuse the same write path without changing core indexing code.

[Andrea-Havron-NOAA]

Thanks for the PR — this is useful infrastructure for subpopulation partitioning.

My main takeaway is that the structural pieces (PartitionSpec, IndexLayout, partitioned-at-age write path) are now in place, while some allocation logic is still sex-specific (e.g., SexStratumSplitFactors, proportion_female_at_age). This is great incremental progress, and I'm looking forward to follow-up work that generalized split behavior and sets up dimensions based on user input.