Skip to content

Bootstrap FIMS memory profiling benchmark scaffold (R stages, Massif runner, Codespaces debug build config) - #1

Merged
Andrea-Havron-NOAA merged 6 commits into
mainfrom
copilot/initialize-repo-for-memory-profiling
Jul 22, 2026
Merged

Bootstrap FIMS memory profiling benchmark scaffold (R stages, Massif runner, Codespaces debug build config)#1
Andrea-Havron-NOAA merged 6 commits into
mainfrom
copilot/initialize-repo-for-memory-profiling

Conversation

Copilot AI commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

This PR initializes the repository as a repeatable benchmark harness for comparing memory behavior across NOAA-FIMS/FIMS refs. It adds the baseline structure, staged R benchmark entrypoints, and profiling-oriented debug build setup for Codespaces.

  • Repository scaffold

    • Added:
      • R/ for benchmark/setup helpers
      • scripts/ for profiler runners
      • outputs/.gitkeep as committed output placeholder
      • .devcontainer/ for Codespaces initialization
  • FIMS debug install entrypoint

    • Added R/setup_FIMS with install_fims_debug(ref) to install NOAA-FIMS/FIMS from a branch/tag/SHA via source build (remotes::install_github).
  • Staged benchmark flow (R)

    • Added R/run_benchmark.R with explicit stages:
      1. Static model construction through MakeADFun()
      2. Single eval calls (obj$fn(), obj$gr())
      3. nlminb run without sdreport
      4. nlminb run with sdreport
      5. Cleanup/retention check (TMB::FreeADFun(obj), gc())
  • Branch comparison driver

    • Added R/main.R to run the same benchmark flow for:
      • main
      • xptr-refactor
  • Massif execution script

    • Added scripts/run_massif.sh to run Valgrind Massif per ref and emit:
      • outputs/massif_main.out
      • outputs/massif_xptr.out
  • Codespaces debug symbol preservation

    • Added .devcontainer/postCreate.sh + .devcontainer/devcontainer.json to ensure ~/.R/Makevars contains:
      • PKG_CXXFLAGS += -g -O0 -fno-omit-frame-pointer -fvisibility=default
      • PKG_STRIP = true
  • Documentation

    • Rewrote README.md with setup and execution guidance for debug installs and Massif benchmark runs.
# Run memory profiling for both refs
bash scripts/run_massif.sh
# -> outputs/massif_main.out
# -> outputs/massif_xptr.out
Original prompt

Initialize a repository for memory profiling and benchmarking the NOAA-FIMS/FIMS package

Please set up the following directory structure and starter files:

  1. Directory Structure:

    • R/: Helper scripts for running benchmark stages and parsing profiling output.
    • scripts/: Shell scripts to run benchmarks under Valgrind Massif or gproftools.
    • outputs/: Placeholder directory for storing .out files and summary reports (include a .gitkeep).
    • .devcontainer/ (or setup script): Pre-configure user level ~/.R/Makevars inside Codespaces with debug flags (PKG_CXXFLAGS += -g -O0 -fno-omit-frame-pointer -fvisibility=default and PKG_STRIP = true) to prevent stripping debug symbols during builds.
  2. Starter Files:

    • R/run_benchmark.R: An R script structured into 3 distinct stages:
      • Stage 1: Static model construction up through MakeADFun().
      • Stage 2: Single evaluation calls (obj$fn() and obj$gr()).
      • Stage 3: a full model run with nlminb but without sdreport
      • Stage 4: a full model run with nlminb and sdreport
      • Stage 5: Memory cleanup/retention check (TMB::FreeADFun(obj) & gc()).
        -R/setup_FIMS: an R script that installs FIMS from the NOAA-FIMS/FIMS repo:

#' Install a specific FIMS branch compiled in Debug Mode
#'
#' @param ref Branch name, tag, or commit hash (e.g. "main", "xptr-refactor")
install_fims_debug <- function(ref = "main") {
message(sprintf("Installing NOAA-FIMS/FIMS@%s in debug mode...", ref))

Ensure remotes is available

if (!requireNamespace("remotes", quietly = TRUE)) {
install.packages("remotes")
}

Force compilation from source with user's ~/.R/Makevars applied

remotes::install_github(
repo = "NOAA-FIMS/FIMS",
ref = ref,
force = TRUE,
build_vignettes = FALSE,
INSTALL_opts = c("--no-multiarch")
)
}

  -`R/main.R`:

1. Benchmark Main

install_fims_debug("main")
source("R/run_benchmark.R") # Outputs to outputs/massif_main.out

2. Benchmark Feature Branch

install_fims_debug("xptr-refactor")
source("R/run_benchmark.R") # Outputs to outputs/massif_xptr.out

   - `scripts/run_massif.sh`: A shell script that executes the benchmark script using `valgrind --tool=massif` and outputs results to the `outputs/` folder.
   - `README.md`: Documenting instructions on how to install FIMS from GitHub/branches in debug mode and run the benchmark suite.

Copilot AI changed the title [WIP] Initialize repository for memory profiling and benchmarking NOAA-FIMS/FIMS package Bootstrap FIMS memory profiling benchmark scaffold (R stages, Massif runner, Codespaces debug build config) Jul 22, 2026
Copilot AI requested a review from Andrea-Havron-NOAA July 22, 2026 20:38
Comment thread R/setup_FIMS
@@ -0,0 +1,20 @@
#' Install a specific FIMS branch compiled in Debug Mode

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in f8452dd. The script is now named R/install_FIMS_debug and all references were updated.

Comment thread .devcontainer/devcontainer.json
Comment thread R/run_benchmark.R Outdated

@Andrea-Havron-NOAA Andrea-Havron-NOAA left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you squash before pushing commits?

@Andrea-Havron-NOAA
Andrea-Havron-NOAA marked this pull request as ready for review July 22, 2026 21:11
@Andrea-Havron-NOAA
Andrea-Havron-NOAA merged commit 25f31d6 into main Jul 22, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants