[FEATURE]: FIMS research project tracking website

[Bai-Li-NOAA]

Please describe the feature you are requesting.
A centralized and structured system for tracking the usage of FIMS across research projects (not stock assessments). This would include a standardized way to organize and showcase research projects and publications connected to FIMS.

Describe the solution you'd like
For research projects, leverage the {workflowr} R package as a recommended package for organizing analyses. Each project would maintain its own GitHub repository and generate a reproducible R Markdown-based website. Rather than FIMS hosting all project outputs, FIMS would maintain a central website repository that aggregates and links to these external project sites.

Because {workflowr} uses a consistent structure, the FIMS website could automatically extract data (e.g., project descriptions, authors, publications) from individual project repositories. This would enable:

• A Projects page listing all FIMS-related research efforts
• A Publications page aggregating outputs across projects

This approach promotes reproducibility without requiring FIMS to manage all project data directly.
This is not for tracking stock assessment inputs and outputs. That would require implementing a dedicated database in a consistent and queryable format.

Describe alternatives you've considered
Not have a website to track research projects.

Additional context
Briefly discussed this idea with @kellijohnson-NOAA and she suggested posting an issue here. This idea builds on prior discussions about improving how FIMS usage is tracked and shared.

[e-perl-NOAA]

@Bai-Li-NOAA @kellijohnson-NOAA
So even if we used {workflowr}, we wouldn't be able to use the workflowr.io app because we don't have any of the information needed to run this workflow that actually runs this script calling the {wio} package to get the github repos and publications since we are not the creators of the app, which feels like it defeats the purpose of using {workflowr} in this particular case.

So essentially what {workflowr} is doing to be able to get access to the right repos is that they created their github app which is linked above, and have users install that. This gives them the privileges that they need to find repos that use workflowr and get the publication info which is stored in a data/publications.yml file of a repository.

If we wanted to do this for FIMS, it doesn't matter what format a project uses (workflowr, bookdown, etc.), what we need is the following:

1. A format/template that we want them to follow like bookdown with all of the right instructions and github actions ready to go, I think we could use whatever the Case studies used (bookdown) and make a template from that, maybe even the workflows team could help us with this. I also am not sure if this should be a template or maybe a function in FIMS itself to create the file structure needed?? This is a question we would need to discuss.
2. What it needs to have to be discoverable for the "projects" page is that it has to have the our FIMSflowr.io (stand-in name) app enabled, and it has to have a github.io page for the project enabled.
3. For it to discover publications, would would just have to have some sort of dedicated publications file like they did, though it doesn't have to exactly be data/publications.
4. Write scripts and workflows that grab all this info
5. Add all of this info to a page on our FIMS website

This is a pretty big lift that is actually mostly GitHub/actions/scripting heavy. I think that we will want to have a bigger conversation about how to actually put something like this into effect for FIMS.

[Bai-Li-NOAA]

@e-perl-NOAA Yes, I was thinking about developing our own website for tracking projects. Lately, I haven’t been sure whether we want to create and maintain more packages or rely on existing ones 😆 , but I like the idea of teaming up with the workflows team to set up a template. They’re currently working on a template repository for organizing and standardizing files created in the stock assessment workflow.

[kellijohnson-NOAA]

@Bai-Li-NOAA thanks for bringing this up, I would ❤️ to have an automated way to track publications that use FIMS. I am not sure of the best way to do that though. I agree with @e-perl-NOAA that the structure of {workflowr} is probably too restrictive for us and people will end up not using it, which will lead to not knowing where FIMS is actually used. I think it wouldn't be too big of a lift to suggest a file be added to repositories so we can search for it. I don't think we have to go as far as creating a GitHub App. I think much of the information can be gleaned from information that is included in GitHub automatically. We could also create a database that stores the same information for publications that are not on GitHub because not everything is open source. I would like to use this issue to determine the information that we want to be in that file? And create a script/action to publish that information on this website.

[e-perl-NOAA]

@kellijohnson-NOAA We can ask users to add a file with a name like fims-metadata.yml to their repository root, and/or tag their repo with the topic fims-project and then run a weekly GitHub Action that searches for fims-metadata.yml using the GitHub API, parses the data, and builds the "Projects" page. This of course won't allow us to get any info on private repos though. However, a big question I have is how do we get users to do this by default, because otherwise I don't see us getting the info we need.?

Options I can think of right now (I welcome any more ideas of course):

1. Work with the workflows team to come up with a FIMS template repo or file/folder structure repo via a function that auto creates the file we want and in our FIMS training we teach FIMS users to exclusively use that to set up FIMS projects. For example: fims::create_project()
2. Write a function like fims::track_fims_usage() that creates the file we want and sneakily put it into a function that FOR SURE everyone uses (like fitFIMS), obviously disclose that we do this, and have an option to turn that off
3. Write the fims::track_fims_usage() function, keeping it on it's own, and hope for the best that people use it

Another question that I have if we do this is that we may get people starting repos just to play around with FIMS, or having multiple repos with the same info. How are we going to QA/QC this list projects? We could have the fims-metadata.yml have a line that is status: development and then have users switch it to status: complete or something like that, but then that get's us back to the same issue as before, are users going to actually remember to do this?

[kellijohnson-NOAA]

@e-perl-NOAA I completely agree with you that we have to find some way to do this that limits the amount of work people have to do. I think our best bet is to just scrape GitHub for use of library(FIMS). I don't think we need to do it as often as once a week, maybe once a month. We can start with seeing what information we can get out of the API from the case study repository. For example, we can see who has made commits and the title of the repository. I think the action could search for the repos and if new repos are found then it can alert us. We can then email the individuals and ask if they would be willing to fill out a fims-metatda.yml file, we could eventually even make that part more automated. We can make a list of "false-positive" repos that the action should ignore. And, if we end up changing the information needed in the fims-metadata.yml file we can always open PRs to the repos that already have it asking them to change what is available.

[e-perl-NOAA]

The following could be the first stage for our "Discovery" of repos. There would be a second GitHub action to actually update the projects page. Two things that I would like input on @kellijohnson-NOAA, @Bai-Li-NOAA:

1. What information needs to be in the fims-metadata.yml? For example, Gemini put the following in the yml

project_name: "My FIMS Project"
author: ""
publications: []

2. What information for the project on GitHub needs to go on the eventual Projects page? Some of this can come from the API but some information that we might want we won't be able to get from the API so we would want it to be in the fims-metadata.yml

name: FIMS Package Discovery, Report, & Outreach

# Run on the 1st of every month at 14:00 UTC, and allow manual triggering
on:
  schedule:
    - cron: '0 14 1 * *'
  workflow_dispatch: 

jobs:
  discover-and-report:
    runs-on: ubuntu-latest
    permissions:
      issues: write   
      contents: read

    steps:
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: Set up R
        uses: r-lib/actions/setup-r@v2
        with:
          r-version: 'release'
          
      - name: Install required R packages
        run: |
          Rscript -e 'install.packages(c("httr", "jsonlite"), repos = "https://cloud.r-project.org")'

      - name: Run Discovery Script (R)
        id: discovery
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          library(httr)
          library(jsonlite)

          # Set up authentication headers
          token <- Sys.getenv("GITHUB_TOKEN")
          auth_header <- add_headers(
            Authorization = paste("Bearer", token), 
            Accept = "application/vnd.github.v3+json"
          )

          # 1. Helper function to load text lists
          load_list <- function(file_path) {
            if (file.exists(file_path)) {
              lines <- readLines(file_path, warn = FALSE)
              # Remove empty lines
              return(lines[lines != ""])
            }
            return(character(0))
          }

          ignored <- load_list("ignore_repos.txt")
          contacted <- load_list("contacted_repos.txt")
          skip_list <- unique(c(ignored, contacted))

          # 2. Search GitHub for FIMS usage
          # Query looks for "library FIMS" in R, Rmd, and qmd files
          query <- "q=library+FIMS+extension:R+extension:Rmd+extension:qmd"
          url <- paste0("https://api.github.com/search/code?", query)
          
          res <- GET(url, auth_header)
          stop_for_status(res)
          
          search_results <- content(res, as = "parsed", type = "application/json")

          # Extract unique repository names
          if (length(search_results$items) > 0) {
            found_repos <- sapply(search_results$items, function(x) x$repository$full_name)
            found_repos <- unique(found_repos)
          } else {
            found_repos <- character(0)
          }

          # 3. Filter out known repos
          new_repos <- setdiff(found_repos, skip_list)

          # 4. Check remaining repos for fims-metadata.yml
          final_targets <- list()
          
          if (length(new_repos) > 0) {
            for (repo in new_repos) {
              check_url <- paste0("https://api.github.com/repos/", repo, "/contents/fims-metadata.yml")
              check_res <- GET(check_url, auth_header)
              
              # If we get a 404, the file does not exist, so add them to our target list
              if (status_code(check_res) == 404) {
                
                # Grab repo details for the report
                info_url <- paste0("https://api.github.com/repos/", repo)
                info_res <- GET(info_url, auth_header)
                
                if (status_code(info_res) == 200) {
                  info <- content(info_res, as = "parsed", type = "application/json")
                  
                  owner <- if (!is.null(info$owner$login)) info$owner$login else "Unknown"
                  desc <- if (!is.null(info$description)) info$description else "No description provided."
                  
                  final_targets[[length(final_targets) + 1]] <- list(name = repo, owner = owner, desc = desc)
                }
              }
            }
          }

          # 5. Generate Markdown Report and Set Output
          env_file <- Sys.getenv("GITHUB_OUTPUT")
          
          if (length(final_targets) == 0) {
            message("No new repositories found this month.")
            cat("found_new=false\n", file = env_file, append = TRUE)
          } else {
            cat("### 🚀 New FIMS Repositories Discovered!\n\n", file = "report.md")
            cat("The monthly discovery action found the following repositories using FIMS that do not have a `fims-metadata.yml` file.\n\n", file = "report.md", append = TRUE)
            
            for (t in final_targets) {
              cat(sprintf("**Repository:** [%s](https://github.com/%s)\n", t$name, t$name), file = "report.md", append = TRUE)
              cat(sprintf("**Owner/Commiter:** @%s\n", t$owner), file = "report.md", append = TRUE)
              cat(sprintf("**Description:** %s\n\n", t$desc), file = "report.md", append = TRUE)
              cat("---\n", file = "report.md", append = TRUE)
            }
            
            cat("found_new=true\n", file = env_file, append = TRUE)
          }
        shell: Rscript {0}

      - name: Create GitHub Issue
        # Only run this step if the R script actually found new repos
        if: steps.discovery.outputs.found_new == 'true'
        env:
          GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        run: |
          gh issue create \
            --title "Monthly FIMS Discovery Report" \
            --body-file report.md \
            --label "outreach"

[Bai-Li-NOAA]

@e-perl-NOAA You are super fast! Just a FYI: I came across the Ecopath with Ecosim impact tracking page. I was really impressed by how they’ve documented capacity-building training since 1992, as well as publications and policy impacts across different countries since 2000. Their publication section may not be collecting as much information as we’d ideally want (e.g., repositories, code, etc.). I’ll reach out to learn more about how the data is gathered on their backend, and I’ll share what I find since I have a question about their testing data anyway. I also think this impact website might be connected to the outreach website issue #86

[kellijohnson-NOAA]

Just a note about another resource that might be helpful is Hadley Wickham has a repository called cran-deadlines that creates this website that is a searchable database of CRAN packages that are set to be removed from CRAN.

Regarding (1):

• doi
• topic
• type: simulation, production assessment, research, development fork
• species

Regarding (2): It was hard to see from the R script what is being show but it appears as though the following

• Repository
• Owner/Commiter
• Description
  I would add
• Location
• FIMS Version
• Date created

I think it would be great if we could have a searchable database like what Hadley has and then graphics like Bai's example with EWE.

@Rick-Methot-NOAA do you have any additional suggestions here?

[Rick-Methot-NOAA]

I'm really glad to see this effort. Finding the EWE site was good work Bai.
It seems that the effort is being built around github. Have you considered how to access FIMS applications that do not use github, or will that even be possible?
@k-doering-NOAA - How do you see this interfacing with FIT?
Also, there is an effort starting up to revamp the RAM assessment database, which has results and meta-data about assessments. I suggest an inquiry in that direction also. RAM already directly connects with SIS.

[e-perl-NOAA]

@kellijohnson-NOAA I guess dreaming bigger here, but I could see it eventually being useful to have some stats on what features the model is using. I wonder if down the line a function could be created that looks at the FIMS model and produces whatever that info would be. I could see as a part of a PR to new repos that use FIMS, the PR asks them to run that function which puts the additional info into the fims-metadata.yml file

[kellijohnson-NOAA]

Yes, I agree that down the line that would be great! Right now, I was largely thinking tracking publications that use FIMS would be a good step forward. Secondarily, we could track model type/use.