Skip to content

Reproducibility & data-transparency evaluation β€” feedback welcomeΒ #10

Description

@fractastical

Hi πŸ‘‹ β€” I'm building BioEval, an open evaluation of publicly available insect colony / swarm simulations, scoring each on data transparency, code availability, and computational reproducibility. Your project was included, and I wanted to share the results directly and β€” more importantly β€” ask for your feedback. If I've gotten something wrong or missed context, I'd genuinely like to correct it.

Overall: 64.3/100

Dimension Score
Data Disclosure 82
Dataset Resolvability 75
Code Availability 72
Traceability 48
Simulation Clarity 52
Reproducibility Package 58

(Scores include a documented +20 baseline calibration, capped at 100; the overall is the weighted mean of the six dimensions.)

Ways to improve the score:

  • Map algorithmic choices back to their sources to raise traceability
  • Add a versioned release / pinned commit / Zenodo archive
  • General: pin dependency versions + add a deterministic seed; add a parameters table mapping each constant to its source; tag a release and archive to Zenodo for a DOI; commit one example output with a checksum.

Methodology and full rubric: https://github.com/fractastical/bioinformatics-eval

Thanks for building and open-sourcing this β€” any corrections welcome!

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions