Skip to content

[BUILD] Generate synthetic test data: text files and pattern sets #4

Description

@hubertwojcik

Goal

Create scripts that generate reproducible synthetic text files and pattern sets of varying sizes for all benchmarking experiments.

Tasks

  • Write scripts/gen_text.py: generate random text files of configurable size (10 MB / 100 MB / 1 GB)
  • Write scripts/gen_patterns.py: generate random pattern sets of configurable count (10 / 100 / 1000 patterns) and length
  • Seed RNG for reproducibility; document seed in benchmarks/README.md
  • Add a make data target that generates the standard benchmark dataset

Acceptance Criteria

  • Scripts generate files of the exact requested size
  • Pattern files contain no duplicates
  • Generated data is excluded from git via .gitignore; generation is fully reproducible from seed

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    benchmarkingPerformance measurement, throughput, scalingbuildCMake build system and toolchain configuration

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions