Alter operation grouping to minimise number of jobs

[MattWellie]

• the current approach is creating one or two additional QOB jobs per gene, loading the chromosome MT in and running gene-specific filtering
• this is causing some batch chaos

This change mirrors the approach here: https://github.com/populationgenomics/saige-tenk10k/blob/main/saige_assoc_set_test.py#L275-L282

• the only gene-specific arguments to make_group_file are the gene name and the output file, all other arguments are constant across the whole chromosome
• instead of starting a pythonJob per gene, group multiple genes into bigger jobs
  • we loop over all genes, and where an output file doesn't exist yet we add the gene name and output file to a list
  • once the list is long enough (default 50) we start a single job, passing all the arguments in once
  • make_group_file is edited to start a Hail runtime and open the MT once, then to loop over [gene, group_file], carrying out the existing process to generate group files.
  • if we reach the end of the gene loop and there are genes which weren't packaged into a job, shove them in a final job

Hopefully this brings down the number of jobs involved.

[Copilot]

Pull Request Overview

This PR refactors the gene group file generation process to reduce the number of batch jobs by grouping multiple genes into single jobs instead of creating one job per gene.

• Batches multiple genes (default 50) into single jobs to reduce batch system overhead
• Modifies the make_group_file function to process multiple genes in one execution
• Loads the chromosome matrix table once per job and processes all genes in that batch

---

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}