Investigate performance and AC+AF cutoffs for LD Code

[matren395]

Find appropriate AC and/or AF cutoff for LD data , such that the code does not encounter out-of-memory issues.

Wenhan mentioned errors at 10 million variants per run (so per genetic ancestry group) but prior gnomAD v2 code mentioned the same at 30 million variants , both on standard workers ? where will I encounter this when running ?

And for cutoffs, we are running on both NFE and AFR. Is there the same cutoff that would get appropriate numbers (Wenhan ballparked 7-9million) for both ?

Relevant to #1656

[matren395]

this is continuing! in collaboration with Wenhan in Konrad's lab. She'll be continuing with most of the investigation on this (and how exactly to return more variants)

[matren395]

I've done a re-write of this code on a per-contig basis, to avoid this issue, and it seems to be moving forward under this framework ?

I'm not aware of the wheels ever getting turning on the other end of this to investigate why this is happening or to thoroughly debug this on the Hail end, but this also fulls on me