_read_dice_outputs is very slow

[caleb-johnson]

Look for ways to speed this up.

Suggestions from user:

• parallelize loops in _read_dice_outputs (and/or whatever that function calls). Looking at the code this seems straightforward (here i just mean, to parallelize all the relevant loops on a single node, with multiprocess.Pool.map or some variant).
• see if we can speed up _read_wave_function_magnitudes and other functions that are called; i.e. get rid of list.appends (just preallocate mem and fill things in), and potentially parallelize loop(s) if limiting.

[ashsaki]

#79 can make it faster significantly.

The _read_dice_outputs() function has three main calls:

1. _read_wave_function_magnitudes(): I played with a Rust-Python where the binary file read operation was offloaded to Rust. It brought visible acceleration. However, two downsides of the approach:
   • My experiment was less memory-efficient. I loaded the whole binary file into the memory (could be few Gbs to 10s of Gbs) whereas current approach reads chunk-by-chunk (only a few bytes).
   • Current pure Python is easy to install and use compared to Rust/Python hybrid.
2. _ci_strs_from_occupancies(): Currently, it takes a long time. PR Faster CI strs from occupancies #79 speeds it up. In some earlier test cases, we observed an acceleration of from ~10 hours to ~30 minutes.
3. _construct_ci_vec_from_amplitudes(): We can merge some operations, i.e., we can create sci_coefficients, ci_strs_a, ci_strs_b while reading the binary file instead of multi-step approach: binary file -> occs, amps -> ci strs -> sci_coefficients, ci_strs_a, ci_strs_b (something like here).

Adopting only item "2" (major acceleration) above can make it fast enough such that we may not need to over engineer with items "1" (some acceleration) and "3" (minor acceleration).