Segmentation model implementation speedup ideas (model.predict is slow)

[ddbuscombe]

At the moment, Unet models are called in a loop, one image at a time. Model inference time reported by keras is approximately constant, but the overall time per image increases steadily. Over the course of ~1000 images, the slow-down is about 4x - 5x, on Windows and Linux.

This is not a priority right now, but should be fixed eventually. There are at least two aspects

1. model inference time. what is the quickest way to get the label? Should we be batching all images together, or continue to use a batch size of 1?
2. file I/O and other operations - is there a memory leak causing the slowdown?

Would this eventually be solved by switching to dask?

Leaving this issue here to be looked at later

[2320sharon]

Shoreline Extraction Process - Performance Issues

Overview

We are experiencing slowdowns in our shoreline extraction process. Below are the key differences in our approach compared to the standard CoastSat method, which we believe might be contributing to these performance issues.

Key Differences

1. Use of Dask: Our implementation heavily relies on Dask, which might not be ideal due to the intensive I/O operations involved in reading and creating files.

2. Handling of NPZ Files:

   • Reading NPZ Files: Unlike CoastSat, which generates predictions on the fly, our process involves reading predictions from NPZ files for each image.
   • Merging Labels from NPZ Files: We are also merging labels from these NPZ files, though this step's impact on performance is unclear.

The remaining steps are largely similar to the CoastSat approach and are not detailed here.

Planned Improvements

Please try the following to see if they cause an performance increases in the shoreline extraction process for the zoo workflow.

Tasks

• Remove Dask from the shoreline extraction process

  • I believe we could definitely improve this process by removing dask because its adding overhead that we can't take advantage of because of the heavy I/O

• Run the model on the images and extract shorelines at the same time like the coastsat process

  • We might also want to consider generating predictions on the fly like coastsat does to extract shorelines. That way we no longer need to read from the NPZ file and it reduces the number of steps the user has to go through to extract shorelines.

[ddbuscombe]

Okay, there are two problems being articulated here. I opened the issue with the doodleverse/do_seg or model.predict() performance, i.e., the process that generates the npz files.

I think you are referring to the subsequent process of using the npz files (image segmentation outputs) from shoreline extraction.

My focus right now is how very slow in general I made the doodleverse-utils do_seg style of model inference. The reason initially for the complex code with the need to resize imagery, and apply image standardization, neither of which at the time had a good keras implementation.

However now I see that keras.utils has image_dataset_from_directory so imagery can be called using

dataset = tf.keras.utils.image_dataset_from_directory(
    folder,
    labels=None,
    color_mode="rgb",
    batch_size=32,
    image_size=TARGET_SIZE,
    shuffle=False,
    seed=None,
    validation_split=None,
    subset=None,
    interpolation="bilinear"
)

and standardization cam be applied using

normalization_layer = tf.keras.layers.Normalization()

https://www.tensorflow.org/api_docs/python/tf/keras/layers/Normalization

This layer will shift and scale inputs into a distribution centered around 0
with standard deviation 1. It accomplishes this by precomputing the mean and variance of the data,
and calling (input - mean) / sqrt(var) at runtime.

which finally I figured out could be applied using .map

normalized_ds = dataset.map(lambda x: (normalization_layer(x)))

Now we can call the mode.predict step on all the jpeg files using all cores like so

predictions = model.predict(normalized_ds, workers=-1)

:)

Now I need to figure out how to implement it in ensemble model, etc, and how to get it in the doodleverse, but I just tested it on 1175 images and it worked great

[ddbuscombe]

This approach would be adapted to NDWI and MNDWI jpegs using color_mode="gray"

[ddbuscombe]

I think maybe this issue should be in doodleverse_utils. Ultimately, I think you just need to call do_seg from doodleverse-utils

[2320sharon]

@dbuscombe-usgs
This is a major win! I'm so glad there is a native way to do image preprocessing now in keras.
So it sounds like we can keep most of the zoo workflow in CoastSeg intact it just means when the the new do_seg function is ready we will update the version of doodleverse_utils required and it should run faster.