Bug : config.json overwritten when continuing a download

[2320sharon]

When a user loads an session with an ROI that has already been downloaded and they want to download an additional satellite, they may only select the satellite they want to download then click download imagery. If they do this the sat_list stored in the config.json will be overwritten to only include the satellite they selected. For example lets say the initial session had downloaded L5 and L8; the user loads it up and selected S2 from the drop down in settings, then click download imagery. This will overwrite the sat_list in config.json to be overwritten from [L5,L8] to [S2]. This will cause issues when extracting shorelines.

Additionally it should be checked that the dates are not being overwritten in the config.json file as well.

Tasks

• Separate the download settings from the loaded session settings to avoid this issue

If I can detect an existing session was loaded and extend the settings instead of overwriting them this would work.

[2320sharon]

I believe this issue stems from the update_roi_settings_with_global_settings which is called in the download_imagery and extract_all_shorelines functions. Basically if the user wants to only download/extract shorelines for certain group of satellites for instance this will overwrite the sat_list parameter with the smaller list of satellites.

Instead coastseg should make a copy of the ROI settings for extracting_shorelines/downloading imagery, modify the copy and NOT save the copy. Instead if a new satellite was downloaded it should add this to existing sat list OR for the date range being changed it should extend the dates to cover this range.

def update_roi_settings_with_global_settings(roi_settings: dict, global_settings: dict) -> dict:
    """
    Update the ROI settings with the global settings.

    Args:
        roi_settings (dict): A dictionary containing the ROI settings.
        global_settings (dict): A dictionary containing the global settings.

    Returns:
        dict: The updated ROI settings dictionary.
    """
    # get the sat_list and dates from the global settings
    sat_list = global_settings.get("sat_list", [])
    dates = global_settings.get("dates", [])
    # update the roi_settings with the global settings
    updated_roi_settings = update_roi_settings(roi_settings, "sat_list", sat_list)
    updated_roi_settings = update_roi_settings(updated_roi_settings, "dates", dates)
    return updated_roi_settings

[2320sharon]

To fix the existing broken sessions created from this issue we could read the min & max dates from the metadata files for each satellite and use this to update the dates parameter in the config.json. As for the satellite list just read the satellite folders from the ROI directory from /data.

Example Downloaded ROI Directory Structure in /data

In this example we can easily see that S2 and L9 were downloaded for this ROI

ID_wra5_datetime01-10-25__03_58_44
|_jpg_files
|_S2
|_L9

[2320sharon]

I think the root of much of the confusion comes from the fact its very difficult to tell whats a session's settings for extracting shorelines and whats downloaded data session settings. Downloaded data settings should NOT be modified generally, while a session's settings can be modified. I think I need to build something that manages both sets of settings and better separates them