Net2Cog fails on MODIS V7 MOD10A1 data due to the presence of both _FillValue and missing_value attributes. This results in the message: "ValueError: Variable None has conflicting _FillValue (255) and missing_value (200). Cannot encode data." Indeed, these attribute settings can be seen in the source NetCDF (HDF-5) data file.
Unfortunately, the CF and NetCDF conventions do not explicitly state this is invalid, leaving this use-case as an open possibility. These are apparently distinct issues in processing the MODIS V7 data that the data producers decided to capture in this way.
The GeoTIFF/COG format is less friendly on these metadata details, only allowing for a single "missing_value" setting. Many applications, and I expect Rio_Xarray is one of these, convert the _FillValue attribute to missing_value when writing to GeoTIFF format. Clearly, this causes an issue if the NetCDF data also has a missing_value.
I think this might be an issue with Rio_Xarray not handling this situation well, as it can certainly happen, and some response other than failure should be available. E.g., the missing_value could be changed to _FillValue in the data array, and this _FillValue written as the missing_value attribute. There might need to be some options in how to handle this case. However, I'm not sure we can expect Rio_Xarray to make this change, or to do so quickly. In particular, we need some solution for MODIS V7 data.
Net2Cog fails on MODIS V7 MOD10A1 data due to the presence of both _FillValue and missing_value attributes. This results in the message: "ValueError: Variable None has conflicting _FillValue (255) and missing_value (200). Cannot encode data." Indeed, these attribute settings can be seen in the source NetCDF (HDF-5) data file.
Unfortunately, the CF and NetCDF conventions do not explicitly state this is invalid, leaving this use-case as an open possibility. These are apparently distinct issues in processing the MODIS V7 data that the data producers decided to capture in this way.
The GeoTIFF/COG format is less friendly on these metadata details, only allowing for a single "missing_value" setting. Many applications, and I expect Rio_Xarray is one of these, convert the _FillValue attribute to missing_value when writing to GeoTIFF format. Clearly, this causes an issue if the NetCDF data also has a missing_value.
I think this might be an issue with Rio_Xarray not handling this situation well, as it can certainly happen, and some response other than failure should be available. E.g., the missing_value could be changed to _FillValue in the data array, and this _FillValue written as the missing_value attribute. There might need to be some options in how to handle this case. However, I'm not sure we can expect Rio_Xarray to make this change, or to do so quickly. In particular, we need some solution for MODIS V7 data.