Is spatially parallel processing possible?

[TonioF]

This is related to #1 . Is the computation done pixel-by-pixel or are neighbouring pixels considered, too. I think the answer is that neighbouring pixels are considered by the atmospheric correction, but not for the others. Correct? @MarcYin , is there a way to make the atmospheric correction not use neighbouring pixels?

[jgomezdans]

Currently, individual pixels are retrieved, so splitting into non-overlapping spatial blocks results in a simple parallelisation strategy (we discussed this already in London). However, there's probably a need to add spatial constraints, in which case, the blocks could be designed as to overlap with a "seam area" to minimise sharp discontinuities. The size is of the seam area is a good question, by the way ;)

[MarcYin]

As I have commented in #1, it is possible, but generally, all the AtCor algorithm are tile based, considering which is the smallest unit for a global product. Another reason of using a larger extend instead of smaller area is that MCD43 (or simulated surface reflectance) may not be available for that subset area, in which case, neighbouring pixels information becomes important and making it still possible to do the inversion of atmospheric parameters.

[TonioF]

@jgomezdans : Spatial constraints? What do you have in mind there? Do you think of a kind of filter to smoothen along the "seams"? Having such an overlap is exactly what might have big implications on the orchestration, so we should be clear about this early.

[jgomezdans]

A spatial constraint is something that ties a the value of the state in one pixel with others. In the model used in KaFKA and MULTIPLY, this means that we have correlations between parameters in different pixels. While these correlations decay with distance (e.g. one pixel away at 10m we might expect a lot of correlation, 10km away we might assume that the correlation is negligible. For the atmospheric parameters we might say that (say) >100km is when we ignore spatial correlation). So, if you process two pixels independently on the edge of their respective ROI/state grid/mask/whatever, they will be different. If you have a long enough seam area, they will tend to be far more similar, as the differences in the inference are mostly governed by close neighbouring pixels. If the seam is "big enough", then the edge effect should be negligible.

Funky annoyingly rotated diagram: pixels A and B are neighbours and on the edge of each other's processing windows, W1 and W2 (respectively for A and B). As such, a spatial inference on A wouldn't see B, or any of the pixels around B, meaning that A could be very different to B just because of the edge. However, if you extend the windows to the dashed lines (W1' and W2'), they'd be mostly seeing the same local neighbourhood, and things ought to be safe.