The overlap corrected signal in PicassoPy often shows strange jumps along the time dimension. This is especially visible in the reprocessed case 25/08/2024 at Mindelo shown in the plot below.
The plot shows the overlap corrected range corrected signal at 355 nm total FR with the cloud free periods with a retrieved Raman overlap function marked in red. The second x-axis (on-top) shows the middle point of each marked cloud free period.
The reason for these jumps is that PicassoPy uses a varying overlap function for the high resolution (time, height) signal. Individual overlap functions are estimated per cloud free period, and then applied individually to each cloud free period. For the periods between (cloudy periods, or cloud free periods without a retrieved overlap function) the nearest overlap function is used. Thus the overlap correction applies different overlap functions at different times during the signal. If the estimated overlap function are not stable this solution will cause jumps in the overlap corrected signal.
The individual used overlap functions per cloud free period is shown in the plot below. As these are retrieved through the Raman method they at least show values strictly between 0 and 1. Overlap functions retrieved through the FRNR method often includes values outside this range. However, the height of the full overlap changes dramatically between the estimated function. This is especially clare for cloud free period 12 between 19:23 - 20:23 UTC which causes the big jump we see at this time in the overlap corrected signal.
When looking at the pure RCS profiles of each cloud free period we can see that most of them seams to be cloud contaminated. There are several heigh altitude clouds (above our search range of 7km) that goes undetected through the cloud screening module. This can explain the strange overlap functions.
This solution of using varying overlap functions is inherently prone to this kind of jumps. when there are slight changes in the overlap functions. After going through the recent reprocessed cases form the month of August 2024 in Mindelo. I can say that this jumps can be seen quit frequently, though not always as strong as in the case above. Thus the question is how can we avoid this jumps. Only applying one overlap function for the whole signal would fix the issue, but how do we choose which overlap function to apply?
The Matlab versions only applies one overlap function for the whole signal. However, the overlap function applied in the Matlab version is calculated based on the sum of the signal at all cloud free timestamps. I do not know if this solution is more favourable then the current one implemented in PicassoPy.
The overlap corrected signal in PicassoPy often shows strange jumps along the time dimension. This is especially visible in the reprocessed case 25/08/2024 at Mindelo shown in the plot below.
The plot shows the overlap corrected range corrected signal at 355 nm total FR with the cloud free periods with a retrieved Raman overlap function marked in red. The second x-axis (on-top) shows the middle point of each marked cloud free period.
The reason for these jumps is that PicassoPy uses a varying overlap function for the high resolution (time, height) signal. Individual overlap functions are estimated per cloud free period, and then applied individually to each cloud free period. For the periods between (cloudy periods, or cloud free periods without a retrieved overlap function) the nearest overlap function is used. Thus the overlap correction applies different overlap functions at different times during the signal. If the estimated overlap function are not stable this solution will cause jumps in the overlap corrected signal.
The individual used overlap functions per cloud free period is shown in the plot below. As these are retrieved through the Raman method they at least show values strictly between 0 and 1. Overlap functions retrieved through the FRNR method often includes values outside this range. However, the height of the full overlap changes dramatically between the estimated function. This is especially clare for cloud free period 12 between 19:23 - 20:23 UTC which causes the big jump we see at this time in the overlap corrected signal.
When looking at the pure RCS profiles of each cloud free period we can see that most of them seams to be cloud contaminated. There are several heigh altitude clouds (above our search range of 7km) that goes undetected through the cloud screening module. This can explain the strange overlap functions.
This solution of using varying overlap functions is inherently prone to this kind of jumps. when there are slight changes in the overlap functions. After going through the recent reprocessed cases form the month of August 2024 in Mindelo. I can say that this jumps can be seen quit frequently, though not always as strong as in the case above. Thus the question is how can we avoid this jumps. Only applying one overlap function for the whole signal would fix the issue, but how do we choose which overlap function to apply?
The Matlab versions only applies one overlap function for the whole signal. However, the overlap function applied in the Matlab version is calculated based on the sum of the signal at all cloud free timestamps. I do not know if this solution is more favourable then the current one implemented in PicassoPy.