Restore lost fixes - #67
Conversation
…cal variables CMIP7 region-selector fx files (basin, siline, similar) carry their data variable as a CF flag-valued integer (flag_values + flag_meanings mapping integer codes to region/section names). compliance-checker's is_geophysical excludes any variable with flag_meanings from the geophysical-variable set as a heuristic for status flags. The exclusion is right for QC flags but wrong for these region selectors, whose standard_name is "region" not "status_flag", and which ARE the file's data variable by CMIP7 design. The existing variable_id disambiguation in _get_geo_var only fires when CF returned multiple candidates. When CF returns zero (the basin/siline case) the function bails with "No geophysical variable detected in the file." at HIGH severity, which blocks ESGF publication. Extend the variable_id fallback to the zero-candidate branch. If CF detection finds nothing and the global variable_id attribute names an existing variable, accept that variable as the file's geophysical variable. The strict CF heuristic stays as the primary path; the fallback only kicks in when both (a) CF found zero candidates AND (b) the CMIP7-canonical variable_id attribute is present.
CMIP7 (and CMIP6 archive practice) ships sub-daily tavg files with midpointed time stamps. _is_instantaneous already handles tpt rules via cell_methods, so adding the bare freq names only changes behaviour for tavg sub-daily, in the direction the spec implies. Closes #51.
Closes #59. CMIP7 region-selector variables (basin, siline, similar) are CF flag- valued integers. The default geophysical_variable.toml rules describe a canonical continuous float field (data_type = float, _FillValue = 1e20, missing_value = 1e20), which cannot express the flag case and cannot be conditionalised in TOML. After #49 taught the plugin to recognise flag-valued variables as the file's geo var, five HIGH findings still fire on basin fx files: ATTR001 (missing) or ATTR002/ATTR004 (wrong type/constant) for both _FillValue and missing_value, plus VAR005 (int vs float type check). Adds _is_flag_variable(ds, var_name) helper and threads a single is_flag gate through check_Geophysical_Variable: - [variable.type] check skipped for flag variables. - Attribute rules whose attribute_name is _FillValue or missing_value skipped for flag variables. Existence, dimensions, shape, and all other attribute rules still apply. Non-flag variables unchanged. Verified against synthetic reproducer from #59 (5 -> 0 findings for _FillValue / missing_value / VAR005) and against real AWI-ESM3-4-2-veg- HR piControl 1851 basin fx file.
Confirmed, and thanks for picking this up. I had started a restoring PR from #66 before spotting this one; it turned out to be a strict subset (I had only the two Two things from my end that may be useful for the review. The regression reached the releases, not just
Verified on real data, not just the synthetic reproducer. A CMIP7 That is the only HIGH left in our current output, so this PR clears it. On the extra |
2 participants
This PR deals with the merged fixes that were overwritten as detailed in issue #66
I found that three fixes have disappeared (but a second pair of eyes conforming this would not hurt):
I also added the
decfrequency to theAVERAGE_CORRECTION_FREQs that are evaluated at the midpoint of the time interval, if nottime: pointis specified ascell_methods.closes #66