Should ExtData have an option to set an offset of the clock DT rather than ISO duration

[bena-nasa]

When trying to trace problem in GAAS (which we eventually traced to the fact that no one ever updated the files in the AMIP directory there for GAAS to correctly function) I remembered I have to set an offset in the ExtData file for GAAS that is the model heartbeat like so as in this example:

Collections:
  aod_a.sfc.%y4%m2%d2_%h2%n2z.nc4:
   template: das.aod_a.sfc.%y4%m2%d2_%h2%n2z.nc4
   freq: PT3H

Samplings:
  GAAS_sample_0:
   update_offset: PT450S
   exact: True

Exports:
  aod_a:
    collection: aod_a.sfc.%y4%m2%d2_%h2%n2z.nc4
    sample: GAAS_sample_0
    variable: AOD
    fail_on_missing_file: false

The reason for this is that GAAS runs 1 heartbeat before the start of the hour. So if you start at 21z, GAAS runs 1 heartbeat before 0z, 3z, 6z, etc. But the files GAAS needs are timestamped on the hours, so essentially GAAS needs the file timestamped 1 heartbeat in the future.
Notice that I have to set an offset, and that offset needs to be the heartbeat of the clock and apparently when this file was made it was for a 7.5 minute heartbeat. So this is an accident waiting to happen, if the user changes the heartbeat in CAP.rc, this needs to change too.

I'm thinking we have some notion of a HEARTBEAT you specify for that offset frequency. So rather than an ISO duration you could alternatively specify a number of hearbeats of the clock. That fixes this accident waiting to happen.

[tclune]

I think I understand. I'm wondering if it should be in terms of DT of the component as opposed to the heartbeat itself? Current case is presumably degenerate, but you might want to consider what happens if GAAS runs at a different time step size than the rest of the model.

[bena-nasa]

@tclune In MAPL2 every component has the same heartbeat as far as the actual clock since they are all sharing the clock. The component DT defines an alarm a component can choose to do something with or not, but ExtData has no way knowing this, nor would it be appropriate even if it could for this particular use case. The files and when GAAS actually runs are out of sync by a single heartbeat of the clock created by CAP.

[tclune]

Yes, but they do not all have the same DT. Hence my point. Bill runs a number of components at a different frequency.

Presumably you are saying that even if GAAS were to run say twice as often, you still want the offset to be heartbeat. Which is fine, just wanted to be certain that you'd thought it through. I can imagine vaguely analogous situations where we'd want to use DT as a multiplier/offset.

[bena-nasa]

@tclune, nothing can run a finer timescale than the heartbeat of the clock as created by Cap in MAPL2 as far as ExtData is concerned. So GAAS would never running 2x more than this. Besides ExtData does not have access to any of the components run DT's which once again whether or not they are used at the mercy of the component writers in MAPL2. It doesn't matter what they are doing, if it say ExtData knew the DT of component foo, it doesn't matter. The component writer of foo could have not used this alarm in foo so it doesn't tell ExtData anything as far as what is really happening. Since everyone is not going through a generic run no point in worry in MAPL2 about any component DT. ExtData can see the only clock provided, that's it.

Also remember it is far worse. GAAS actually does work based on if the fields provided by ExtData are not all MAPL_UNDEF which only happens when you are exactly the file time, that's what the exact: true is for in the sampling definition. If you are not at an exact file time, the field is filled with MAPL_UNDEF Because was pretty much the only way forward when we had that emergency when GAAS was modified to use ExtData without consultation. So as far as GAAS is concerned this offset has to be 1 DT.

[bena-nasa]

@tclune case in point, someone added a GAAS_DT to AGCM.rc , but GAAS never checks the run alarm, so not sure why one bothered adding this to the AGCM.rc. Certainly not because they wanted to use the run alarm created in the component...

[mathomp4]

Just as a note. I talked with @vbuchard and if need be, I don't think it would be too hard to do run-time detemplating of the GAAS control yamls where we grab the DT: from CAP.rc and then shove that into the yaml.

But my guess is @bena-nasa when he has the time could add a toggle in ExtData2G with like offset_by_heartbeat: true or whatever.

[bena-nasa]

Unfortunately this might be worse, apparently @wmputman was running chemistry at a different heartbeat from the model as reported to me by @vbuchard so it may not be a single heartbeat of the model clock. This apparently was breaking GAAS since now if the yaml file has the cap heartbeat and chemistry running at a longer heartbeat he did not see GAAS being triggered, I need to see what the offset needs to be in the yaml file. Is 2 hearbeats, if that is the frequency of chemistry, I don't know I will find out if I can figure out how to run GAAS. This becomes hard to handle outside of scripting since ExtData for good reason only has one clock and that's caps.

Of course the real issue is that GAAS SHOULD NEVER HAVE BEEN CONVERTED TO USE EXTDATA IN THE FIRST PLACE WITH NO CONSULTATION OR TESTING. The logic GAAS was using to decide to run (do I have a file) was simply something that ExtData had no way to transmit so required hoops.

Anything that needs to read specific files at specific times that is run to run dependent, like I need to read say my assimilation files for the first 5 hours of a run, or 2 files in the assimilation period, that is not what ExtData was ever meant for

[bena-nasa]

@wputman @vbuchard Yep, looks like you need to set the offset in the GAAS extdata.yaml file to the CHEMISTRY_DT rather than the heartbeat. Unfortunately since ExtData has no way to ever detecting this, this will have to be a scripting rather than code solution

[mathomp4]

@wputman @vbuchard Yep, looks like you need to set the offset in the GAAS extdata.yaml file to the CHEMISTRY_DT rather than the heartbeat. Unfortunately since ExtData has no way to ever detecting this, this will have to be a scripting rather than code solution

This shouldn't be too hard to do. I have a quick possible code change here:

GEOS-ESM/GEOSgcm_App#647

We can test it on Monday