Statefulness in AST to allow experiment tracking in Intake Catalog

[charles-turner-1]

          > @charles-turner-1 added my two cents on the updates. I think my previous requests were:

• Wondering about emission of warnings, notice to users etc.

• I've currently set the telemetry to be completely silent. No warnings, notices, info, print statements, etc. should escape from the telemetry and be shown in the notebook to the user. This seems to be roughly what was agreed on in this comment thread.
• I've added the relevant info to the docs.

• Ability to do statistics on a per-experiment basis

• Unfortunately, I think this is going to be difficult to do cleanly - at least with the way that I've currently built the telemetry. Consider the following situation:

expt_1 = "some_str"
expt_2 = "some_other_str"

import intake
catalog = intake.cat.access_nri
esm_ds1, esm_ds2 = catalog[expt1], catalog[expt2]

esm_ds1.search(variable="xyz", ...)
esm_ds2.search(variable="abc", ...)

As of right now, the telemetry uses some wacky AST chicanery to resolve that esm_ds1 and esm_ds2 are of type esm_datastore, and then it sends off telemetry records like:

{
    "timestamp": "2025-04-01T01:07:53.146786Z",
    "name": "unknown",
    "function": "esm_datastore.search",
    "args": [],
    "kwargs": {
        "variable": "xyz"
    },
    "session_id": "df07384b-acca-49e9-949c-dbef13b45562",
    "catalog_version": "v2025-03-04"
},
{
    "timestamp": "2025-04-01T01:07:53.146786Z",
    "name": "unknown",
    "function": "esm_datastore.search",
    "args": [],
    "kwargs": {
        "variable": "abc"
    },
    "session_id": "df07384b-acca-49e9-949c-dbef13b45562",
    "catalog_version": "v2025-03-04"
},

Without adding some more complicated state tracking to the AST NodeVistor/CallListener, it's going to be tough to know which is which. For example, the session_id field is a singleton, so we know it's unique to that interpreter - which removes the need for disambiguation.
The ApiHandler class in the telemetry module contains a last_call attribute, which we could extend to cache the last experiment loaded - but in the code block above, both searches would then point at expt_2.

I agree that it would be good to include per-experiment information, but I'm not quite sure how to do it yet.

My hope is that people will generally only want to interact with a single experiment per notebook, and as such, we could generally associate each session_id with a single call to DfFileCatalog.__getitem__, which means that we can treat that session_id as a proxy for the experiment we would find in the args field on the DfFileCatalog.__getitem__ call for that session_id.

TLDR; it's messy

Originally posted by @charles-turner-1 in ACCESS-NRI/access-nri-intake-catalog#312 (comment)

---

Feature to add

We need to extend the AST to be able to keep track of how certain types are initialised, and then attach that info to the telemetry we post if so desired.

[charles-turner-1]

This should become tractable by using AST rewriting, as introduced in #64, by replacing objects with their ID and or parent node in an expr and attaching that to telemetry calls.