Design a schema for meta.yaml

[jeanetteclark]

Along with our changes to the API, while we are in design mode we should also write a schema document for meta.yaml.

This schema ideally will:

• be consistent in naming with the ogdc-runner API
• enable features available in Argo that we don't yet (but might) support
• mirror the Argo worfklows spec reasonably closely to prevent confusion

I think to accomplish this we should consider rewriting the schema to allow for multiple workflow steps. For example, one might want to run both a shell step and a visualization step in the same recipe. We don't have to copy the Argo workflow specs exactly, but re-using the template param seems like it might make sense, where each template has a type that corresponds to the current workflow:type (shell, visualization).

Below is an example that is just an idea, would love to get feedback @rushirajnenuji @trey-stafford @rmarow

name: "ADC Visualization workflow"
version: 1.0.0

recipe:
  type: "steps"
  steps:
      - name: pre-process
        template: pre-process
      - name: data-visualization
        template: data-visualization
          # possibly specify input data here explicitly as the output data from pre-process step

templates:
  - name: pre-process
    type: shell
    input:
      datasets:
        - type: "url"
          value: "https://demo.arcticdata.io/tiles/OGDC/QGnet/viz-workflow/ice_basins.gpkg"
    output:
      - type: "temporary" # possibly allow for saving intermediate outputs?

  - name: data-visualization
    type: visualization
    parallel:
      enabled: true
      partition_strategy: "files"
      partition_size: 2
    output:
      type: "pvc"