develop method to display structured metadata in docs sites

[weblorin]

Goal

The goal of the task is to take structured metadata and display it in user-friendly format on the web page in our docs sites (primarily but not exclusively tools.riverscapes.net).

Output

The tools "data" pages provide a fairly good example of what the final output will look like:

https://tools.riverscapes.net/vbet/data
https://tools.riverscapes.net/anthro/data
https://tools.riverscapes.net/brat/data

Notice how all the pages include:

1. Table of input layers and descriptions, with data type and source
2. List of Intermediates and Outputs layers and descriptions
3. Table of attributes (aka fields or columns) with units and descriptions for output layers

We can make changes and improvements as we build this programmatically, rather than by hand. For example

1. We have text at the top that includes the tool schema version
2. We could put the column attributes inline with the layers, but collapsed by default
3. There are additional, optional, attributes that we might include for both layers and columns, such as 'friendly name' and data type for attributes

Input

The new ultimate source for this information is a json file, one per tool, that is part of the tool code repository in git. Here is an example for VBET in feature branch.

• The schema for this file is published at xml.riverscapes.net/riverscapes_metadata/schema/layer_definitions.schema.json.
• We have a mechanism to 'harvest', ie find, parse, and transform all the layer_definitions.json files into a single database table in Athena. This can occur automatically as a Github action triggered by commits to a specified branch. The table is partitioned by repository (ie tools.riverscapes.net, we call this the authority), the tool name (authority name) and tool schema version.
• As part of this action, or separately, we can write additional scripts (e.g. python, sql, shell) to transform this data into another format/structure and/or load it to another location (e.g. a docs folder) for loading the purpose of making it into a Docusaurus friendly output. There is no need for a front end / react to do any heavy data transformations.

Process

There are numerous questions that need to be decided on, so the first task is to consider options. The following was generated by AI in response to search "data-driven docusaurus pages':

Data Driven Docusaurus pages

In Docusaurus, "data-driven pages" refers to generating content dynamically from external data sources (like JSON, CSV, or APIs) rather than writing static Markdown files. This is achieved using MDX and custom React components within your Docusaurus site during the build process or client-side.

Methods for Data-Driven Pages

There are two primary approaches to creating data-driven pages:
During the Build (Recommended): This approach generates static HTML files at build time, which is better for performance and SEO. This typically involves using Docusaurus plugins or Node.js scripts that read your data and programmatically create Markdown or MDX files in your docs directory.

Client-Side: You can fetch data using standard JavaScript within custom React components after the page has loaded in the user's browser. This is useful for content that updates frequently.

Resources

All this is on branch vbet_newlayerdescriptions