Hjm/doc 1.x to 2.x

[HectorJM2018Cosmo]

cf. https://cosmo-tech.atlassian.net/browse/PROD-16391

[Copilot]

Pull request overview

Updates the tutorial documentation and example scripts to align with the CoAL / csm-data 2.x API surface (notably the new cosmotech_api.apis.* wrappers and Parquet support), and adjusts CLI/tutorial workflows from scenario-run concepts to runner/run data concepts.

Changes:

• Added a Parquet datastore tutorial example and referenced it from the datastore docs.
• Migrated CosmoTech API tutorial scripts/docs from legacy helper functions (get_api_client, workspace/runner helpers, TDL helpers) to Connection + WorkspaceApi/RunnerApi/DatasetApi wrappers.
• Updated csm-data tutorial scripts and integration JSON to use run-load-data / runner terminology and simplified flags.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
tutorial/datastore/parquet_files.py	New tutorial script demonstrating loading/exporting Parquet with the datastore
tutorial/csm-data/run_load_data.bash	Updates CLI example invocation (removes older flags)
tutorial/csm-data/csm_orc_integration.json	Updates ORC integration step IDs/commands for runner data
tutorial/csm-data/complete_pipeline.bash	Updates pipeline script to the new run-load-data invocation shape
tutorial/csm-data/api_env_variables.bash	Switches env var example from scenario ID to runner ID
tutorial/cosmotech-api/workspace_operations.py	Migrates workspace tutorial to WorkspaceApi wrapper
tutorial/cosmotech-api/twin_data_layer.py	Updates TDL tutorial to use raw SDK TwinGraphApi via Connection
tutorial/cosmotech-api/runner_operations.py	Migrates runner tutorial to RunnerApi + Configuration
tutorial/cosmotech-api/connection_setup.py	Migrates connection tutorial to Connection object
tutorial/cosmotech-api/complete_workflow.py	Refactors complete workflow tutorial to new wrappers/configuration approach
docs/tutorials/datastore.md	Adds Parquet to datastore docs + includes new tutorial script
docs/tutorials/csm-data.md	Updates csm-data store capability list (incl. Parquet/output/delete)
docs/tutorials/cosmotech-api.md	Reworks CosmoTech API tutorial docs to new wrappers and dataset section
docs/csm-data/store/output.md	New command-help include page for csm-data store output
docs/csm-data/store/load-parquet-folder.md	New command-help include page for csm-data store load-parquet-folder

Suppressed comments (2)

tutorial/cosmotech-api/complete_workflow.py:94

• DatasetApi.upload_dataset() is being called with keyword arguments (organization_id, dataset_id, file_path) that do not match CoAL’s DatasetApi.upload_dataset(dataset_name, as_files, ...) signature. This will raise a TypeError at runtime and won’t update the existing dataset.

    dataset_api = DatasetApi(config)
    dataset_api.upload_dataset(
        organization_id=organization_id,
        dataset_id=output_dataset_id,
        file_path=str(processed_file),
    )

tutorial/cosmotech-api/complete_workflow.py:67

• If customers.csv exists but is empty (or only has headers), customers will be empty and customers[0] will raise IndexError (and later divisions by len(customers) will fail). Add a guard before using customers[0].

processed_file = processed_dir / "customers_with_loyalty.csv"
with open(processed_file, "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=list(customers[0].keys()))
    writer.writeheader()
    writer.writerows(customers)

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.