Skip to content

Reintegrate as a coral-compatible backend for the DealiiX platform #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pogmat merged 8 commits into from
Jul 13, 2026
Merged

Reintegrate as a coral-compatible backend for the DealiiX platform #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
ef408a9
#12 Chore: adopt uv project layout and document it
pcolt Jul 7, 2026
33deec4
#12 Feat: coral-compatible CLI (register/run) + launcher
pcolt Jul 7, 2026
d755034
#12 Feat: emit platform-native node registry
pcolt Jul 8, 2026
c02ea77
#12 Feat: type-based executor dispatch; drop method_name from the pro…
pcolt Jul 8, 2026
16b1bfe
#12 Refactor: drop is_valid from the registry (parity with C++ coral)
pcolt Jul 8, 2026
80f616c
#12 Docs: restore one_obstacle example and fix module-registration docs
pcolt Jul 9, 2026
0cd3bad
#12 Docs: add onboarding guide (EN + IT) and reference it from README…
pcolt Jul 9, 2026
f7b7c4c
#12 Docs: document ffmpeg prerequisite for mp4 export
pcolt Jul 10, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
135 changes: 72 additions & 63 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,55 +12,58 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co

The project uses PhiFlow for physics simulations and numerical computing.

> For the narrative — goals, architecture rationale, the two contracts with the DealiiX platform, and how to
> extend the project — see [`docs/ONBOARDING.md`](docs/ONBOARDING.md) (Italian: `docs/ONBOARDING.it.md`). This
> `CLAUDE.md` is the mechanics reference; `README.md` covers setup and commands.

## Development Commands

### Environment Setup
```bash
# Create and activate virtual environment
uv venv
source .venv/bin/activate # Linux/Mac

# Install dependencies using uv pip sync (recommended)
uv pip sync requirements.txt

# Or install directly
uv pip install -r requirements.txt
```
This is a uv project (`pyproject.toml` + `uv.lock`); `requirements.in` / `requirements.txt` are legacy.

### Package Management
```bash
# Add a new package (recommended workflow)
echo "<package-name>" >> requirements.in
uv pip compile requirements.in -o requirements.txt
uv pip sync requirements.txt

# Alternative: direct install (must still update requirements.txt)
uv pip install <package-name>
uv pip freeze > requirements.txt
# Create .venv and install all deps (incl. the dev group) from the lockfile
uv sync

# Then activate, or prefix commands with `uv run` (e.g. `uv run python main.py`)
source .venv/bin/activate # Linux/Mac
```

### Running Workflows
### Package Management
```bash
# Execute default workflow (network-from-fe.json) with default modules
python main.py
# Add a runtime dependency (updates pyproject.toml + uv.lock, then syncs)
uv add <package-name>

# Execute specific workflow file
python main.py path/to/workflow.json
# Add a dev-only dependency
uv add --dev <package-name>

# Execute with specific modules loaded
python main.py path/to/workflow.json --modules="math"
python main.py path/to/workflow.json --modules="math,string,phiflow"
# Re-resolve the lockfile and sync the environment
uv lock && uv sync
```

# Generate registry from selected modules
python main.py --generate-registry
python main.py --generate-registry --modules="math"
python main.py --generate-registry --modules="math,string,phiflow"
### Running Workflows

# Generate registry to custom file
python main.py --generate-registry --registry-output="custom-registry.json" --modules="math"
`main.py` is a coral-compatible CLI: a global `-p/--plugin` option (comma-separated modules; empty = all) plus
`register` / `run` subcommands. `-p/--plugin` must precede the subcommand.
```bash
# Run a workflow graph (default file network-from-fe.json, all modules)
python main.py run
python main.py run path/to/workflow.json
python main.py -p "math" run path/to/workflow.json
python main.py -p "math,string,phiflow" run path/to/workflow.json

# Generate the node registry (writes node_types.json into the cwd)
python main.py register
python main.py -p "math" register
python main.py register --output="custom-registry.json"
```

**Default module behavior**: When no `--modules` flag is provided, only the `phiflow` module is loaded (optimal for physics simulations). Primitives are always included.
The `coral-py` launcher wraps this for the DealiiX platform: it runs `main.py` inside the uv project while
preserving the caller's cwd (so `register` writes `node_types.json` there). Point the platform's `coralBinaryPath`
at `coral-py` and set `coralPluginPath` to the module list.

**Default module behavior**: When `-p/--plugin` is omitted, all available modules are loaded. Primitives are always included.

**Available modules**:
- `math` - Mathematical operations and Calculator class
Expand Down Expand Up @@ -117,7 +120,7 @@ The system is organized into four main modules:
- **`definitions/string_ops.py`**: `StringProcessor` class and `print_result` function
- **`definitions/phiflow_defs.py`**: PhiFlow physics simulation wrappers (if available)

**Module loading**: Use `--modules` flag to control which definitions are loaded. Default: `phiflow` only.
**Module loading**: Use the `-p/--plugin` option to control which definitions are loaded (comma-separated). Default when omitted: all modules. `AVAILABLE_MODULES` (in `definitions/__init__.py`) lists them.

2. **registry.py** - Registry generation and type conversion:
- `generate_registry()`: Introspects functions and classes to create JSON schema
Expand All @@ -136,29 +139,35 @@ The system is organized into four main modules:
- Calling instance methods via method nodes
- Storing results for downstream nodes

4. **main.py** - CLI entry point with argparse interface:
- Accepts `--modules` flag to control which definition modules are loaded
- Passes module list to both `WorkflowExecutor` and `save_registry_to_file()`
4. **main.py** - Coral-compatible CLI entry point (argparse):
- Global `-p/--plugin` option names the definition modules to load (comma-separated; empty = all)
- `register` subcommand → `save_registry_to_file()` (writes `node_types.json` into the cwd)
- `run` subcommand → `WorkflowExecutor(...).execute()`
- Wrapped by the `coral-py` launcher for the DealiiX platform

### Workflow JSON Structure

**Network Files** (e.g., `network-from-fe.json`):
Located at: `workflow.nodes` and `workflow.edges`

Node types:
- Primitive: `{"value": <val>, "node_type": "primitive", "type": "<type>"}`
- Function: `{"node_type": "function", "method_name": "<func_name>"}`
- Constructor: `{"node_type": "constructor", "type": "<ClassName>"}`
- Method: `{"node_type": "method", "method_name": "<ClassName>.<method_name>"}`
Nodes are **lean**: each carries only its `type` (plus `value` for primitives); the executor infers
the kind from `type`, so `node_type`/`method_name` are not part of the graph.
- Primitive: `{"type": "<type>", "value": <val>}`
- Function: `{"type": "<func_name>"}`
- Constructor: `{"type": "<ClassName>"}`
- Method: `{"type": "<ClassName>.<method_name>"}`

Edge format:
- `{"source": "<source_id>", "target": "<target_id>", "source_output": <idx>, "target_input": <idx>}`
- **CRITICAL**: `target_input` determines parameter ordering for function/method calls

**Registry Files** (e.g., `registry-py.json`):
- Auto-generated schema describing all available node types
- Auto-generated schema describing all available node types, in the DealiiX platform format
- **Keyed by each node's `type`** (primitives by type name, functions by name, constructors by class
name, methods by `Class.method`) — the editor looks entries up as `registry[type]`
- Each entry has:
- `arguments`: Array with `connection_type` ("input"/"output") and `type`
- `type`: the node type string (equals the entry's key)
- `arguments`: Array with `connection_type` ("input"/"output"), `type`, and `name` (empty `[]` for primitives)
- `inputs`: List of input indices
- `outputs`: List of output indices (or `[-1]` for constructors/primitives)
- `node_type`: "primitive", "function", "constructor", or "method"
Expand All @@ -176,27 +185,28 @@ Edge format:

### Node Execution Model

Each node type follows a specific execution pattern in [executor.py](executor.py):
Each node's kind is inferred from its `type` by `WorkflowExecutor._classify()` (membership in
`PRIMITIVES_MAP` / `FUNCTION_MAP` / `CLASS_MAP`, plus the `Class.method` split), then executed:

**Primitive nodes** (`node_type: "primitive"`):
**Primitive nodes** (`type` in `PRIMITIVES_MAP`):
- Extract `value` and `type` from node definition
- Convert value using `PRIMITIVES_MAP[type]` (handles string-to-type conversion from JSON)
- Store result directly

**Function nodes** (`node_type: "function"`):
- Look up function in `FUNCTION_MAP` using `method_name`
**Function nodes** (`type` in `FUNCTION_MAP`):
- Look up function in `FUNCTION_MAP` using `type`
- Collect inputs from incoming edges sorted by `target_input` index
- Map inputs to function parameters positionally using `inspect.signature()`
- Execute function with kwargs, store result

**Constructor nodes** (`node_type: "constructor"`):
**Constructor nodes** (`type` in `CLASS_MAP`):
- Look up class in `CLASS_MAP` using `type` field
- Collect constructor inputs from edges (sorted by `target_input`)
- Map inputs to `__init__` parameters (excluding `self`)
- Instantiate class, store instance

**Method nodes** (`node_type: "method"`):
- Parse fully qualified `method_name` (format: `"ClassName.method_name"`)
**Method nodes** (`type` is `Class.method` with the class in `CLASS_MAP`):
- Parse the fully qualified `type` (format: `"ClassName.method_name"`)
- First input (lowest `target_input`) must be instance of `ClassName`
- Remaining inputs are method parameters
- Use `getattr(instance, method_name)` to get bound method
Expand Down Expand Up @@ -260,7 +270,7 @@ def get_functions() -> Dict[str, Any]:

4. **Regenerate registry with the appropriate module**:
```bash
python main.py --generate-registry --modules="math"
python main.py -p "math" register
```

5. The function is now available when the module is loaded
Expand Down Expand Up @@ -291,18 +301,17 @@ def get_classes() -> Dict[str, Any]:
return {}
```

Then register the module in `definitions/__init__.py`:
Then register the module in `definitions/__init__.py` — add the import and one entry to `_MODULES`
(`build_function_map`/`build_class_map` read from `_MODULES`, so no other edit is needed):
```python
from . import math_ops, string_ops, phiflow_defs, primitives, numpy_ops

def build_function_map(include=None, exclude=None):
modules = {
'math': math_ops,
'string': string_ops,
'phiflow': phiflow_defs,
'numpy': numpy_ops, # Add here
}
# ... rest of implementation
_MODULES = {
'math': math_ops,
'string': string_ops,
'phiflow': phiflow_defs,
'numpy': numpy_ops, # Add here
}
```

**Why wrappers are needed:**
Expand Down Expand Up @@ -335,7 +344,7 @@ def get_classes() -> Dict[str, Any]:

After updating the module, regenerate the registry:
```bash
python main.py --generate-registry --modules="math"
python main.py -p "math" register
```

**How it works:**
Expand Down
105 changes: 67 additions & 38 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,70 +4,80 @@ Coral for python libraries
## Installation

### Prerequisites
- Python 3.6+
- [uv](https://github.com/astral-sh/uv) installed
- Python 3.12+
- [uv](https://github.com/astral-sh/uv) installed
- `ffmpeg` installed and on `PATH` (e.g. `apt install ffmpeg` / `brew install ffmpeg`) — required for `.mp4`
export from the PhiFlow scripts and the `phiflow_plot_and_save` workflow node, which call matplotlib's
`anim.save(..., writer='ffmpeg')`. Not needed for `.gif` export.

## Setup

```bash
# Create virtual environment
uv venv

# Activate virtual environment
source .venv/bin/activate # Linux/Mac
# .venv\Scripts\activate # Windows
This is a [uv project](https://docs.astral.sh/uv/concepts/projects/): dependencies are declared in
`pyproject.toml` and pinned in `uv.lock`.

# Install dependencies without extras
uv pip sync requirements.txt
```bash
# Create .venv and install all dependencies (incl. the dev group) from the lockfile
uv sync
```

### Managing Dependencies
Then either activate the environment (`source .venv/bin/activate`) or prefix commands with `uv run`
(e.g. `uv run python main.py`). `uv run` auto-syncs the environment against `uv.lock` before running.

#### Install new packages with [uv pip compile](https://docs.astral.sh/uv/pip/compile/)
### Managing Dependencies

```bash
# Add the package to requirements.in

echo "jax" >> requirements.in
# Add a runtime dependency (updates pyproject.toml + uv.lock, then syncs the env)
uv add <package-name>

#Recompile requirements.txt
uv pip compile requirements.in -o requirements.txt
# Add a dev-only dependency
uv add --dev <package-name>

# Sync your environment
uv pip sync requirements.txt
# Re-resolve / update the lockfile and sync the environment
uv lock
uv sync
```

> `requirements.in` / `requirements.txt` are retained for reference only; `pyproject.toml` + `uv.lock`
> are now the source of truth for dependencies.

## Usage

`main.py` is a **coral-compatible CLI**: a global `-p/--plugin` option naming the definition modules to load
(comma-separated, e.g. `"math,string"`; empty = all) plus two subcommands — `register` (emit the node registry)
and `run` (execute a workflow). `-p/--plugin` must precede the subcommand. This mirrors the C++ `coral` binary so
the DealiiX platform can drive this backend via the [`coral-py` launcher](#coral-launcher-for-the-dealiix-platform).

> Run the commands below inside an activated venv, or prefix each with `uv run` (e.g. `uv run python main.py run`).

### 1. Running a stand-alone Phi-flow simulation

```bash
# Run a simulation and then check the mp4 or gif file produced
python one_obstacle.py
python phi_flow/one_obstacle.py
```

### 2. Running the Workflow Executer

Run with default workflow file (network-from-fe.json):
Use the `run` subcommand. With no graph argument it defaults to `network-from-fe.json`:
```bash
python main.py
python main.py run
```

Run with a specific workflow file:
Run a specific workflow file:
```bash
python main.py path/to/your/workflow.json
python main.py run path/to/your/workflow.json
```

Run with specific modules loaded:
Load specific modules with `-p/--plugin` (before the subcommand):
```bash
# Load only math operations
python main.py workflow.json --modules="math"
python main.py -p "math" run workflow.json

# Load multiple modules
python main.py workflow.json --modules="math,string,phiflow"
python main.py -p "math,string,phiflow" run workflow.json
```

**Default behavior**: When no `--modules` flag is provided, only the `phiflow` module is loaded (optimal for physics simulations). Primitives are always included.
**Default behavior**: When `-p/--plugin` is omitted, all available modules are loaded. Primitives are always included.

**Available modules**:
- `phiflow` - PhiFlow physics simulation wrappers (default)
Expand All @@ -76,23 +86,34 @@ python main.py workflow.json --modules="math,string,phiflow"

### 3. Generating the Workflow Registry File

Generate the default registry file registry-py.json:
Use the `register` subcommand. It writes `node_types.json` into the current directory (the filename the
DealiiX platform probes for):
```bash
python main.py --generate-registry
python main.py register
```

Generate registry with specific modules:
Generate the registry for specific modules:
```bash
# Generate registry for math operations only
python main.py --generate-registry --modules="math"
# Math operations only
python main.py -p "math" register

# Generate registry for all modules
python main.py --generate-registry --modules="math,string,phiflow"
# Multiple modules
python main.py -p "math,string,phiflow" register
```

**Generate custom output path for registry file:**
**Custom output filename:**
```bash
python main.py --generate-registry --registry-output="custom_registry.json" --modules="math"
python main.py register --output="custom_registry.json"
```

### Coral launcher (for the DealiiX platform)

`coral-py` runs `main.py` inside this repo's uv project while preserving the caller's working directory, so
`register` writes `node_types.json` into that directory. Point the platform's `coralBinaryPath` at it and set
`coralPluginPath` to the module list:
```bash
./coral-py -p "math" register # writes node_types.json into the current directory
./coral-py -p "math" run workflow.json
```

### More info about the definitions package
Expand All @@ -103,6 +124,14 @@ See [README.md](definitions/README.md) in the `definitions` directory for more d
python main.py --help
```

## Development

Extending or modifying coral-python? Start with [`docs/ONBOARDING.md`](docs/ONBOARDING.md) — the onboarding
guide covering goals, architecture, the two contracts with the DealiiX platform, how to add a library or
change internals, design rationale, and an honest account of strengths and weaknesses. An Italian version is
at [`docs/ONBOARDING.it.md`](docs/ONBOARDING.it.md). (This `README.md` is setup + commands; `CLAUDE.md` is the
AI-assisted-development mechanics reference.)

## Testing

Run All Tests:
Expand Down
Loading