Enable versioning of agents by default with ability to disable for de…

.gitignore

@@ -6,4 +6,5 @@ logs/
 integration_tests/.env
 integration_tests/.user.yml
 integration_tests/dbt_packages/
-integration_tests/package-lock.yml
+integration_tests/package-lock.yml
+graphify-out

CHANGELOG.md

@@ -8,12 +8,22 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
 
 ### Added
+- `enable_versioning` config option — controls Snowflake Cortex Agent versioning; when `true` (default), uses `CREATE AGENT IF NOT EXISTS` + `ALTER AGENT MODIFY LIVE VERSION SET SPECIFICATION` instead of `CREATE OR REPLACE AGENT`, preserving version history across runs. `dbt run --full-refresh` falls back to `CREATE OR REPLACE`, resetting history. Set to `false` in dev environments to skip versioning overhead.
+- `auto_commit` config option — when `enable_versioning=true`, automatically snapshot the LIVE version into a new immutable named version (`VERSION$1`, `VERSION$2`, …) after each run. Defaults to `true`. Set to `false` to accumulate spec changes in LIVE without committing, then commit manually via `ALTER AGENT COMMIT`.
+- `version_comment` config option — optional comment string attached to the committed version snapshot; only used when `enable_versioning=true` and `auto_commit=true`.
+- `snowflake__create_cortex_agent_if_not_exists` macro — `CREATE AGENT IF NOT EXISTS` DDL
+- `snowflake__add_cortex_agent_live_version` macro — `ALTER AGENT ADD LIVE VERSION FROM LAST` DDL; recreates the LIVE working copy after `COMMIT` consumes it
+- `snowflake__alter_cortex_agent_live_spec` macro — `ALTER AGENT MODIFY LIVE VERSION SET SPECIFICATION` DDL
+- `snowflake__commit_cortex_agent_version` macro — `ALTER AGENT COMMIT` DDL
 - `mcp_server` materialization for Snowflake-managed MCP servers — model body is the raw MCP server YAML specification (a `tools:` array), sent verbatim to `CREATE OR REPLACE MCP SERVER ... FROM SPECIFICATION`
 - `snowflake__create_mcp_server` macro for DDL generation
 - `snowflake__get_drop_mcp_server_sql` macro for DROP DDL generation
 - Integration tests for the MCP server materialization (`mcp_server_test`), verified via `DESCRIBE MCP SERVER` captured into a table, since `GET_DDL` does not support MCP servers
 - Local development setup: `integration_tests/.env` template, `scripts/run_tests.ps1` runner, and `profiles.yml` SSO support (`externalbrowser` authenticator)
 
+### Changed
+- `enable_versioning` now defaults to `true` — versioning is on by default for all `cortex_agent` models; set `enable_versioning: false` in `dbt_project.yml` or per-model config to opt out (e.g. in dev environments)
+
 ### Fixed
 - Singular integration tests now declare `-- depends_on: {{ ref('cortex_agent_test') }}` so `dbt build` runs models before tests
 

README.md

@@ -83,19 +83,55 @@ SHOW AGENTS IN SCHEMA my_db.my_schema;
 | `feedback_schema` | string | No | Schema for the feedback table and `AGENT_SUBMIT_FEEDBACK` procedure. Accepts `'SCHEMA'` or `'DB.SCHEMA'`. Defaults to the agent's own database and schema. See [Feedback Tool](#feedback-tool). |
 | `feedback_table` | string | No | Fully-qualified table name override for user feedback. Defaults to `{feedback_schema}.AGENT_FEEDBACK`. Ignored when `create_feedback_table` is `false`. See [Feedback Tool](#feedback-tool). |
 | `feedback_execute_as` | string | No | Execution rights for the `AGENT_SUBMIT_FEEDBACK` procedure. `'caller'` (default) captures the end user via `current_user()`. Use `'owner'` if the calling role lacks `INSERT` on the feedback table. See [Feedback Tool](#feedback-tool). |
+| `enable_versioning` | bool | No | Whether to use Snowflake agent versioning. Defaults to `true`. When `true`, uses `CREATE AGENT IF NOT EXISTS` + `ALTER AGENT MODIFY LIVE VERSION` instead of `CREATE OR REPLACE AGENT`, preserving version history across runs. Set to `false` in dev environments to skip versioning overhead. `dbt run --full-refresh` always falls back to `CREATE OR REPLACE`, resetting history. |
+| `auto_commit` | bool | No | When `enable_versioning=true`, automatically snapshot the LIVE version into a new named version after each run. Defaults to `true`. Set to `false` to accumulate spec changes in LIVE without committing, then commit manually via `ALTER AGENT COMMIT`. |
+| `version_comment` | string | No | Comment attached to each committed version snapshot. Only used when `enable_versioning=true` and `auto_commit=true`. |
 
 ## How It Works
 
-The model body is passed verbatim as the agent YAML specification to Snowflake's `CREATE OR REPLACE AGENT ... FROM SPECIFICATION $$ ... $$`. Because it's a direct passthrough:
+The model body is passed verbatim as the agent YAML specification to Snowflake. Because it's a direct passthrough:
 
 - All current and future Snowflake agent YAML options work automatically
 - No package updates needed when Snowflake adds new features
 - You can use Jinja (`{{ ref() }}`, `{{ var() }}`, etc.) anywhere in the spec
 - `{{ ref() }}` calls in `tool_resources` resolve to fully qualified names **and** wire the agent into the dbt DAG — the agent will always run after its upstream semantic views
 
+## Versioning
+
+By default (`enable_versioning=true`, `auto_commit=true`) every `dbt run`:
+
+1. Creates the agent if it doesn't exist (`CREATE AGENT IF NOT EXISTS`)
+2. Restores a LIVE working copy from the last committed version (`ALTER AGENT ADD LIVE VERSION FROM LAST`)
+3. Updates the LIVE copy with the latest compiled spec (`ALTER AGENT MODIFY LIVE VERSION SET SPECIFICATION`)
+4. Commits the LIVE copy as a new named version — `VERSION$1`, `VERSION$2`, … (`ALTER AGENT COMMIT`)
+
+Version history accumulates across runs. `dbt run --full-refresh` falls back to `CREATE OR REPLACE AGENT`, resetting all history.
+
+**Disable versioning in dev** to avoid the overhead when iterating quickly:
+
+```yaml
+# dbt_project.yml
+models:
+  my_project:
+    agents:
+      +enable_versioning: false
+```
+
+Or per-model:
+
+```sql
+{{ config(materialized='cortex_agent', enable_versioning=false) }}
+```
+
+**Commit manually** by setting `auto_commit=false` — the materialization keeps the LIVE version open across runs and you commit explicitly when ready:
+
+```sql
+{{ config(materialized='cortex_agent', auto_commit=false) }}
+```
+
 ## Idempotency
 
-Every `dbt run` issues `CREATE OR REPLACE AGENT`, so re-runs are safe and fully idempotent. `dbt run --full-refresh` behaves identically.
+With versioning enabled (default), every `dbt run` preserves history — re-runs are safe and create a new committed version. With `enable_versioning=false`, every run issues `CREATE OR REPLACE AGENT`. `dbt run --full-refresh` always uses `CREATE OR REPLACE AGENT`, resetting all version history.
 
 ## Supported Tools in Specification
 

integration_tests/models/cortex_agent_test.sql

@@ -7,7 +7,8 @@
         tags=['integration'],
         agent_grants=['dbt_demo_role'],
         create_feedback_table=true,
-        feedback_execute_as='owner'
+        feedback_execute_as='owner',
+        enable_versioning=false
     )
 }}
 

integration_tests/models/cortex_agent_versioned_test.sql

@@ -0,0 +1,41 @@
+{{
+    config(
+        materialized='cortex_agent',
+        alias='cortex_agent_versioned_test',
+        comment='Versioned integration test agent — exercises enable_versioning, auto_commit, and version_comment',
+        tags=['integration'],
+        version_comment='CI test commit'
+    )
+}}
+
+models:
+  orchestration: auto
+
+orchestration:
+  budget:
+    seconds: 30
+    tokens: 8000
+
+instructions:
+  response: 'You are a versioning integration test assistant.'
+  orchestration: 'Use the analyst tool for structured data questions.'
+  sample_questions:
+    - question: 'What was total revenue last quarter?'
+
+tools:
+  - tool_spec:
+      type: cortex_analyst_text_to_sql
+      name: analyst_tool
+      description: 'Answers structured data questions using a semantic view.'
+      input_schema:
+        type: object
+        properties:
+          question:
+            type: string
+            description: 'A natural language question about the data.'
+        required:
+          - question
+
+tool_resources:
+  analyst_tool:
+    semantic_view: '{{ ref('test_semantic_view') }}'

integration_tests/models/cortex_agent_versioned_test_versions.sql

@@ -0,0 +1,12 @@
+-- SHOW VERSIONS can't be used as a subquery, so this model captures its output into a table
+-- using the same pre_hook pattern as mcp_server_test_describe. The ref() in the pre_hook
+-- forces dbt to run cortex_agent_versioned_test first. Singular tests assert on this table.
+{{
+    config(
+        materialized='table',
+        pre_hook="show versions in agent {{ ref('cortex_agent_versioned_test') }}"
+    )
+}}
+
+select *
+from table(result_scan(last_query_id()))

integration_tests/models/schema.yml

@@ -22,5 +22,8 @@ models:
       alias: mcp_server_test
       tags: ['integration']
 
+  - name: cortex_agent_versioned_test_versions
+    description: "Captures SHOW VERSIONS IN AGENT output for cortex_agent_versioned_test so singular tests can assert on committed version count."
+
   - name: mcp_server_test_describe
     description: "Captures DESCRIBE MCP SERVER output for mcp_server_test so singular tests can assert on the spec (GET_DDL does not support MCP servers)."

integration_tests/tests/cortex_agent_versioned_test_has_versions.sql

@@ -0,0 +1,6 @@
+-- Verify the versioned agent has at least one committed named version.
+-- A successful auto_commit produces VERSION$1 on first run and increments on each subsequent run.
+-- Returns 0 rows on success (standard dbt test contract).
+
+select 'no committed versions found for cortex_agent_versioned_test' as error
+where (select count(*) from {{ ref('cortex_agent_versioned_test_versions') }}) < 1

macros/materializations/cortex_agent.sql

@@ -19,6 +19,19 @@
 --                                            Accepts 'caller' (default) or 'owner'.
 --                                            Use 'caller' to capture the end user via current_user().
 --                                            Use 'owner' if the calling role lacks INSERT on the feedback table.
+--   enable_versioning (bool, optional) : opt into Snowflake agent versioning. Defaults to true.
+--                                        When true, uses CREATE AGENT IF NOT EXISTS + ALTER AGENT
+--                                        MODIFY LIVE VERSION instead of CREATE OR REPLACE AGENT,
+--                                        preserving version history across runs.
+--                                        Set to false in dev environments to skip versioning overhead.
+--                                        dbt run --full-refresh falls back to CREATE OR REPLACE,
+--                                        resetting all version history.
+--   auto_commit (bool, optional)      : when enable_versioning=true, automatically snapshot the
+--                                       LIVE version into a new named version after each run.
+--                                       Defaults to true. Set to false to accumulate changes in
+--                                       LIVE without committing, then commit manually.
+--   version_comment (string, optional) : comment attached to the committed version snapshot.
+--                                        Only used when enable_versioning=true and auto_commit=true.
 
 {% materialization cortex_agent, adapter='snowflake' %}
 
@@ -29,6 +42,9 @@
   {%- set feedback_schema_config = config.get('feedback_schema', default=none) -%}
   {%- set feedback_table         = config.get('feedback_table', default=none) -%}
   {%- set feedback_execute_as    = config.get('feedback_execute_as', default='caller') -%}
+  {%- set enable_versioning      = config.get('enable_versioning', default=true) -%}
+  {%- set auto_commit            = config.get('auto_commit', default=true) -%}
+  {%- set version_comment        = config.get('version_comment', default=none) -%}
 
   {%- set target_relation = api.Relation.create(
       identifier=this.identifier,
@@ -67,10 +83,55 @@
   {% endcall %}
   {%- endif %}
 
+  {%- if enable_versioning and not should_full_refresh() %}
+
+  {# Determine whether any committed versions exist.
+     On first run (agent absent or no committed versions) skip MODIFY LIVE VERSION —
+     the LIVE working copy does not exist until after the first COMMIT.
+     On subsequent runs COMMIT has already established LIVE, so we can update it. #}
+  {%- set _show_agents_sql -%}
+    SHOW AGENTS LIKE '{{ target_relation.identifier }}' IN SCHEMA {{ target_relation.database }}.{{ target_relation.schema }}
+  {%- endset -%}
+  {%- set _agent_rows = run_query(_show_agents_sql) -%}
+  {%- set _has_committed_versions = false -%}
+  {%- if _agent_rows | length > 0 -%}
+    {%- set _show_versions_sql -%}
+      SHOW VERSIONS IN AGENT {{ target_relation.database }}.{{ target_relation.schema }}.{{ target_relation.identifier }}
+    {%- endset -%}
+    {%- set _version_rows = run_query(_show_versions_sql) -%}
+    {%- set _has_committed_versions = (_version_rows | length) > 0 -%}
+  {%- endif -%}
+
+  {% call statement('main') %}
+    {{ dbt_cortex_agent.snowflake__create_cortex_agent_if_not_exists(target_relation, sql, comment, profile) }}
+  {% endcall %}
+
+  {%- if _has_committed_versions %}
+  {# COMMIT consumes LIVE and does not recreate it. Explicitly restore LIVE from the
+     last committed version before modifying, so MODIFY LIVE VERSION has a target. #}
+  {% call statement('add_live') %}
+    {{ dbt_cortex_agent.snowflake__add_cortex_agent_live_version(target_relation) }}
+  {% endcall %}
+  {%- endif %}
+
+  {% call statement('update_live') %}
+    {{ dbt_cortex_agent.snowflake__alter_cortex_agent_live_spec(target_relation, sql) }}
+  {% endcall %}
+
+  {%- if auto_commit %}
+  {% call statement('commit_version') %}
+    {{ dbt_cortex_agent.snowflake__commit_cortex_agent_version(target_relation, version_comment) }}
+  {% endcall %}
+  {%- endif %}
+
+  {%- else %}
+
   {% call statement('main') %}
     {{ dbt_cortex_agent.snowflake__create_cortex_agent(target_relation, sql, comment, profile) }}
   {% endcall %}
 
+  {%- endif %}
+
   {{ run_hooks(post_hooks) }}
 
   {%- if agent_grants | length > 0 %}

macros/relations/cortex_agent/versioned_agent.sql

@@ -0,0 +1,63 @@
+-- Macros supporting Snowflake Cortex Agent versioning.
+-- Used by the cortex_agent materialization when enable_versioning=true.
+
+-- Creates the agent only if it does not already exist.
+-- On first run this initialises VERSION$1 and a LIVE working copy.
+-- On subsequent runs the CREATE is a no-op; spec changes go through
+-- snowflake__alter_cortex_agent_live_spec instead.
+{% macro snowflake__create_cortex_agent_if_not_exists(relation, specification, comment, profile) %}
+
+  create agent if not exists
+    {{ relation.database }}.{{ relation.schema }}.{{ relation.identifier }}
+  {%- if comment is not none %}
+  comment = '{{ comment }}'
+  {%- endif %}
+  {%- if profile is not none %}
+  profile = '{{ profile }}'
+  {%- endif %}
+  from specification
+  $$
+{{ specification | trim }}
+  $$
+
+{% endmacro %}
+
+
+-- Recreates the LIVE working copy from the last committed version.
+-- Required before MODIFY LIVE VERSION on any run after the first COMMIT,
+-- because COMMIT consumes the LIVE version and does not recreate it automatically.
+{% macro snowflake__add_cortex_agent_live_version(relation) %}
+
+  alter agent
+    {{ relation.database }}.{{ relation.schema }}.{{ relation.identifier }}
+  add live version from last
+
+{% endmacro %}
+
+
+-- Pushes the latest compiled spec into the mutable LIVE version.
+-- Runs on every dbt run when versioning is enabled, keeping LIVE current.
+{% macro snowflake__alter_cortex_agent_live_spec(relation, specification) %}
+
+  alter agent
+    {{ relation.database }}.{{ relation.schema }}.{{ relation.identifier }}
+  modify live version set specification =
+  $$
+{{ specification | trim }}
+  $$
+
+{% endmacro %}
+
+
+-- Snapshots the current LIVE version into a new immutable named version
+-- (VERSION$2, VERSION$3, …). Called when auto_commit=true.
+{% macro snowflake__commit_cortex_agent_version(relation, version_comment) %}
+
+  alter agent
+    {{ relation.database }}.{{ relation.schema }}.{{ relation.identifier }}
+  commit
+  {%- if version_comment is not none %}
+  comment = '{{ version_comment }}'
+  {%- endif %}
+
+{% endmacro %}