Standardize unioning macros to use fivetran_utils versions

.gitignore

@@ -74,3 +74,4 @@ venv.bak/
 
 # Claude instructions
 CLAUDE.md
+worktrees/

CHANGELOG.md

@@ -1,3 +1,13 @@
+# dbt_zuora v1.4.0
+
+[PR #42](https://github.com/fivetran/dbt_zuora/pull/42) includes the following updates:
+
+## Under the Hood
+- Updated `int_zuora__transaction_date_spine` model to reference `stg_zuora__invoice` vs `stg_zuora__invoice_tmp` when generating the date spine in order to ensure source casing compatibility.
+- Updated `int_zuora__mrr_date_spine` model to reference `stg_zuora__invoice_item` vs `stg_zuora__invoice_item_tmp` when generating the date spine in order to ensure source casing compatibility.
+- Migrates the `union_connections`, `apply_source_relation`, and `partition_by_source_relation` macros to the `dbt_fivetran_utils` package.
+- Adds the `fivetran_using_source_casing` variable for case-sensitive destination support. When enabled, downstream transformations respect source casing to ensure consistent results. See the [Additional Configurations](https://github.com/fivetran/dbt_zuora/#source-casing-for-case-sensitive-destinations) section of the README for details.
+
 # dbt_zuora v1.3.0
 
 [PR #38](https://github.com/fivetran/dbt_zuora/pull/38) includes the following updates:

README.md

@@ -75,7 +75,7 @@ Include the following zuora package version in your `packages.yml` file.
 ```yaml
 packages:
   - package: fivetran/zuora
-    version: [">=1.3.0", "<1.4.0"]
+    version: [">=1.4.0", "<1.5.0"]
 ```
 > All required sources and staging models are now bundled into this transformation package. Do not include `fivetran/zuora_source` in your `packages.yml` since this package has been deprecated.
 
@@ -89,14 +89,12 @@ dispatch:
 ```
 
 ### Define database and schema variables
-
 #### Option A: Single connection
-By default, this package runs using your [destination](https://docs.getdbt.com/docs/running-a-dbt-project/using-the-command-line-interface/configure-your-profile) and the `zuora` schema. If this is not where your Zuora data is (for example, if your Zuora schema is named `zuora_fivetran`), add the following configuration to your root `dbt_project.yml` file:
+By default, this package runs using your destination and the `zuora` schema. If this is not where your Zuora data is (for example, if your Zuora schema is named `zuora_fivetran`), add the following configuration to your root `dbt_project.yml` file:
 
 ```yml
 vars:
-  zuora:
-    zuora_database: your_database_name
+    zuora_database: your_destination_name
     zuora_schema: your_schema_name
 ```
 
@@ -120,42 +118,9 @@ vars:
         name: connection_2_source_name
 ```
 
-##### Recommended: Incorporate unioned sources into DAG
-> *If you are running the package through [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt#transformationsfordbtcore), the below step is necessary in order to synchronize model runs with your Zuora connections. Alternatively, you may choose to run the package through Fivetran [Quickstart](https://fivetran.com/docs/transformations/quickstart), which would create separate sets of models for each Zuora source rather than one set of unioned models.*
-
-By default, this package defines one single-connection source, called `zuora`, which will be disabled if you are unioning multiple connections. This means that your DAG will not include your Zuora sources, though the package will run successfully.
+#### Optional: Incorporate unioned sources into DAG
 
-To properly incorporate all of your Zuora connections into your project's DAG:
-1. Define each of your sources in a `.yml` file in your project. Utilize the following template for the `source`-level configurations, and, **most importantly**, copy and paste the table and column-level definitions from the package's `src_zuora.yml` [file](https://github.com/fivetran/dbt_zuora/blob/main/models/staging/src_zuora.yml).
-
-```yml
-# a .yml file in your root project
-
-version: 2
-
-sources:
-  - name: <name> # ex: Should match name in zuora_sources
-    schema: <schema_name>
-    database: <database_name>
-    loader: fivetran
-    config:
-      loaded_at_field: _fivetran_synced
-    freshness: # feel free to adjust to your liking
-      warn_after: {count: 72, period: hour}
-      error_after: {count: 168, period: hour}
-
-    tables: # copy and paste from zuora/models/staging/src_zuora.yml - see https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/ for how to use anchors to only do so once
-```
-
-> **Note**: If there are source tables you do not have (see [Disable models for non-existent sources](https://github.com/fivetran/dbt_zuora?tab=readme-ov-file#disable-models-for-non-existent-sources)), you may still include them, as long as you have set the right variables to `False`.
-
-2. Set the `has_defined_sources` variable (scoped to the `zuora` package) to `True`, like such:
-```yml
-# dbt_project.yml
-vars:
-  zuora:
-    has_defined_sources: true
-```
+If you use [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt#transformationsfordbtcore) and are unioning multiple Zuora connections, you can define your sources in a property `.yml` file, [using this as a template](https://github.com/fivetran/dbt_zuora/blob/main/models/staging/src_zuora.yml). Set the variable `has_defined_sources: true` under the Zuora namespace in your `dbt_project.yml`. Otherwise, your Zuora connections won't appear in your DAG. See the `union_connections` macro [documentation](https://github.com/fivetran/dbt_fivetran_utils/tree/releases/v0.4.latest#optional-union-connections-defined-sources-configuration) for full configuration details.
 
 ### Disable models for non-existent sources
 Your Zuora connection may not sync every table that this package expects. This might be because you are excluding those tables. If you are not using those tables, you can disable the corresponding functionality in the package by specifying the variable in your dbt_project.yml. By default, all packages are assumed to be true. You only have to add variables for tables you want to disable in the following way:
@@ -243,6 +208,14 @@ vars:
     zuora_<default_source_table_name>_identifier: your_table_name 
 ```
 
+#### Source casing for case-sensitive destinations
+By default, the package applies case-insensitive comparisons when resolving `source_relation` values. If your destination is case-sensitive and you want downstream transformations to respect the exact casing of your source database and schema names, set the following variable:
+
+```yml
+vars:
+    fivetran_using_source_casing: true
+```
+
 #### Snowflake Users
 If you do **not** use the default all-caps naming conventions for Snowflake, you may need to provide the case-sensitive spelling of your source tables that are also Snowflake reserved words.
 
@@ -300,4 +273,4 @@ If you would like a deeper explanation of the decisions we made to our models in
 
 ## Are there any resources available?
 - If you have questions or want to reach out for help, see the [GitHub Issue](https://github.com/fivetran/dbt_zuora/issues/new/choose) section to find the right avenue of support for you.
-- If you would like to provide feedback to the dbt package team at Fivetran or would like to request a new dbt package, fill out our [Feedback Form](https://www.surveymonkey.com/r/DQ7K7WW).
+- If you would like to provide feedback to the dbt package team at Fivetran or would like to request a new dbt package, fill out our [Feedback Form](https://www.surveymonkey.com/r/DQ7K7WW).

analyses/zuora__account_churn_analysis.sql

@@ -35,8 +35,8 @@ account_current_month as (
 account_previous_month as (
 
     select *,
-      lag(rate_plan_charges) over (partition by account_id {{ zuora.partition_by_source_relation() }} order by account_month) as rate_plan_charges_last_month,
-      lag(is_current_month_active) over (partition by account_id {{ zuora.partition_by_source_relation() }} order by account_month) as is_previous_month_active
+      lag(rate_plan_charges) over (partition by account_id {{ fivetran_utils.partition_by_source_relation(package_name='zuora') }} order by account_month) as rate_plan_charges_last_month,
+      lag(is_current_month_active) over (partition by account_id {{ fivetran_utils.partition_by_source_relation(package_name='zuora') }} order by account_month) as is_previous_month_active
     from account_current_month
 ),
 

dbt_project.yml

@@ -1,5 +1,5 @@
 name: 'zuora'
-version: '1.3.0'
+version: '1.4.0'
 config-version: 2
 require-dbt-version: [">=1.3.0", "<3.0.0"]
 

integration_tests/ci/test_scenarios.yml

@@ -23,4 +23,10 @@ test_scenarios:
       zuora__using_taxation_item: false
       zuora__using_refund: false
       zuora__using_refund_invoice_payment: false
-    include_incremental: false
+    include_incremental: false
+
+  - name: "MDLS"
+    vars:
+      fivetran_using_source_casing: true
+    include_incremental: false
+    include_dbt_seed: true

integration_tests/dbt_project.yml

@@ -1,5 +1,5 @@
 name: 'zuora_integration_tests'
-version: '1.3.0'
+version: '1.4.0'
 profile: 'integration_tests'
 config-version: 2
 
@@ -40,6 +40,7 @@ dispatch:
 seeds:
   +docs:
     show: False
+  +quote_columns: "{{ target.type == 'redshift' or var('fivetran_using_source_casing', false) }}"
   zuora_integration_tests:
     zuora_account_data:
       +column_types: