Standardize unioning macros to use fivetran_utils versions

.gitignore

@@ -75,3 +75,4 @@ venv.bak/
 
 # Claude instructions
 CLAUDE.md
+worktrees/

CHANGELOG.md

@@ -1,3 +1,15 @@
+# dbt_recurly v1.5.0
+
+[PR #47](https://github.com/fivetran/dbt_recurly/pull/47) includes the following updates:
+
+## Breaking Changes
+- Source table identifier variables have been renamed to include the `recurly_` prefix for consistency with other Fivetran dbt packages and to ensure union functionality works as expected. If you have configured any identifier variables in your project, update them to the new format (e.g., `account_history_identifier` → `recurly_account_history_identifier`).
+
+## Under the Hood
+- Migrates the `union_connections`, `apply_source_relation`, and `partition_by_source_relation` macros to the `dbt_fivetran_utils` package.
+- Adds the `fivetran_using_source_casing` variable for case-sensitive destination support. When enabled, downstream transformations respect source casing to ensure consistent results. See the [Additional Configurations](https://github.com/fivetran/dbt_recurly/#source-casing-for-case-sensitive-destinations) section of the README for details.
+- Removed column from `get_billing_info_history_columns()` macro which is not referenced in staging model.
+
 # dbt_recurly v1.4.0
 
 [PR #45](https://github.com/fivetran/dbt_recurly/pull/45) includes the following updates:

README.md

@@ -74,7 +74,7 @@ Include the following recurly package version in your `packages.yml` file.
 ```yaml
 packages:
   - package: fivetran/recurly
-    version: [">=1.4.0", "<1.5.0"]
+    version: [">=1.5.0", "<1.6.0"]
 ```
 
 > All required sources and staging models are now bundled into this transformation package. Do not include `fivetran/recurly_source` in your `packages.yml` since this package has been deprecated.
@@ -88,14 +88,12 @@ dispatch:
 ```
 
 ### Define database and schema variables
-
 #### Option A: Single connection
-By default, this package runs using your [destination](https://docs.getdbt.com/docs/running-a-dbt-project/using-the-command-line-interface/configure-your-profile) and the `recurly` schema. If this is not where your Recurly data is (for example, if your Recurly schema is named `recurly_fivetran`), add the following configuration to your root `dbt_project.yml` file:
+By default, this package runs using your destination and the `recurly` schema. If this is not where your Recurly data is (for example, if your Recurly schema is named `recurly_fivetran`), add the following configuration to your root `dbt_project.yml` file:
 
 ```yml
 vars:
-  recurly:
-    recurly_database: your_database_name
+    recurly_database: your_destination_name
     recurly_schema: your_schema_name
 ```
 
@@ -119,42 +117,9 @@ vars:
         name: connection_2_source_name
 ```
 
-##### Recommended: Incorporate unioned sources into DAG
-> *If you are running the package through [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt#transformationsfordbtcore), the below step is necessary in order to synchronize model runs with your Recurly connections. Alternatively, you may choose to run the package through Fivetran [Quickstart](https://fivetran.com/docs/transformations/quickstart), which would create separate sets of models for each Recurly source rather than one set of unioned models.*
-
-By default, this package defines one single-connection source, called `recurly`, which will be disabled if you are unioning multiple connections. This means that your DAG will not include your Recurly sources, though the package will run successfully.
+#### Optional: Incorporate unioned sources into DAG
 
-To properly incorporate all of your Recurly connections into your project's DAG:
-1. Define each of your sources in a `.yml` file in your project. Utilize the following template for the `source`-level configurations, and, **most importantly**, copy and paste the table and column-level definitions from the package's `src_recurly.yml` [file](https://github.com/fivetran/dbt_recurly/blob/main/models/staging/src_recurly.yml).
-
-```yml
-# a .yml file in your root project
-
-version: 2
-
-sources:
-  - name: <name> # ex: Should match name in recurly_sources
-    schema: <schema_name>
-    database: <database_name>
-    loader: fivetran
-    config:
-      loaded_at_field: _fivetran_synced
-      freshness: # feel free to adjust to your liking
-        warn_after: {count: 72, period: hour}
-        error_after: {count: 168, period: hour}
-
-    tables: # copy and paste from recurly/models/staging/src_recurly.yml - see https://support.atlassian.com/bitbucket-cloud/docs/yaml-anchors/ for how to use anchors to only do so once
-```
-
-> **Note**: If there are source tables you do not have (see [Disable models for non-existent sources](#disable-models-for-non-existent-sources)), you may still include them, as long as you have set the right variables to `False`.
-
-2. Set the `has_defined_sources` variable (scoped to the `recurly` package) to `True`, like such:
-```yml
-# dbt_project.yml
-vars:
-  recurly:
-    has_defined_sources: true
-```
+If you use [Fivetran Transformations for dbt Core™](https://fivetran.com/docs/transformations/dbt#transformationsfordbtcore) and are unioning multiple Recurly connections, you can define your sources in a property `.yml` file, [using this as a template](https://github.com/fivetran/dbt_recurly/blob/main/models/staging/src_recurly.yml). Set the variable `has_defined_sources: true` under the Recurly namespace in your `dbt_project.yml`. Otherwise, your Recurly connections won't appear in your DAG. See the `union_connections` macro [documentation](https://github.com/fivetran/dbt_fivetran_utils/tree/releases/v0.4.latest#optional-union-connections-defined-sources-configuration) for full configuration details.
 
 ### Disable models for non-existent sources
 Your Recurly connection may not sync every table that this package expects. This might be because you are excluding those tables. If you are not using those tables, you can disable the corresponding functionality in the package by specifying the variable in your dbt_project.yml. By default, all packages are assumed to be true. You only have to add variables for tables you want to disable, like so:
@@ -211,6 +176,14 @@ vars:
     <default_source_table_name>_identifier: your_table_name 
 ```
 
+#### Source casing for case-sensitive destinations
+By default, the package applies case-insensitive comparisons when resolving `source_relation` values. If your destination is case-sensitive and you want downstream transformations to respect the exact casing of your source database and schema names, set the following variable:
+
+```yml
+vars:
+    fivetran_using_source_casing: true
+```
+
 </details>
 
 ### (Optional) Orchestrate your models with Fivetran Transformations for dbt Core™

dbt_project.yml

@@ -1,5 +1,5 @@
 name: 'recurly'
-version: '1.4.0'
+version: '1.5.0'
 config-version: 2
 require-dbt-version: [">=1.3.0", "<3.0.0"]
 

integration_tests/ci/test_scenarios.yml

@@ -21,3 +21,9 @@ test_scenarios:
     vars:
       recurly__standardized_billing_model_enabled: false
     include_incremental: false
+
+  - name: "MDLS"
+    vars:
+      fivetran_using_source_casing: true
+    include_incremental: false
+    include_dbt_seed: true

integration_tests/dbt_project.yml

@@ -1,29 +1,29 @@
 name: 'recurly_integration_tests'
-version: '1.4.0'
+version: '1.5.0'
 profile: 'integration_tests'
 config-version: 2
 
 vars:
   recurly_schema: recurly_integrations_tests
   recurly:
-    account_balance_history_identifier: "account_balance_history_data"
-    account_history_identifier: "account_history_data"
-    account_note_history_identifier: "account_note_history_data"
-    billing_info_history_identifier: "billing_info_history_data"
-    coupon_discount_identifier: "coupon_discount_data"
-    coupon_redemption_history_identifier: "coupon_redemption_history_data"
-    credit_payment_history_identifier: "credit_payment_history_data"
-    invoice_coupon_redemption_history_identifier: "invoice_coupon_redemption_history_data"
-    invoice_history_identifier: "invoice_history_data"
-    invoice_subscription_history_identifier: "invoice_subscription_history_data"
-    line_item_history_identifier: "line_item_history_data"
-    plan_currency_history_identifier: "plan_currency_history_data"
-    plan_history_identifier: "plan_history_data"
-    subscription_add_on_history_identifier: "subscription_add_on_history_data"
-    subscription_change_history_identifier: "subscription_change_history_data"
-    subscription_history_identifier: "subscription_history_data"
-    transaction_identifier: "transaction_data"
-    transaction_subscription_identifier: "transaction_subscription_data"
+    recurly_account_balance_history_identifier: "account_balance_history_data"
+    recurly_account_history_identifier: "account_history_data"
+    recurly_account_note_history_identifier: "account_note_history_data"
+    recurly_billing_info_history_identifier: "billing_info_history_data"
+    recurly_coupon_discount_identifier: "coupon_discount_data"
+    recurly_coupon_redemption_history_identifier: "coupon_redemption_history_data"
+    recurly_credit_payment_history_identifier: "credit_payment_history_data"
+    recurly_invoice_coupon_redemption_history_identifier: "invoice_coupon_redemption_history_data"
+    recurly_invoice_history_identifier: "invoice_history_data"
+    recurly_invoice_subscription_history_identifier: "invoice_subscription_history_data"
+    recurly_line_item_history_identifier: "line_item_history_data"
+    recurly_plan_currency_history_identifier: "plan_currency_history_data"
+    recurly_plan_history_identifier: "plan_history_data"
+    recurly_subscription_add_on_history_identifier: "subscription_add_on_history_data"
+    recurly_subscription_change_history_identifier: "subscription_change_history_data"
+    recurly_subscription_history_identifier: "subscription_history_data"
+    recurly_transaction_identifier: "transaction_data"
+    recurly_transaction_subscription_identifier: "transaction_subscription_data"
 
 # For use with validations
 models:
@@ -39,6 +39,7 @@ dispatch:
 seeds:
   +docs:
     show: False
+  +quote_columns: "{{ target.type == 'redshift' or var('fivetran_using_source_casing', false) }}"
   recurly_integration_tests:
     account_balance_history_data:
       +column_types: 

macros/staging/get_billing_info_history_columns.sql

@@ -25,7 +25,6 @@
     {"name": "payment_method_exp_year", "datatype": dbt.type_int()},
     {"name": "fraud_score", "datatype": dbt.type_string()},
     {"name": "fraud_decision", "datatype": dbt.type_string()},
-    {"name": "fraud_risk_rules_triggered", "datatype": "variant"},
     {"name": "created_at", "datatype": dbt.type_timestamp()},
     {"name": "updated_by_ip", "datatype": dbt.type_string()},
     {"name": "updated_by_country", "datatype": dbt.type_string()}