Databricks: enable columns / column-level analysis

[tasozgurcem11]

Summary

Enable column-level unused detection for Databricks (dbt-debt scan --columns).

Context

Databricks support landed in #3 with column analysis disabled because complete query-text or column-lineage coverage has not been proven across all supported compute paths (SQL warehouse, serverless, classic compute). Reporting unused columns without that coverage would risk false "unused" verdicts.

Current behavior

• cli.py skips the column stage on warehouse == "databricks" with a warning.
• databricks_queries.query_text_query exists but is unused in v1.
• Column-lineage rows were visible in Free Edition validation for SQL-warehouse workloads only; that does not prove cross-compute coverage.

Proposed approach

1. Validate query-text and/or system.access.column_lineage coverage across SQL warehouse, serverless, and classic job compute.
2. Confirm dbt query-comment exclusion works on joined query history for column parsing.
3. Re-enable --columns when coverage is completeness-proven (same bar as other warehouses).
4. Add tests pinning SQL builders and mock client behavior.

Related

• PR feat: add Databricks support for dbt-debt scan #3
• Closes part of the Databricks v1 deferred scope

[obrienciaran]

We can also check semantic models as part of this work.