Is there an existing issue for this?
Problem statement
Setting Unity Catalog tags is not currently natively supported by SDP as part of declarative table DDLs, and ALTER TABLE/SET TAGS SQL commands are prohibited within the SDP execution runtime. This makes UC tagging hard within the context of SDP. Even so, it is a much-demanded capability from customers to support their tag-based ABAC governance policies. It also something that is supported by competitor tooling, such as DBT.
Proposed Solution
Extend LFF to set Unity Catalog table- and column-level tags via an SDP event_hook. The hook reconciles configured tags against system.information_schema through the EntityTagAssignments REST API. Opt in by adding a tags map to a streaming table/materialized view target, or to columns in a JSON/YAML schema file.
Additional Context
No response
Is there an existing issue for this?
Problem statement
Setting Unity Catalog tags is not currently natively supported by SDP as part of declarative table DDLs, and ALTER TABLE/SET TAGS SQL commands are prohibited within the SDP execution runtime. This makes UC tagging hard within the context of SDP. Even so, it is a much-demanded capability from customers to support their tag-based ABAC governance policies. It also something that is supported by competitor tooling, such as DBT.
Proposed Solution
Extend LFF to set Unity Catalog table- and column-level tags via an SDP event_hook. The hook reconciles configured tags against system.information_schema through the EntityTagAssignments REST API. Opt in by adding a tags map to a streaming table/materialized view target, or to columns in a JSON/YAML schema file.
Additional Context
No response