Senior Data Engineer · Microsoft Fabric & Azure · Lakehouses, real-time streaming and data governance as code

I build medallion lakehouses and real-time streaming pipelines on Microsoft Fabric, deployed as code (Terraform + GitHub Actions, dev → test → prod) and governed with Microsoft Purview and Unity Catalog. 8+ years in data engineering, currently at EPAM Systems.

Focus: lakehouse architecture · streaming · CI/CD and IaC for data platforms · data governance and data mesh

🛠️ Top Tech: Microsoft Fabric | Delta Lake | Spark Structured Streaming | PySpark | Python

Selected work showcasing end-to-end data engineering and AI-ready architectures.

• Real-Time Stock Volatility Tech: Microsoft Fabric, Spark Structured Streaming, PySpark, yfinance API. Impact: Engineered a real-time streaming pipeline using Hopping Windows and Watermarking to analyze market volatility spikes with exactly-once processing semantics.

• Fabric Medallion CI/CD with Terraform Tech: Microsoft Fabric, Terraform, GitHub Actions, Deployment Pipelines, PySpark, Delta Lake. Impact: Built a production-grade Medallion lakehouse (1.6M Chicago bike trips + weather) provisioned with Terraform and promoted dev → test → prod by GitHub Actions with OIDC (no secrets) and an approval gate, guarded by 117 unit tests and a 45-check data-quality gate that blocks bad releases.

• Lakehouse Governance as Code Tech: Databricks, Unity Catalog, Python, Databricks Asset Bundles, GitHub Actions, Delta Lake, AI/BI Dashboards. Impact: Declared Unity Catalog governance as code: YAML policy rendered by a Python engine into idempotent SQL and promoted dev → prod by GitHub Actions with a service principal and an approval gate, enforcing ABAC masking of PII, failing the pipeline on grant drift, honoring GDPR right-to-erasure, and publishing an auditor scorecard (100% owned, classified and PII-masked, 0 drift) guarded by 90 unit tests.

• Fabric Data Mesh with Purview Governance Tech: Microsoft Fabric, Microsoft Purview, OneLake, Terraform, Microsoft Entra ID, GitHub Actions, Python. Impact: Designed a 3-domain data mesh on Fabric as code: producer and consumer domains publish data products through 12 OneLake shortcuts (485 MB served in place, 0 bytes copied), with least-privilege OneLake security roles enforced at every hop, plus Purview sensitivity labels, endorsement, tags and lineage. Two Terraform stacks deploy through GitHub Actions with OIDC and an approval gate, and a daily compliance check fails on any drift between the tenant and the code.

• Medallion Data Dictionary RAG Tech: Python, Databricks SQL, MCP, Anthropic API, Voyage AI, ChromaDB, Streamlit. Impact: Built a RAG chatbot that auto-generates and answers questions about a data dictionary over a Databricks Medallion lakehouse, using a Claude-powered subagent to draft column descriptions and Voyage AI/ChromaDB retrieval to answer natural-language queries via the Anthropic API.

• Self-Healing Data Pipeline Tech: Claude Code, MCP, Databricks, PySpark, Python. Impact: Built a diagnostic multi-agent system for failed Databricks PySpark jobs, chaining log-parser → classifier → patch-proposer subagents to classify each failure and propose a templated fix, with every patch gated behind human approval and logged to a full evidence trail.

• 🏅 DP-700: Microsoft Certified: Fabric Data Engineer Associate
• 🏅 DP-600: Microsoft Certified: Fabric Analytics Engineer Associate
• 🏅 PL-300: Microsoft Certified: Power BI Data Analyst Associate

💬 Let's connect and build amazing things with data. 🔗 LinkedIn