This project demonstrates an end-to-end data engineering pipeline using Microsoft Azure. It is designed to extract, transform, and analyze data from an on-premises SQL database to generate business insights.
The project extracts data from the AdventureWorks database, processes it using Azure Data Factory and Azure Databricks, and loads it into Azure Synapse Analytics. Finally, the data is visualized through Power BI for business analysis.
The objective of this project is to build a scalable and efficient data pipeline that integrates on-premises SQL data with Azure cloud services, enabling advanced analytics and business intelligence through Power BI.
- Azure Data Factory
- Azure Data Lake Storage (ADLS)
- Azure Databricks
- Azure Synapse Analytics
- Power BI
- Azure Key Vault
- SQL Server (On-Premises)
This project was developed as a hands-on learning experience in cloud data engineering, drawing inspiration from educational materials and industry best practices shared by Luke J Byrne.