Back to Projects

Data Pipeline and Visualization

Software Engineer @PALO IT Hong Kong Limited

Achieved >60% reduction in data processing time

Enabled near real-time business insights for sales managers

Automated ETL pipeline for leading global wine & spirits brand

What is this project about?

Sales managers at a leading global wine and spirits brand were drowning in data but starving for insights. We built an automated ETL pipeline that transformed chaos into clarity. Real-time business intelligence became accessible at their fingertips, cutting processing time dramatically. The pipeline pulled data from Snowflake, Azure Data Lake, and Magento, then visualized it through Microsoft PowerBI dashboards that actually made sense. Behind the scenes, Azure Functions orchestrated the entire flow, and Agile under Scrum Principles kept the team moving fast.

🛠️Tech Stack

Backend

Python
Python

Data

Snowflake
Snowflake
Magento
Magento
PowerBI
PowerBI

Cloud / Infra / SaaS

Azure Cloud
Azure Cloud

Project Management

Agile / Scrum
Agile / Scrum