Senior Data Engineer (PySpark & Microsoft Fabric)

Key Responsibilities

• Design, build, and maintain scalable data pipelines that extract, transform, and load data from multiple sources into centralized platforms such as data warehouses, lakehouses, and operational databases.


• Develop and manage end-to-end data workflows using Microsoft Fabric, including integration with Azure Data Factory and related Azure data services.


• Implement physical and logical data models that support efficient storage, performance, and analytical use cases, while maintaining data integrity and reliability.


• Transform raw and semi-structured data into analytics-ready datasets through cleansing, standardization, and deduplication processes.


• Integrate data from disparate systems and ensure consistency, accuracy, and quality across the data pipeline.


• Optimize database and pipeline performance through tuning, monitoring, and proactive issue resolution.


• Collaborate with analytics, reporting, and business teams to translate data requirements into robust technical solutions.

Required Competencies

• Bachelor’s degree in Computer Science, Information Technology, Mathematics, or a related field.


• At least 6 years of progressive, hands-on experience in Data Engineering.


• Minimum 6 years of experience in ETL/ELT development, data warehousing, and data modeling.


• At least 3 years of experience delivering data solutions on the Microsoft Azure platform, with strong exposure to Microsoft Fabric and Azure Data Factory.


• Strong proficiency in PySpark and/or Python for large-scale data processing.


• Solid experience with T-SQL and relational database concepts.


• Proven expertise in designing, building, and maintaining robust ETL pipelines for structured and semi-structured data in warehouse and lakehouse environments.


• Strong understanding of data architecture, performance optimization, and best practices for scalable data platforms.