Data Engineer

Data Engineer Microsoft Fabric

Key Responsibilities

• Build and maintain data pipelines using Microsoft Fabric.
• Work on ingestion and transformation of data from Kafka, Oracle, Azure SQL, REST APIs, and ADLS.
• Develop ETL/ELT workflows using PySpark and Spark SQL.
• Design and manage Lakehouse and Warehouse structures in Fabric.
• Use Fabric Shortcuts to access data across OneLake and external storage efficiently, without unnecessary duplication.
• Support real-time and batch data processing needs.
• Develop and maintain Power BI reports and dashboards.
• Ensure data models are clean, reliable, and optimized for reporting and analytics.
• Monitor pipelines, fix issues, and continuously improve performance.
• Follow coding standards, version control (Git), and CI/CD practices.

Required Skills

• 5–8 years of experience in Data Engineering.
• Strong hands-on experience with Microsoft Fabric.
• Good knowledge of PySpark and Spark SQL.
• Strong SQL skills, including work with Azure SQL.
• Experience building data pipelines using Fabric or Azure Data Factory.
• Hands-on experience working with Kafka, Oracle, REST APIs, and ADLS.
• Understanding of Lakehouse architecture, Delta tables, and data modeling concepts (Star/Snowflake schema).
• Working knowledge of Power BI (datasets, semantic models, integration with Fabric).
• Experience with Git and basic CI/CD workflows.
• Understanding of data quality, performance tuning, and security basics.

Preferred Skills

• Exposure to Azure Databricks or Synapse.
• Experience with Azure DevOps.
• Familiarity with streaming / near real-time pipelines.
• Experience handling semi-structured data (JSON, APIs).

Qualifications

• Bachelor’s degree in Computer Science, IT, or related field.
• Azure or Fabric certifications are a plus.