Senior Data Engineer

Experience: 7+ Years
Location: Chennai, Pune, Hyderabad, Bengaluru
Work Mode: Work from Office (Onsite)

Job Summary

We are looking for an experienced Senior Data Engineer with strong hands-on expertise in Microsoft Fabric, Lakehouse architecture, Medallion Architecture, and scalable data ingestion and transformation pipelines. The ideal candidate should have experience working with Azure Databricks, Azure Data Factory, Fivetran or similar ELT tools, Python/PySpark, and SQL.

Key Responsibilities

• Design, develop, and optimize data ingestion and transformation pipelines using Microsoft Fabric Lakehouse.
• Configure and manage Fivetran connectors for Oracle, SQL Server, and other enterprise data sources.
• Implement Bronze-to-Silver transformation pipelines using Medallion Architecture.
• Develop incremental loading, Change Data Capture (CDC), and historical backfill processes.
• Build and maintain ETL/ELT pipelines using Microsoft Fabric, Azure Data Factory, and Azure Databricks.
• Handle large-scale datasets involving hundreds of millions of records.
• Optimize Lakehouse performance, storage, and data processing efficiency.
• Implement pipeline monitoring, data quality checks, record count validation, and failure tracking.
• Support pipeline deployment and promotion across Development, Testing, and Production environments.
• Develop and optimize complex SQL queries and data transformation logic using Python or PySpark.

Required Skills

• 7+ years of experience in Data Engineering.
• Strong hands-on experience with Microsoft Fabric Lakehouse, preferably 1.5+ years.
• Good understanding of Medallion Architecture (Bronze, Silver, and Gold layers).
• Hands-on experience with Azure Databricks and/or Azure Data Factory.
• Experience with Fivetran or similar data ingestion and ELT tools.
• Strong proficiency in SQL and either Python or PySpark.
• Experience with CDC, incremental loads, data backfills, and pipeline orchestration.
• Experience handling high-volume datasets and optimizing data pipelines.
• Understanding of data modeling, data quality, monitoring, and troubleshooting.

Preferred Skills

• Experience configuring Fivetran connectors for Oracle and SQL Server.
• Production experience with Microsoft Fabric implementation.
• Experience with Lakehouse performance tuning and storage optimization.
• Knowledge of CI/CD and Dev/Test/Prod deployment processes.

Candidate Requirements

• Candidates must be comfortable working onsite in Chennai, Pune, Hyderabad, or Bengaluru.
• Chennai-based candidates are preferred.
• Candidates with strong Microsoft Fabric expertise and relevant Azure data engineering experience will be given priority.

Keywords: Microsoft Fabric, Fabric Lakehouse, Senior Data Engineer, Azure Databricks, Azure Data Factory, Fivetran, Medallion Architecture, PySpark, Python, SQL, ETL, ELT, CDC, Incremental Load, Data Pipelines, Data Engineering.