DATA Engineer - Azure data bricks - All GDS Locations
Posted Updated
Description
EY in India seeks a strong Azure Data Engineer to design and implement scalable ETL/ELT pipelines using Azure Databricks, PySpark, Python, and SQL. The role requires expertise with Delta Lake, Spark SQL, DataFrame transformations, and CDC, plus experience with ADF, ADLS Gen2, and Medallion Architecture.
The ideal candidate will optimize Spark performance, ensure data quality, and contribute to dimensional modeling and SQL tuning.
Design and develop scalable ETL/ELT pipelines for large datasets. Optimize Spark jobs and ensure data quality. Implement Medallion Architecture patterns with Delta Lake. Collaborate using Azure Data Factory and ADLS Gen2. Monitor pipelines and manage production deployments with CI/CD. Strong hands-on expertise in Azure Databricks, PySpark, Python and SQL. Experience designing scalable ETL/ELT pipelines for large datasets. Proficient with Delta Lake, Spark SQL, DataFrame transformations and CDC processing. Experience with Azure Data Factory, ADLS Gen2 and Medallion Architecture. Knowledge of Spark performance optimization and SQL query tuning. Familiarity with Azure DevOps, Git, CI/CD, monitoring and production deployment.Skills
Sign in to read and join the discussion.