Data Engineer - Python/PySpark

Responsibilities :

- Design, develop, and maintain scalable data pipelines and backend services using Python.

- Build and optimize ETL/ELT workflows using PySpark, Spark SQL, and Azure Data Factory (ADF).

- Develop and manage data ingestion, transformation, and processing solutions on Databricks.

- Implement and maintain Delta Lake architectures to support reliable and high-performance data platforms.

- Write efficient and optimized SQL queries for large-scale datasets.

- Collaborate with business stakeholders, architects, data analysts, and cross-functional teams to understand requirements and deliver robust solutions.

- Work with Azure cloud services to deploy, monitor, and maintain data applications.

- Implement CI/CD pipelines and deployment automation using Azure DevOps.

- Perform troubleshooting, performance tuning, and optimization of data workloads.

- Ensure adherence to coding standards, security guidelines, and best engineering practices.

- Participate in solution design discussions, code reviews, and technical documentation.

Tech Stack :

- Python, PySpark, Spark SQL, Databricks, Delta Lake, SQL, Azure Data Factory, Azure DevOps.

Qualifications :

- Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.

- 6 years of relevant industry experience in Data Engineering, Data Platform Engineering, or Backend Python Development.