Data Engineer - Python/SQL

Role Overview :

We are looking for a Data Engineer with strong hands-on experience in Python, SQL, and Spark to build and maintain scalable data pipelines and integration solutions. The ideal candidate should have experience with Azure data technologies and ETL/ELT frameworks.

Key Responsibilities :

- Develop and maintain scalable data pipelines using Python, SQL, and Spark/PySpark.

- Build and manage data workflows using Azure Data Factory, Azure Databricks, and Synapse Analytics.

- Design and implement ETL/ELT processes for data ingestion and transformation.

- Integrate data from multiple sources and ensure reliable data movement across platforms.

- Optimize data processing jobs, queries, and pipelines for performance and scalability.

- Perform data validation, quality checks, and troubleshooting of pipeline failures.

- Monitor data pipelines and resolve issues across development and production environments.

- Collaborate with data scientists, analysts, and technology teams to deliver data solutions.

Tech Stack & Requirements :

- Strong proficiency in Python, SQL, and Spark/PySpark.

- Hands-on experience with Azure Data Factory, Azure Databricks, and/or Synapse Analytics.

- Experience with ETL/ELT frameworks and data integration tools.

- Good understanding of data engineering concepts and data pipeline architecture.

- Experience with data transformation, processing, and optimization.

- Strong troubleshooting and analytical skills.

- Good understanding of cloud-based data platforms.