PySpark Data Engineer (ID: 3887)
Summary
Build and maintain scalable PySpark data pipelines on Databricks, orchestrating workflows with Python and Airflow to deliver reliable analytics-ready data.
- Design, develop, and maintain scalable data pipelines using PySpark on Databricks
- Build and optimize data processing workflows using Python
- Implement workflow orchestration and scheduling (preferably using Airflow, if applicable)
- Work in an Agile delivery environment with cross-functional teams
- Ensure data quality, performance, and reliability of data solutions
- Support integration of data from multiple sources into analytics-ready structures
What You Bring to the Table
- 5+ years of hands-on experience in PySpark (Databricks) and Python
- Strong experience in building and maintaining data engineering pipelines
- Exposure to Airflow (preferred, not mandatory)
- Overall 6–8 years of professional experience in data engineering or related roles
- Strong communication skills and ability to work with distributed teams
- Good understanding of Agile development practices
You should possess the ability to
- Develop efficient and scalable big data processing solutions using PySpark
- Debug, optimize, and enhance existing data workflows and pipelines
- Work independently as well as collaboratively in Agile teams
- Translate business requirements into technical data solutions
- Manage multiple tasks and deliver within deadlines in a fast-paced environment
What we bring to the table
- Opportunity to work on modern data engineering stack including Databricks and Python
- 6-month engagement duration with potential for extension based on performance
- Exposure to large-scale data engineering projects in an international environment
- Agile-driven collaborative working culture
Let’s Connect
Want to discuss this opportunity in more detail? Feel free to reach out.