Lead Data Engineer
Posted Updated 29
views
Summary
Lead Data Engineer in Chennai designing and building scalable data pipelines and architectures on Azure using Python, PySpark, and Databricks to support analytics and BI. The role covers data quality, workflow optimization, and mentoring junior data engineers.
We are seeking a highly skilled Lead Data Engineer to join our dynamic team in Tamil Nadu. The ideal candidate will have extensive experience in data engineering, with a strong focus on Python, Azure, Pyspark, and Databricks. As a Lead Data Engineer, you will be responsible for designing and implementing robust data pipelines, ensuring data quality, and optimizing data workflows to support our analytics and business intelligence initiatives.
Key Responsibilities:
- Lead the design and development of scalable data architectures and pipelines using Python and Azure technologies.
- Collaborate with data scientists and analysts to understand data requirements and deliver high-quality data solutions.
- Implement data processing frameworks using Pyspark and Databricks to handle large datasets efficiently.
- Ensure data integrity and quality through rigorous testing and validation processes.
- Mentor and guide junior data engineers, fostering a culture of continuous learning and improvement within the team.
- Stay updated with the latest industry trends and technologies to drive innovation in data engineering practices.
Required Skills:
- Proficiency in Python programming and experience with data engineering frameworks.
- Strong knowledge of Azure cloud services and data storage solutions.
- Experience with Pyspark and Databricks for big data processing.
- Excellent problem-solving skills and the ability to work in a fast-paced environment.
- Strong communication skills to collaborate effectively with cross-functional teams.
If you are passionate about data engineering and looking to take your career to the next level, we encourage you to apply for this exciting opportunity.