Data Engineer Lead

Overview

We are looking for a Lead Data Engineer who will help in implementing data analytics products using on-prem and cloud data platforms.

Position Responsibilities

• Design and build scalable, cloud-native data platforms for modern data engineering practices.
• Mentor and guide other engineers, sharing knowledge, reviewing code, and fostering a culture of curiosity, growth, and continuous improvement.
• Create robust, maintainable ETL/ELT pipelines that integrate with diverse systems and serve business-critical use cases.
• Lead by example-write high-quality, testable code and participate in architecture and design discussions with a long-term view in mind.
• Decompose complex problems into modular, efficient, and scalable components that align with platform and product goals.
• Support data governance and quality efforts, ensuring data lineage, cataloging, and access management are built into the platform.
• Participate in architectural discussions, iteration planning, and feature sizing meetings

SKILLS

• Strong hands-on experience in data engineering building production-grade pipelines on big data platforms (Hadoop ecosystem and cloud data platforms - databricks).
• High proficiency in using Python, Spark, Hadoop platforms & tools (Hive, Impala, Airflow, NiFi), SQL to build Big Data products.
• Hands-on experience with cloud data platforms such as databricks, snowflake (databricks preferred)
• Experience with orchestration/integration tools such as Apache Airflow, Apache NiFi, or Talend.
• Working knowledge of DevOps/CI-CD practices: version control (Git), automated testing, release pipelines, and observability.
• Strong problem-solving skills with the ability to debug complex data issues and communicate clearly with technical and non-technical stakeholders.
• Experience developing Java based applications is an added advantage

hybrid GDL offices techM

remote