Lead Data Engineer

• The Lead Data Engineer is responsible for designing, developing, and maintaining scalable data engineering solutions that support the migration from legacy big data platforms to modern, cloud-based data environments. The role ensures reliable data operations while enabling ongoing and new business initiatives.

Duties and Responsibilities

• Design, build, and optimize automated data pipelines, ETL/ELT processes, and data models to ingest, process, and store large volumes of data within cloud-based platforms.


• Support large-scale data migration initiatives, ensuring data accuracy, performance efficiency, and minimal business disruption.


• Develop and maintain ETL/ELT workflows to ingest, transform, and load data from multiple internal and external sources with a focus on scalability and reliability.


• Partner with business, analytics, and product teams to translate data requirements into effective technical solutions that support strategic initiatives.


• Design and deliver data marts and customized data extractions aligned with business and reporting needs.


• Ensure compliance with enterprise data governance, security, and regulatory standards.


• Monitor data pipeline health and performance, troubleshoot data incidents, and implement preventive and corrective measures.


• Document data workflows, schemas, technical specifications, and operational runbooks to support operational stability and knowledge transfer.


• Collaborate closely with product owners, data architects, and data scientists to maintain a reliable and efficient data infrastructure.


• Drive continuous improvement of data engineering practices, tools, and automation frameworks.


• Provide technical guidance and mentorship to junior engineers through code reviews, best-practice sharing, and troubleshooting support.


• Manage and deliver multiple data engineering initiatives concurrently while meeting quality, scope, and timeline expectations.

Required Experience, Technical Knowledge, and Skills

Experience

• At least 8 years of total Data Engineering experience, with strong exposure to large-scale data pipelines, ETL/ELT development, and enterprise or cloud-based data platforms.


• Proven experience designing, building, and optimizing scalable data solutions in modern data environments.

Knowledge

• Python – At least 4 out of 5 proficiency level, with strong hands-on experience in data transformation, automation, and pipeline development.


• SQL – At least 3 out of 5 proficiency level, with demonstrated capability in complex queries, data modeling, and performance tuning.


• Experience working with modern data cloud platforms, such as Databricks and/or Snowflake.


• Platform and Cloud Exposure - Currently working with or has recent hands-on experience in enterprise-grade cloud data ecosystems. With a strong understanding of cloud-native data architecture and best practices.

Soft Skills

• Strong verbal and written communication skills


• Demonstrated leadership and technical influence


• Strong analytical, critical thinking, and problem-solving abilities


• Process orientation with the ability to enforce standards and best practices


• Strong organizational, multitasking, and time-management skills


• Stakeholder and cross-functional collaboration skills