Lead Data Engineer
Posted Updated
Summary
The Lead Data Engineer will design and develop scalable data platforms on GCP, building batch and streaming pipelines while providing technical leadership and mentorship to the team. The role requires extensive experience with big data technologies like Spark, Hadoop, and cloud-based data architecture.
Responsibilities
- Lead the design and development of scalable data platforms on GCP.
- Build and optimize batch and streaming data pipelines.
- Define data architecture, standards, and engineering best practices.
- Mentor data engineers and provide technical leadership.
- Collaborate with business and technical stakeholders.
- Drive Agile delivery and continuous improvement initiatives.
Requirements
- 8+ years of Data Engineering experience.
- Strong expertise in GCP, PySpark, Apache Spark, Scala or Python, SQL, Airflow
- Hands-on experience with Hadoop and Hive.
- Knowledge of cloud architecture and design patterns.
- Experience with Linux, Git, Jenkins, Ansible, and JIRA.
- Strong debugging and performance tuning skills.
- Openness to AI-driven technologies and solutions.