Lead Data Engineer
Summary
Lead Data Engineer at Link Group in Warsaw: lead the design of scalable data platforms on GCP, build and optimize batch and streaming pipelines, define architecture standards, and mentor data engineers. Core stack: GCP, PySpark/Apache Spark, Scala or Python, SQL, Airflow, Hadoop and Hive.
Responsibilities
- Lead the design and development of scalable data platforms on GCP.
- Build and optimize batch and streaming data pipelines.
- Define data architecture, standards, and engineering best practices.
- Mentor data engineers and provide technical leadership.
- Collaborate with business and technical stakeholders.
- Drive Agile delivery and continuous improvement initiatives.
Requirements
- 8+ years of Data Engineering experience.
- Strong expertise in GCP, PySpark, Apache Spark, Scala or Python, SQL, Airflow
- Hands-on experience with Hadoop and Hive.
- Knowledge of cloud architecture and design patterns.
- Experience with Linux, Git, Jenkins, Ansible, and JIRA.
- Strong debugging and performance tuning skills.
- Openness to AI-driven technologies and solutions.