Lead Data Engineer
Summary
Lead a team building and optimizing scalable ETL/ELT pipelines in Python using Hadoop, Spark and Splunk to deliver reliable data for HR analytics and reporting.
You will
- Design, develop and optimize scalable ETL/ELT data pipelines.
- Build and maintain data ingestion and transformation processes for structured, semi-structured and unstructured data.
- Work with enterprise data platforms based on Hadoop, Spark and Splunk.
- Develop data engineering solutions using Python while ensuring high quality, performance and maintainability.
- Collaborate closely with BI, Analytics and cross-functional engineering teams to deliver reliable data solutions.
- Support CI/CD implementation and follow software engineering best practices, including source control and automated testing.
- Ensure service resilience, sustainability, and compliance with governance and risk management standards.
- Troubleshoot and resolve complex data engineering challenges in production environments.
- Mentor junior engineers and share technical knowledge across the team.
- Contribute to continuous improvement by promoting engineering best practices and innovative technologies.
Kogo poszukujemy?
Must have
- Strong commercial experience as a Data Engineer in enterprise environments.
- Hands-on experience with Hadoop, Spark, and Splunk.
- Strong Python skills with experience building ETL/ELT pipelines.
- Experience with data modelling and designing scalable data pipelines.
- Experience working with SQL and NoSQL databases.
- Experience processing raw, structured, semi-structured and unstructured data.
- Good understanding of the complete data lifecycle, including ingestion, transformation and delivery.
- Experience with Git and CI/CD pipelines.
- Strong analytical and problem-solving skills.
- Experience collaborating with BI and Analytics teams.
- Ability to mentor less experienced engineers and provide technical guidance.
- Excellent communication and stakeholder management skills.
- Understanding of engineering governance, risk management, and compliance principles.
Nice to have
- Experience working in Agile environments (Scrum, Kanban).
- Experience leading technical initiatives or coordinating multiple engineering projects.
- Experience implementing automated unit and integration testing.
- Passion for coaching and supporting the professional development of other engineers.
- Experience working with external vendors and cross-functional stakeholders.