Lead Data Engineer
Summary
Leads technical delivery of data migration and engineering initiatives: designing scalable data pipelines in GCP, guiding an engineering pod, and promoting development standards within an Agile process. Core stack includes PySpark/Scala, Spark, Hadoop ecosystem tools, Airflow, Python, SQL, and GCP DevOps tooling.
We are GFT Poland
WE KNOW how to tackle complex issues with innovative approach to deliver the highest value. Our reputation has been built around one simple rule: we do not overpromise, WE DELIVER. We deliver to our employees, clients and partners. WE GROW as you grow, so investing in you is our business strategy. Caring for each other is our priority. WE CARE who you are, what you need, how you feel. WE CARE to smile, have fun and develop as human beings.
Why Choose GFT?
- A culture of top performance
- Deep tech IT engineering & consulting
- 1200 skilled & top-class experts
- 77% of the team are regular/senior
- Products that contribute to a sustainable world
- Competitive salary and benefits
- Ambitious projects, trainings and tools you need to flourish
- Google Cloud Partner of the Year - for going above and beyond for customers
What will you do?
You will lead the technical delivery of data migration and engineering initiatives, working closely with Engineers, Data Analysts and Business Analysts. You will design scalable data solutions in GCP, guide the engineering pod, promote development standards and support the team throughout the Agile development process.
We offer
- Hybrid work in our client's office (Kraków) 8 days a month
- Working in a highly experienced and dedicated team
- Benefit package tailored to your needs (medical, sport, lunch subsidy, life insurance, etc.)
- Online training and certifications
- Access to e-learning platform
- Social events
- At least 10 years of experience in software design, data engineering and development in Agile and DevOps environments
- Strong development and solution design experience with PySpark or Scala
- Hands-on experience with Google Cloud Platform and building data pipelines in GCP
- Experience with Apache Spark, Spark SQL, Hadoop, YARN, Hive, MapReduce and ETL frameworks
- Experience with Python, SQL, PL/SQL and data integration
- Experience with Apache Airflow or similar scheduling tools
- Experience with REST APIs and RESTful services
- Strong knowledge of Unix and Linux environments
- Experience with Git, GitHub, Jenkins, Ansible and Jira
- Experience with automated testing, deployment and monitoring
- Understanding of cloud architecture and cloud design patterns
- Ability to lead an engineering pod and guide the work of other engineers
- Ability to work autonomously and act as a technical subject-matter expert
- Strong communication and presentation skills for technical, senior and non-technical audiences
- Strong written and spoken English
- Openness to work hybrid from our client's office (Kraków) 8 days a month
Nice to have
- Experience with Elasticsearch
- Experience developing Java APIs
- Experience with data ingestion solutions
- Experience working with Scrum and Kanban
- Interest in AI-based tools that can accelerate engineering workflows