Data Engineer - Java & Scala (f/m/x)
Summary
Data Engineer designing and maintaining scalable data pipelines with Scala, Java, and Apache Spark to modernize a backend data processing platform for an investment bank.
O projekcie:
We are looking for a skilled Data Engineer to join our team in Warsaw, working on a project focused on modernizing the backend data processing platform for an investment bank. This role involves collaborating with feature teams to enhance data processing capabilities using Big Data technology on an internal cloud. You will be part of a modern, data-driven environment, with a hybrid work model that includes 2-3 days in the office each week. If you are passionate about backend development and data engineering, this opportunity may be the right fit for you.
Wymagania:
- - Minimum 4 years of experience as a Data Engineer working on projects involving large-scale data processing
- - Strong knowledge of Scala and Apache Spark
- - Practical experience with Java and developing backend applications or data processing solutions
- - Experience designing and implementing ETL/ELT processes and data pipelines
- - Good knowledge of SQL and relational databases
- - Experience working with distributed systems and Big Data environments
- - Advanced knowledge of both Polish and English, and residency in Poland
Codzienne zadania:
- - Designing, developing, and maintaining scalable data pipelines using Scala and Apache Spark
- - Implementing ETL/ELT processes and transformations for large volumes of data
- - Developing and maintaining data processing components using Java
- - Optimizing the performance of Spark applications and data processing workflows
- - Integrating data from various sources and preparing it for further analysis and use by banking systems
- - Collaborating with Data Engineers, Developers, Analysts, and business teams to design and deliver data solutions
- - Monitoring code quality and conducting continuous integration