Senior PySpark Data Engineer – Big Data & Cloud Integration
Summary
Senior data engineer at ITDS Polska working for a client in the financial industry, converting experimental code into production-ready, high-performance PySpark/Spark-on-YARN data pipelines and integrating them with GCP. Kraków-based hybrid role.
Nasze wymagania:
- 5+ years of experience in data engineering and PySpark/Spark development.
- Strong SQL skills and experience with Spark optimization techniques.
- Hands‑on experience with the Hadoop ecosystem, especially Spark on YARN.
- Knowledge of cloud platforms, preferably GCP, is a strong plus.
Mile widziane:
- Certifications or additional experience in cloud computing or big data solutions.
O projekcie:
As a Senior PySpark Data Engineer, you will be working for our client, a leader in large-scale data engineering and cloud integration. You will play a pivotal role in converting cutting-edge experimental code into robust, production-ready data pipelines, enabling innovative solutions across diverse projects. Join a dynamic environment where your expertise accelerates data-driven decision-making and fosters technological advancement.
Unleash the power of data — transform experimental insights into scalable, high-performance workflows!
Kraków-based opportunity with hybrid work model.
Only candidates with an existing legal right to work in Europe will be considered for this role.
Zakres obowiązków:
- Convert experimental code into scalable, efficient workflows and modules.
- Develop and optimize data pipelines using PySpark and Spark on YARN.
- Collaborate with data scientists and engineers to productionize analytical solutions.
- Ensure code quality, perform performance tuning, and implement best data engineering practices.
- Maintain and improve performance of data workflows in a large-scale environment.
- Integrate data pipelines within cloud ecosystems, primarily Google Cloud Platform (GCP).
- Support continuous improvement through performance monitoring and optimization.
Oferujemy:
- Stable and long-term cooperation with very good conditions
- Enhance your skills and develop your expertise in the financial industry
- Work on the most strategic projects available in the market
- Define your career roadmap and develop yourself in the best and fastest possible way by delivering strategic projects for different clients of ITDS over several years
- Participate in Social Events, training, and work in an international environment
- Access to attractive Medical Package
- Access to Multisport Program
- Access to Pluralsight
- Flexible hours