Lead Python Data Engineer

We seek a skilled Lead Python Data Engineer to join our team and help us build and maintain our data pipelines.

As a Python Data Engineer, you will be responsible for designing, developing, and testing Python-based data pipelines and performing data cleansing, transformation and quality assurance.

Responsibilities

• Design, develop, test and maintain Python-based data pipelines
• Write high-quality and maintainable Python code to migrate data between different storage systems with modifications
• Create ETL jobs, airflow operators, and Spark jobs to enhance data processing capabilities
• Implement Data Intake processes for Data Sources (ETL/ELT, DWH Modelling and Optimization)
• Perform data cleansing, transformation and quality assurance to maintain data integrity
• Optimize data processing, storage and retrieval to improve performance
• Collaborate closely with cross-functional teams to ensure data accuracy and consistency

Requirements

• In-depth learning of Python programming with the ability to write maintainable and scalable code
• Strong understanding of distributed computation and multiprocessing
• Knowledge of data warehouse concepts and ETL processes
• Proficiency in working state machines, web back-end development, networking and protocols
• Experience with cloud computing platforms, such as AWS or GCP, and tools like Step Functions, Batch
• Knowledge of Spark, Hive, Databricks and other data processing frameworks
• Familiarity with data orchestration tools like Airflow
• Expertise in with SQL, NoSQL and cloud-based data storage solutions

Benefits

We connect like-minded people:

• Delivering innovative solutions to industry leaders, making a global impact
• Enjoyable working environment, whether it is the vibrant office or the comfort of your own home
• Opportunity to work abroad for up to two months per year
• Relocation opportunities within our offices in 55+ countries
• Corporate and social events

We invest in your growth:

• Leadership development, career advising, soft skills and well-being programs
• Certifications, including GCP, Azure and AWS
• Unlimited access to EPAM's internal learning database
• Free English classes with certified teachers
• Discounts in local language schools, including online courses for the Kazakh language

We cover it all:

• Participation in the Employee Stock Purchase Plan
• Monetary bonuses for engaging in the referral program
• Medical & family care package
• Six trust days per year (sick leave without a medical certificate)
• Coverage of psychology sessions of your choice
• Benefits package (sports activities, a variety of stores and services)
• Housing support program: preferential mortgage access via Otbasy Bank partnership