Senior Data Engineer - Azure Databricks, PySpark & ML Ops
Posted Updated
Description
EPAM Systems seeks a Senior/Lead Data Software Engineer to build a scalable ML-ready platform, enhancing portfolio model development with advanced data governance and AI capabilities.
You will migrate from an IaaS Big Data platform to Azure-native Databricks, optimize data workflows, and improve data quality while collaborating with cross-functional teams to deliver compliant, high-value data solutions.
Migrate and optimize 500+ data jobs using Azure Databricks Manage and process 12 TB of data efficiently Tune ML models for Azure environments using Java Spark and Delta tables Develop and maintain ETL/ELT pipelines with PySpark Collaborate to integrate GenAI capabilities into data workflows Monitor data quality and implement improvements for reliability 3+ years in data engineering or related roles Proficiency with Python and Spark Experience with Azure Databricks and PySpark Experience migrating big data platforms to Azure-native services Knowledge of data governance and regulatory compliance Hybrid by design Remote work within Poland Relocation opportunities Stock Purchase Plan English classes Health insuranceSkills
Sign in to read and join the discussion.