Senior Data Integration Engineer: Cloud ETL & Lakehouse
Posted Updated
Description
EPAM Systems, Inc. seeks a Senior Data Integration Engineer to design and optimize scalable ETL/ELT pipelines using Python, PySpark, and SQL within cloud environments.
You will build robust data products, orchestrate complex workflows with Airflow, and implement lakehouse architectures across Snowflake, Synapse, or Redshift. You will ensure data quality with dbt/Soda and lead CI/CD practices, collaborating with product, engineering, and architecture teams to deliver reliable, scalable data
Design, build, and optimize scalable batch and near-real-time ETL/ELT pipelines using Python, PySpark, SQL and cloud integration engines. Develop and manage complex workflow orchestrations (Airflow or cloud schedulers) and automate ingestion routines. Design and implement modern data warehouse/lakehouse layers (Snowflake, Synapse, Redshift) with optimal partitioning and SCD2 patterns. Establish data quality checks and validation frameworks (dbt, Soda, PySpark tests). Collaborate with product owners and architects to define data requirements and STTM mappings; document data dictionaries. Maintain a clean code repository, lead code reviews, and configure CI/CD pipelines (Azure DevOps, GitLab CI, GitHub Actions) with Docker. 5+ years of hands-on experience in data engineering, data warehousing, database design, and end-to-end data integration. Advanced knowledge of cloud data integration tools (ADF, Glue, Dataflow). Proficiency in SQL (advanced optimization), Python, and PySpark / Apache Spark. Experience with high-performance databases and cloud-native systems (Snowflake, ClickHouse, Azure Synapse, PostgreSQL, MS SQL Server). Familiarity with CDC or real-time streaming tools (Kafka, Debezium). Strong communication skills in English; ability to articulate complex ideas to stakeholders. Private health insurance Well-being events IT equipment providedSkills
Sign in to read and join the discussion.