Data Engineer: ETL Pipelines, Lakehouse & Kubernetes
Summary
Data engineer designing and building scalable ETL/ELT pipelines with Python, PySpark and Spark, managing Iceberg-based lakehouse architectures with metadata and partitioning strategies. Day to day involves deploying pipelines via Kubernetes, Docker and CI/CD in on-prem environments, plus L3 support, data security/governance and stakeholder collaboration.
ADECCO PERSONNEL PTE LTD is seeking a Data Engineer to design and build scalable ETL/ELT data pipelines using Python, PySpark and Spark. You will manage lakehouse architectures with Iceberg tables and implement metadata and partitioning strategies.
Responsibilities include deploying pipelines with Kubernetes, Docker, CI/CD, and DevOps practices in on-prem environments, providing L3 support, and ensuring data security and governance while collaborating with stakeholders.