Data Engineer – Databricks | AWS | Azure
Summary
Data Engineer (onsite, Singapore) building ETL/ELT pipelines and Lakehouse/Medallion architecture with Databricks, PySpark, Python and SQL across AWS (S3, Glue, Redshift) and Azure (ADLS, ADF, Synapse). Day-to-day covers batch and real-time pipelines, data quality/governance, and CI/CD with Git.
Salary: $7,000 – $10,000 per month
Data Engineer – Databricks | AWS | Azure
Location: Singapore – Onsite
Experience: 5+ Years
Job Summary
We are looking for a Data Engineer with 5+ years of experience in building scalable data solutions using Databricks, AWS, and Azure.
Key Responsibilities
Develop and maintain ETL/ELT pipelines using Databricks, PySpark, Python, and SQL.
Build Lakehouse/Medallion Architecture using Delta Lake.
Work with AWS services such as S3, Glue, Athena, Redshift and Azure services such as ADLS Gen2, ADF, and Synapse.
Develop batch and real-time data pipelines and optimize Spark workloads.
Implement data modelling, data quality, governance, and performance optimization.
Manage Databricks Jobs/Workflows and production pipelines.
Implement CI/CD using Git, Azure DevOps/GitHub.
Troubleshoot production issues and collaborate with architects and business teams.
Required Skills
Databricks | PySpark | Python | SQL | Apache Spark | Delta Lake | AWS | Azure | S3 | Glue | ADLS | ADF | Synapse | ETL/ELT | Data Modelling | CI/CD | Git
Preferred
Kafka/Spark Streaming, Airflow, Terraform, Docker, Unity Catalog, CDC, and cloud certifications.
Qualification: Bachelor's degree in Computer Science/IT/Engineering or related field.
Requirement: Strong production experience with Databricks and cloud data platforms; willing to work onsite in Singapore.