Data Engineer/Senior Data engineer-Python, PySpark, SQL, Databricks
Summary
Data Engineer/Senior Data Engineer working from the office in Singapore, with 5+ years building reliable, scalable data pipelines. Day-to-day work centers on Python, PySpark, SQL, Databricks and AWS services (S3, Glue, Redshift, Kinesis), plus lakehouse architecture, CI/CD, and pipeline observability.
We're Hiring: Data Engineer/Senior Data Engineer!
We are looking for a skilled and driven Data Engineer/Senior Data Engineer to join our team in Singapore. The ideal candidate will have at least 5 years of experience designing, building, and maintaining robust data pipelines and architectures. You will play a key role in ensuring the reliability, scalability, and performance of our data infrastructure to support business intelligence and analytics initiatives.
Location: Singapore, Singapore
Work Mode: Work from Office
Role: Data Engineer/Senior Data Engineer
What You'll Do:B.
Technical & Professional CompetenciesEssential (Must-Have)
- Strong Python, PySpark and SQL capability, including distributed processing, complex transformations, queryoptimisation, debugging and automated testing.
- Hands-on experience with Databricks, Apache Spark and Delta Lake for production-scale batch or streamingdata engineering.
- Hands-on AWS data engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, Lambda,Step Functions, Event Bridge, CloudWatch, Redshift and MSK/Kinesis.
- Experience designing lakehouse or data-lake architectures, including layered data patterns, dimensional ordomain data modelling and reusable curated datasets.
- Experience implementing incremental ingestion, change data capture, idempotency, late-arriving data handling,backfills, replay and recovery.
- Experience implementing data quality, schema validation, reconciliation, metadata and lineage as automatedparts of production pipelines.
- Experience with workflow orchestration, Git-based development, CI/CD, infrastructure as code such asTerraform, and environment promotion controls.
- Ability to implement pipeline observability, operational dashboards, alerting, runbooks, incident diagnosis andservice recovery.
- Working knowledge of cloud security, least-privilege access, network boundaries, encryption, secretmanagement, data classification and audit logging.
- Ability to review technical designs and code, establish engineering standards, and guide less-experiencedengineers.
Eager to shape the future of data?
Skills
- Analytics
- AWS
- Business Intelligence
- CI/CD
- Cloud
- Cloud Security
- CloudWatch
- Data Engineering
- Data Lake
- Data Modeling
- Data Pipelines
- Data Quality
- Databricks
- Delta Lake
- Git
- IAM
- Infrastructure as Code
- Kinesis
- Lakehouse
- Lambda
- Observability
- PySpark
- Python
- Redshift
- S3
- Spark
- SQL
- Test Automation
- Workflow Orchestration