Data Engineer
Posted
Summary
Senior data engineer (6-10 years) building and operating production-scale batch/streaming data pipelines on AWS and Databricks. Core stack: Python, PySpark, SQL, Apache Spark, Delta Lake, plus orchestration, CI/CD, Terraform, data quality and pipeline observability, with mentoring and technical leadership.
Avensys is a reputed global IT professional services company headquartered in Singapore. Our service spectrum includes enterprise solution consulting, business intelligence, business process automation and managed services. Given our decade of success, we have evolved to become one of the top trusted providers in Singapore and service a client base across banking and financial services, insurance, information technology, healthcare, retail and supply chain.
We are currently looking to hire highly skilled Data Engineer. This is an exciting opportunity to expand your skill set, achieve job satisfaction and work-life balance. More details as below.
Requirements & Qualifications Educational Qualification & Experience Education: Degree in Computer Science, Software Engineering, Data Engineering, Information Systems or a related technical discipline. Experience: Typically 6–10 years of relevant data or software engineering experience, including hands-on design and operation of production data pipelines and at least 2 years working with cloud-based data platforms. Candidates should demonstrate technical leadership through architecture, engineering standards, code review, mentoring or ownership of critical data services. B. Technical & Professional Competencies Essential (Must-Have) Strong Python, PySpark and SQL capability, including distributed processing, complex transformations, query optimisation, debugging and automated testing. Hands-on experience with Databricks, Apache Spark and Delta Lake for production-scale batch or streaming data engineering. Hands-on AWS data engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions, EventBridge, CloudWatch, Redshift and MSK/Kinesis. Experience designing lakehouse or data-lake architectures, including layered data patterns, dimensional or domain data modelling and reusable curated datasets. Experience implementing incremental ingestion, change data capture, idempotency, late-arriving data handling, backfills, replay and recovery. Experience implementing data quality, schema validation, reconciliation, metadata and lineage as automated parts of production pipelines. Experience with workflow orchestration, Git-based development, CI/CD, infrastructure as code such as Terraform, and environment promotion controls. Ability to implement pipeline observability, operational dashboards, alerting, runbooks, incident diagnosis and service recovery. Working knowledge of cloud security, least-privilege access, network boundaries, encryption, secret management, data classification and audit logging. Ability to review technical designs and code, establish engineering standards, and guide less-experienced engineers.
Preferred (Advantageous) Experience with Databricks Unity Catalog, Lakeflow or Delta Live Tables, Auto Loader, Structured Streaming, Delta Sharing and Databricks Asset Bundles. Experience with Apache Airflow, dbt, Kafka, Amazon MSK/Kinesis, AWS DMS or equivalent ingestion and orchestration technologies. Experience designing data products, data contracts, semantic or context layers, master/reference data patterns or data mesh-aligned operating models. Experience supporting feature engineering, ML training datasets, feature stores or MLOps pipelines on Databricks and AWS. Experience in Singapore public-sector environments, including IM8-aligned controls, data classification, VAPT, system assurance or governed cross-agency data sharing. Relevant certifications such as Databricks Certified Data Engineer Professional, AWS Certified Data Engineer – Associate or AWS Certified Solutions Architect – Associate/Professional
WHAT'S ON OFFER: You will be remunerated with an excellent base salary and entitled to attractive company benefits. Additionally, you will get the opportunity to enjoy a fun and collaborative work environment, alongside a strong career progression.
To submit your application, please apply online or email your UPDATED CV in Microsoft Word format to
(HIDDEN TEXT)
Your interest will be treated with strict confidentiality.
CONSULTANT DETAILS : Consultant Name : Bhawana Sharma Avensys Consulting Pte Ltd
Privacy Statement :
Data collected will be used for recruitment purposes only. Personal data provided will be used strictly in accordance with the relevant data protection law and Avensys' privacy policy.
We are currently looking to hire highly skilled Data Engineer. This is an exciting opportunity to expand your skill set, achieve job satisfaction and work-life balance. More details as below.
Requirements & Qualifications Educational Qualification & Experience Education: Degree in Computer Science, Software Engineering, Data Engineering, Information Systems or a related technical discipline. Experience: Typically 6–10 years of relevant data or software engineering experience, including hands-on design and operation of production data pipelines and at least 2 years working with cloud-based data platforms. Candidates should demonstrate technical leadership through architecture, engineering standards, code review, mentoring or ownership of critical data services. B. Technical & Professional Competencies Essential (Must-Have) Strong Python, PySpark and SQL capability, including distributed processing, complex transformations, query optimisation, debugging and automated testing. Hands-on experience with Databricks, Apache Spark and Delta Lake for production-scale batch or streaming data engineering. Hands-on AWS data engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, Lambda, Step Functions, EventBridge, CloudWatch, Redshift and MSK/Kinesis. Experience designing lakehouse or data-lake architectures, including layered data patterns, dimensional or domain data modelling and reusable curated datasets. Experience implementing incremental ingestion, change data capture, idempotency, late-arriving data handling, backfills, replay and recovery. Experience implementing data quality, schema validation, reconciliation, metadata and lineage as automated parts of production pipelines. Experience with workflow orchestration, Git-based development, CI/CD, infrastructure as code such as Terraform, and environment promotion controls. Ability to implement pipeline observability, operational dashboards, alerting, runbooks, incident diagnosis and service recovery. Working knowledge of cloud security, least-privilege access, network boundaries, encryption, secret management, data classification and audit logging. Ability to review technical designs and code, establish engineering standards, and guide less-experienced engineers.
Preferred (Advantageous) Experience with Databricks Unity Catalog, Lakeflow or Delta Live Tables, Auto Loader, Structured Streaming, Delta Sharing and Databricks Asset Bundles. Experience with Apache Airflow, dbt, Kafka, Amazon MSK/Kinesis, AWS DMS or equivalent ingestion and orchestration technologies. Experience designing data products, data contracts, semantic or context layers, master/reference data patterns or data mesh-aligned operating models. Experience supporting feature engineering, ML training datasets, feature stores or MLOps pipelines on Databricks and AWS. Experience in Singapore public-sector environments, including IM8-aligned controls, data classification, VAPT, system assurance or governed cross-agency data sharing. Relevant certifications such as Databricks Certified Data Engineer Professional, AWS Certified Data Engineer – Associate or AWS Certified Solutions Architect – Associate/Professional
WHAT'S ON OFFER: You will be remunerated with an excellent base salary and entitled to attractive company benefits. Additionally, you will get the opportunity to enjoy a fun and collaborative work environment, alongside a strong career progression.
To submit your application, please apply online or email your UPDATED CV in Microsoft Word format to
(HIDDEN TEXT)
Your interest will be treated with strict confidentiality.
CONSULTANT DETAILS : Consultant Name : Bhawana Sharma Avensys Consulting Pte Ltd
Privacy Statement :
Data collected will be used for recruitment purposes only. Personal data provided will be used strictly in accordance with the relevant data protection law and Avensys' privacy policy.
Skills
- Airflow
- Automation
- AWS
- Business Intelligence
- CI/CD
- Cloud
- Cloud Security
- CloudWatch
- Data Engineering
- Data Lake
- Data Mesh
- Data Modeling
- Data Pipelines
- Data Quality
- Databricks
- dbt
- Delta Lake
- Eventbridge
- Feature Engineering
- Git
- IAM
- Infrastructure as Code
- Kafka
- Kinesis
- Lakehouse
- Lambda
- Machine Learning
- MLOps
- Observability
- PySpark
- Python
- Redshift
- S3
- Spark
- SQL
- Terraform
- Test Automation
- Unity
- Unity Catalog
- Workflow Orchestration