Data Engineer | Data Platform & Analytics
Posted Updated
Summary
Data engineer who designs and maintains ETL pipelines, data models and CI/CD workflows for statistical production data, migrating legacy processes to modern tooling. Core stack: Python, SQL, Polars/pandas, Git/GitLab and Linux, with cloud and workflow orchestration experience a plus.
Job Responsibilities
Design, develop and maintain efficient and reliable ETL processes and data pipelines for data ingestion, transformation and integration from multiple source agencies.
Develop and maintain modular, production-quality code for statistical production processes while preserving existing business logic and ensuring outputs are accurately reconciled against current production results.
Support the migration, modernisation and optimisation of legacy processes to improve efficiency, maintainability and reliability.
Design and maintain database schemas, data models and storage structures to support the efficient ingestion, validation, transformation and retrieval of large datasets.
Develop reusable data engineering components, including common libraries, configuration frameworks, logging standards and utilities to streamline production workflows.
Implement and maintain CI/CD pipelines for data engineering workflows, including automated testing and deployment.
Manage workflows across Development, UAT and Production environments to support safe testing, deployment and synchronisation.
Design and configure complex data pipeline workflows and schedules, including dependencies, failure handling, recovery and operational monitoring.
Establish and maintain version control structures using GitLab, including branching strategies, merge requests, code reviews and technical documentation.
Monitor and troubleshoot data pipelines, conduct root-cause analysis and continuously improve system reliability and performance.
Collaborate with technical teams, business users and stakeholders to translate business and statistical processing requirements into robust technical solutions.
Job Requirements 2 years of professional experience in data engineering, data platform development, software engineering or a related field. Strong proficiency in Python and SQL, with experience in complex data transformations, analysis and data quality monitoring. Hands-on experience with dataframe processing libraries such as Polars or pandas for processing large datasets efficiently. Experience with database management, data warehousing and ETL development for data ingestion and integration. Experience with Git and GitLab, including branching strategies, merge requests, code reviews and CI/CD pipelines. Working knowledge of Linux/Unix environments, shell scripting and command-line tools. Understanding of data governance, access control and security practices. Knowledge of software engineering practices including modular design, configuration management, logging, exception handling, testing and technical documentation. Experience with cloud technologies, workflow orchestration or CI/CD tools will be an advantage. Strong problem-solving and analytical skills with good communication and stakeholder management abilities. Ability to work independently and collaboratively within a technical team.
Interested candidates are invited to apply via the 'Apply Now' button. We regret that only shortlisted candidates will be notified. By submitting your personal data and CV, you consent to Nexbridge Recruitment Pte Ltd collecting, using and disclosing your personal data for recruitment purposes. Nexbridge Recruitment Pte Ltd EA License No: 26C3616 EA Personnel: Ching Ning Xin Registration No: R22108651
Job Requirements 2 years of professional experience in data engineering, data platform development, software engineering or a related field. Strong proficiency in Python and SQL, with experience in complex data transformations, analysis and data quality monitoring. Hands-on experience with dataframe processing libraries such as Polars or pandas for processing large datasets efficiently. Experience with database management, data warehousing and ETL development for data ingestion and integration. Experience with Git and GitLab, including branching strategies, merge requests, code reviews and CI/CD pipelines. Working knowledge of Linux/Unix environments, shell scripting and command-line tools. Understanding of data governance, access control and security practices. Knowledge of software engineering practices including modular design, configuration management, logging, exception handling, testing and technical documentation. Experience with cloud technologies, workflow orchestration or CI/CD tools will be an advantage. Strong problem-solving and analytical skills with good communication and stakeholder management abilities. Ability to work independently and collaboratively within a technical team.
Interested candidates are invited to apply via the 'Apply Now' button. We regret that only shortlisted candidates will be notified. By submitting your personal data and CV, you consent to Nexbridge Recruitment Pte Ltd collecting, using and disclosing your personal data for recruitment purposes. Nexbridge Recruitment Pte Ltd EA License No: 26C3616 EA Personnel: Ching Ning Xin Registration No: R22108651