Data Engineer - Python & AWS (Public Sector)
Posted Updated 2
views
Summary
1-year renewable contract (hybrid) on a Singapore government project, maintaining the ODIN HR data platform within PSD. Day to day the engineer builds ETL pipelines, SQL/Python transformations and data quality checks, and delivers AWS Athena datasets and Tableau reporting for downstream HR data consumers.
1-year contract, renewable
Hybrid work arrangement
Government project
Role Overview
The Data Engineer will join the Data Engineering & Infrastructure team to support the operations, maintenance, and enhancement of ODIN, working under the direction of the Team Lead and alongside PSD officers, appointed vendors, source system owners, and business users. The role provides hands-on data engineering capacity for ODIN's ingestion, transformation, data quality, and downstream data services.
This is a hands-on, delivery-focused role. Responsibilities include developing and maintaining ETL pipelines and processing logic, working with existing ODIN data models and datasets, investigating legacy and inconsistent HR data, automating validation and operational processes, executing testing, maintaining technical documentation, and enabling reliable downstream consumption of HR data. The role operates within established PSD processes, security requirements, technical standards, and delivery priorities key decisions, stakeholder commitments, and approvals remain with the relevant PSD officers and system owners.
Key Responsibilities
Data Pipeline & Interface Engineering
- Develop and maintain ETL pipelines and file-based interfaces for ingesting HR data into ODIN, including data profiling, mapping, specifications, and dataset onboarding Data Transformation & Modelling
- Build and maintain SQL and Python logic to cleanse, standardise, transform, and reconcile HR data into reliable, reusable datasets, including historical and legacy data Data Model & Dataset Maintenance
- Enhance ODIN data models, tables, and dependencies to meet new requirements, with traceable mappings, definitions, and processing rules System Enhancements & Testing
- Support CRs and SRs by reviewing requirements, executing tests, validating outputs, documenting defects, and resolving issues through implementation Data Quality, Validation & Lineage
- Automate validation, reconciliation, and cleansing checks investigate anomalies, trace lineage, resolve root causes, and verify fixes Pipeline Operations & Reliability
- Monitor pipelines and interfaces, troubleshoot failures, perform reruns, and improve reliability, performance, maintainability, and operational documentation Downstream Data Consumption
- Develop governed datasets, Athena tables/views and queries, and Tableau data sources/workbooks for approved reporting and analytics needs Technical Collaboration & Delivery
- Translate stakeholder requirements into data mappings, transformation rules, and technical solutions, while documenting decisions and supporting timely delivery Audit & Governance Support
- Provide technical evidence for audits, access reviews, and assurance activities, and support required remediation Requirements Technical Skills
- Strong proficiency in SQL and working proficiency in Python, with hands-on experience developing data pipelines, transformations, validation, and automation. Experience with AWS data services such as S3 ingestion, CSV/Parquet, IAM, partitioning, and Athena tables, views, and queries (or comparable cloud data platforms) is preferred. Familiarity with Spark, Databricks, or similar modern data engineering technologies is advantageous but not required for the current ODIN environment Data Engineering & Data Warehousing
- Hands-on understanding of ETL, data pipeline design and operations, relational and file-based data, data warehousing, and data modelling. Able to work with complex or legacy source structures and transform them into clean, standardised, and reliable datasets for downstream consumption Data Quality, Governance & Troubleshooting
- Strong analytical and problem-solving skills, with practical experience implementing data validation and reconciliation checks, investigating discrepancies, tracing data lineage, and identifying root causes across source, interface, and transformation layers Thinking Clearly & Making Sound Judgement
- Ability to examine complex problems, assess dependencies and risks, and propose workable solutions or escalation paths with clear supporting analysis Working Effectively with Stakeholders
- Strong communication and collaboration skills, with the ability to work effectively with business users, technical teams, source system owners, and vendors, and to explain technical findings clearly HR Data Management & Visualisation
- Keen interest in HR data or workforce processes. Experience with Tableau or similar BI tools, including development or maintenance of datasets and workbooks, is advantageous Delivery & Collaboration
- Able to work independently, manage priorities, and deliver quality engineering work on time. Maintains clear documentation, follows change, UAT, and incident processes, translates requirements into practical solutions, collaborates effectively with stakeholders, and escalates risks promptly /Ref: T24-239
- Develop and maintain ETL pipelines and file-based interfaces for ingesting HR data into ODIN, including data profiling, mapping, specifications, and dataset onboarding Data Transformation & Modelling
- Build and maintain SQL and Python logic to cleanse, standardise, transform, and reconcile HR data into reliable, reusable datasets, including historical and legacy data Data Model & Dataset Maintenance
- Enhance ODIN data models, tables, and dependencies to meet new requirements, with traceable mappings, definitions, and processing rules System Enhancements & Testing
- Support CRs and SRs by reviewing requirements, executing tests, validating outputs, documenting defects, and resolving issues through implementation Data Quality, Validation & Lineage
- Automate validation, reconciliation, and cleansing checks investigate anomalies, trace lineage, resolve root causes, and verify fixes Pipeline Operations & Reliability
- Monitor pipelines and interfaces, troubleshoot failures, perform reruns, and improve reliability, performance, maintainability, and operational documentation Downstream Data Consumption
- Develop governed datasets, Athena tables/views and queries, and Tableau data sources/workbooks for approved reporting and analytics needs Technical Collaboration & Delivery
- Translate stakeholder requirements into data mappings, transformation rules, and technical solutions, while documenting decisions and supporting timely delivery Audit & Governance Support
- Provide technical evidence for audits, access reviews, and assurance activities, and support required remediation Requirements Technical Skills
- Strong proficiency in SQL and working proficiency in Python, with hands-on experience developing data pipelines, transformations, validation, and automation. Experience with AWS data services such as S3 ingestion, CSV/Parquet, IAM, partitioning, and Athena tables, views, and queries (or comparable cloud data platforms) is preferred. Familiarity with Spark, Databricks, or similar modern data engineering technologies is advantageous but not required for the current ODIN environment Data Engineering & Data Warehousing
- Hands-on understanding of ETL, data pipeline design and operations, relational and file-based data, data warehousing, and data modelling. Able to work with complex or legacy source structures and transform them into clean, standardised, and reliable datasets for downstream consumption Data Quality, Governance & Troubleshooting
- Strong analytical and problem-solving skills, with practical experience implementing data validation and reconciliation checks, investigating discrepancies, tracing data lineage, and identifying root causes across source, interface, and transformation layers Thinking Clearly & Making Sound Judgement
- Ability to examine complex problems, assess dependencies and risks, and propose workable solutions or escalation paths with clear supporting analysis Working Effectively with Stakeholders
- Strong communication and collaboration skills, with the ability to work effectively with business users, technical teams, source system owners, and vendors, and to explain technical findings clearly HR Data Management & Visualisation
- Keen interest in HR data or workforce processes. Experience with Tableau or similar BI tools, including development or maintenance of datasets and workbooks, is advantageous Delivery & Collaboration
- Able to work independently, manage priorities, and deliver quality engineering work on time. Maintains clear documentation, follows change, UAT, and incident processes, translates requirements into practical solutions, collaborates effectively with stakeholders, and escalates risks promptly /Ref: T24-239