Data Engineer (ODIN)
The Role
The Data Engineer will be deployed as part of the Data Engineering & Infrastructure team to support the operations, maintenance and enhancement of ODIN. You will work under the direction of the Team Lead and alongside PSD officers, appointed vendors, source system owners and business users, providing hands-on data engineering capacity for ODIN's ingestion, transformation, data quality and downstream data services.
The role is hands-on and delivery focused. You will develop and maintain ETL pipelines and processing logic, work with existing ODIN data models and datasets, investigate legacy and inconsistent HR data, automate validation and operational processes, execute testing, maintain technical documentation, and enable reliable downstream consumption of HR data. You are expected to work within established PSD processes, security requirements, technical standards and delivery priorities. Key decisions, stakeholder commitments and approvals remain with the relevant PSD officers and system owners.
Key Responsibilities
Data Pipeline & Interface Engineering: Develop and maintain ETL pipelines and file-based interfaces for ingesting HR data into ODIN, including data profiling, mapping, specifications and dataset onboarding.
Data Transformation & Modelling: Build and maintain SQL and Python logic to cleanse, standardise, transform and reconcile HR data into reliable, reusable datasets, including historical and legacy data.
Data Model & Dataset Maintenance: Enhance ODIN data models, tables and dependencies to meet new requirements, with traceable mappings, definitions and processing rules.
System Enhancements & Testing: Support CRs and SRs by reviewing requirements, executing tests, validating outputs, documenting defects and resolving issues through implementation.
Data Quality, Validation & Lineage: Automate validation, reconciliation and cleansing checks; investigate anomalies, trace lineage, resolve root causes and verify fixes.
Pipeline Operations & Reliability: Monitor pipelines and interfaces, troubleshoot failures, perform reruns and improve reliability, performance, maintainability and operational documentation.
Downstream Data Consumption: Develop governed datasets, Athena tables/views and queries, and Tableau data sources/workbooks for approved reporting and analytics needs.
Technical Collaboration & Delivery: Translate stakeholder requirements into data mappings, transformation rules and technical solutions, while documenting decisions and supporting timely delivery.
Audit & Governance Support: Provide technical evidence for audits, access reviews and assurance activities, and support required remediation.
Requirements
An ideal candidate should possess the following:
[Technical Skills] Strong proficiency in SQL and working proficiency in Python, with hands-on experience developing data pipelines, transformations, validation and automation. Experience with AWS data services such as S3 ingestion, CSV/Parquet, IAM, portioning, and Athena tables, views and queries, or comparable cloud data platforms, is preferred. Familiarity with Spark, Databricks or similar modern data engineering technologies is advantageous but not required for the current ODIN environment.
[Data Engineering & Data Warehousing] Hands-on understanding of ETL, data pipeline design and operations, relational and file-based data, data warehousing and data modelling. Able to work with complex or legacy source structures and transform them into clean, standardised and reliable datasets for downstream consumption.
[Data Quality, Governance & Troubleshooting] Strong analytical and problem-solving skills, with practical experience implementing data validation and reconciliation checks, investigating discrepancies, tracing data lineage and identifying root causes across source, interface and transformation layers.
[Thinking Clearly & Making Sound Judgement] Ability to examine complex problems, assess dependencies and risks, and propose workable solutions or escalation paths with clear supporting analysis.
[Working Effectively with Stakeholders] Strong communication and collaboration skills, with the ability to work effectively with business users, technical teams, source system owners and vendors, and to explain technical findings clearly.
[HR Data Management & Visualisation] Keen interest in HR data or workforce processes. Experience with Tableau or similar BI tools, including development or maintenance of datasets and workbooks is advantageous.
[Delivery & Collaboration] Able to work independently, manage priorities and deliver quality engineering work on time. Maintains clear documentation, follows change, UAT and incident processes, translates requirements into practical solutions, collaborates effectively with stakeholders and escalates risks promptly.