Senior data engineer: oracle cloud data & ml pipelines
Summary
Owns the data layer for a retail investments technology team in Centurion: building and running cloud (AWS-aligned) ingestion and transformation pipelines with error handling, monitoring, reconciliation and CI/CD, while coaching engineers and setting technical direction. Requires a relevant bachelor's degree and a Microsoft Fabric and/or Azure data certification.
The reporting is only as good as the pipeline underneath it. This role owns that layer: the ingestion, the transformations, the checks, and the support when a number downstream does not match.
It is a permanent seat in Centurion, in a retail investments technology team, reporting to the Head of Application Development. You will build and run data pipelines and the platform they sit on, so analytics, BI and the business get data they can trust. You will also set technical direction in the team. That means design calls, engineering standards, and how AI is used in the work without breaking governance.
What you will do
Deliver data engineering work inside an Agile team. Plan it, estimate it, break features into tasks, and hit the quality bar you agreed with the product owner. Design and build ingestion and transformation pipelines across systems and domains. Batch and incremental. Error handling, monitoring, alerting, validation and reconciliation included. Shape solutions that fit the target data platform. The environment is cloud and AWS-aligned. You will be expected to call the cost, security, performance and support impact of a design before it is built. Coach less experienced data engineers. Give analytics, BI and data science a clear view of structures, availability and how a pipeline actually behaves. Review code and configuration. Simplify what is already there. Use Git, automated testing, CI/CD and structured releases. Document enough that someone else can support it. Use AI in the engineering workflow, on approved tools only. You validate the output. You do not put client or proprietary data into a public model. You stay accountable for what goes to production. Join incident response and root-cause work when a pipeline fails, and build privacy, access control and regulatory requirements in from the start.
Requirements Bachelor's degree in Computer Science, Information Systems, Data Engineering or a related field. Relevant Microsoft Fabric and/or Azure data certification. #J-18808-Ljbffr
