Data Platform Engineer
Summary
Supports day-to-day operations of a data platform: monitoring and recovering data pipeline jobs, troubleshooting technical incidents, fixing security vulnerabilities, setting up DevOps pipelines, and building dashboards. Core stack is SQL, Python, and QuickSight/Tableau, with AWS/cloud, Terraform/IaC, Talend and MS SQL Server as pluses.
- Support the daily operations of the data platform (e.g. handling error notifications, job monitoring & recovery, testing data pipelines);
- Investigate and troubleshoot reported incidents related to technical setup (e.g. connectivity failure, API timeout, service failure);
- Document changes to existing setup (e.g. data sources, pipelines, accounts);
- Research, propose and document technical requirements;
- Review and implement fixes for reported security vulnerabilities;
- Collaborate with the infrastructure team on the data team requests;
- Support product managers in cloud transformation journey for data products;
- Provide technical support and consultancy to business users;
- Develop reports to monitor usage, performance and security events;
- Generate mockup data and create unit tests;
- Set up DevOps pipelines;
- Optimise performance of data pipelines;
- Review data pipelines to ensure adherence to data management standards, policies and procedures.
Requirements
To meet the challenges of this role, you must have/ be:
- Minimum of Bachelor’s Degree in Computer Science, Computer Engineering, or related disciplines;
- 3-5 years of work experience in Data Engineering, Data Science, Software Engineering or IT Infrastructure
- Proficient in developing dashboards (preferably with QuickSight or Tableau);
- Proficient in SQL scripting to analyze, transform and integrate data sources;
- Proficient in Python;
Good to haves:
- Experience IT infrastructure (server, database, network administration experience is an advantage);
- Experience or certification in Tableau Server administration;
- Experience or certification in Talend administration;
- Experience or certification in MS SQL Server administration;
- Experience or certification in SAS Viya administration;
- Proficient in PowerShell;
- Experience in AWS or cloud services;
- Experience in SHIP-HATS or DevOps tools;
- Experience in Terraform or IaC;
- Experience with agile or other rapid application development methodologies;