Data Engineer
Posted
Summary
Data Engineer who designs, builds, and maintains reliable data pipelines and data-processing workflows, integrating data from APIs, databases, cloud and on-premise systems. Core technologies include Python, SQL, ETL/ELT, cloud platforms (AWS/Azure), CI/CD, Infrastructure as Code, and Docker.
- Design, build, and maintain reliable data pipelines and data-processing workflows.
- Integrate data from APIs, databases, enterprise applications, cloud services, files, and other source systems.
- Develop scalable solutions for data ingestion, transformation, storage, processing, and serving.
- Support data platforms operating across cloud, on-premise, or hybrid environments.
- Implement appropriate monitoring, data-quality checks, error handling, and operational controls.
- Automate deployment and operational processes using CI/CD and Infrastructure as Code where applicable.
- Support production systems, troubleshoot issues, and contribute to root-cause analysis and continuous improvements.
- Work closely with application, infrastructure, platform, security, and business teams to deliver reliable data solutions.
- Maintain relevant technical documentation, operational procedures, and engineering standards.
Qualifications
- Around 4–7 years or more of relevant experience in Data Engineering, Cloud Engineering, Platform Engineering, DevOps, SRE, or a related technical discipline.
- Experience with cloud platforms such as AWS and/or Azure.
Required Skills
- Python and SQL
- ETL / ELT and data pipelines
- Batch, streaming, CDC, or event-driven processing
- Data modelling and data quality
- Cloud data platforms and services
- Logging, monitoring, telemetry, or observability
- CI/CD and Infrastructure as Code
- Docker or containerised environments
- Good understanding of software engineering practices, system reliability, scalability, security, and production support.
- Strong problem-solving skills and the ability to work across technical teams.
Preferred Skills
- Experience working across on-premise and cloud environments.
- Experience with streaming or messaging technologies such as Kafka or MQ.
- Experience with Terraform, Ansible, or equivalent automation tools.
- Familiarity with observability or monitoring platforms such as Grafana, Prometheus, Dynatrace, Elastic, or equivalent.
- Experience working in large-scale enterprise or Government environments.