Senior Data Engineer
Summary
A Senior Data Engineer at a technology consultancy in Johannesburg architects and leads scalable batch and real-time data solutions, building ETL pipelines (Talend), processing big data (Hadoop/EMR, Spark/PySpark, Python), and delivering AWS cloud data infrastructure in a hybrid, three-days-in-office model.
We’re working with an established technology consultancy, and they’re looking for a Senior Data Engineer to architect, build and lead scalable, secure data solutions as part of an agile team.
You’ll act as a technical lead — turning complex requirements into high-performing data architecture, and building the infrastructure that turns raw data into insight at scale, in batch and real time.
What you’ll do
- Architect the data-analytics framework and lead batch and real-time analytical solutions
- Translate complex functional and technical requirements into detailed architecture, design and high-performing software
- Lead user-story analysis, design, development, testing and build-automation across multiple projects
- Create data marts using Talend or a similar ETL tool, and manipulate data using Python
- Process data using the Hadoop paradigm (e.g. EMR), and create and support data feeds into AWS Cloud
- Develop for big data and business intelligence, including automated testing and deployment
What you’ll bring
- 6+ years’ data-engineering or software-engineering experience, with 2+ years in big data
- 2+ years with ETL processes and 2+ years’ AWS experience
- Strong Python, and PySpark or Spark
- The ability to architect highly scalable distributed systems using open-source tools
- Expert data modelling, and strong object-oriented design, coding and testing patterns
- A bachelor’s degree in Computer Science, Computer Engineering or equivalent, and AWS certification
Advantageous
- BigQuery, DBT, and a combination of AWS / GCP / Azure
- AI tooling (e.g. Bedrock or similar)
Working model
Hybrid in Johannesburg — three days a week in the office.
#J-18808-Ljbffr