Senior Java Engineer (Data Engineering)
Summary
Senior Java engineer at a software consultancy in Cape Town (hybrid, ~2 days/week on-site) delivering petabyte-scale data migration, retention and privacy projects. Day-to-day: designing large-scale migrations and batch/streaming pipelines, building lineage/validation frameworks and observability, and mentoring engineers. Core stack: Java, Kubernetes, cloud data platforms.
We’re working with an established software consultancy that partners with everyone from startups to global brands, and they’re looking for a Senior Java Engineer with strong data-engineering expertise.
You’ll join an expert team delivering large-scale, production‑grade data projects — data retention, migration and privacy work at petabyte scale — combining hands‑on engineering with technical leadership.
What you’ll do
- Design and implement large‑scale data migration and transformation strategies across very large datasets
- Build data‑lineage, validation and testing frameworks that keep migrations accurate and data intact
- Develop scalable batch and streaming pipelines with parallel orchestration
- Implement monitoring and observability for pipeline performance and system health
- Provide technical leadership — mentoring engineers, running code reviews, leading design discussions and removing blockers
What you’ll bring
- 8+ years in software / data engineering, including several years of technical leadership
- Advanced Java with modern build systems (Gradle or Maven)
- Proven experience with large‑scale data migration and ETL pipeline development
- Distributed storage (HDFS, S3 or GCS) and a cloud data platform (AWS, GCP, Azure or Databricks)
- Hands‑on Kubernetes for containerised data processing
- A degree in Computer Science, Engineering or equivalent experience
Advantageous
- Scala or Python for data processing, and Apache Spark / PySpark
- Hadoop ecosystem, Kafka or Kinesis streaming, and modern data formats (Iceberg, Parquet, ORC, Avro)
- Observability tooling (Prometheus, Grafana), and data‑privacy / compliance experience (e.g. GDPR)
Working model
Hybrid in Cape Town — around two days a week in the office.