Confluent Kafka Specialist
Description
JOB DESCRIPTION
Confluent Kafka Specialist
POSITION TITLE: Confluent Kafka Specialist
NATURE AND NAME OF PROJECT: Kafka Migration
CLIENT: Vodacom
TYPE OF CONTRACT: Fixed Term
LOCATION: Midrand
Job Description
Mission/Role Purpose
The Confluent Kafka Specialist will be responsible for supporting Vodacom on solution architecture technical design of enterprise Kafka, Confluent and Flink environments including the implementation, configuration, deployment and migration from the on-premises Confluent Platform environment to Confluent Cloud across the defined AWS/GCP cloud migration path.
The role will translate the target architecture and architectural standards into implementable technical solutions, supporting migration, cloud integration, security, automation, observability and production-readiness activities.
Responsibilities
Support to the Vodacom Team: Solution Architecture & Design Support
- Detail solution designs for Kafka, Confluent Cloud and Apache Flink solutions.
- Translate enterprise architecture requirements into implementable technical configurations and patterns.
- Design integrations between Kafka, Flink, cloud platforms, applications and downstream data consumers.
- Develop architecture diagrams, interface designs, deployment patterns and technical specifications.
- Document architecture and implementation decisions in Atlassian Confluence.
- Support architecture reviews and ensure implementation remains aligned with approved designs.
Support to the Vodacom Team: Kafka & Confluent
- Design and support Apache Kafka, Confluent Platform and Confluent Cloud solutions.
- Define topics, partitions, replication, retention configurations and consumer/producer integration patterns.
- Support migrations using:
- Cluster Linking
- Mirror Maker 2
- Consumer offset migration
- Design and configure Schema Registry patterns and schema-management approaches.
- Support Kafka Connect and ksqlDB-based integrations.
- Support platform administration and production troubleshooting.
Support to the Vodacom Team: Apache Flink
- Design Flink-based stream-processing applications and integration patterns.
- Support:
- Stateful stream processing
- Checkpoint configuration
- Save point management
- Event-time processing
- Flink SQL
- Design Flink workloads to meet performance, recoverability and availability requirements.
- Support integration between Flink workloads and Kafka/Confluent environments.
Support to the Vodacom Team: Cloud, Networking & Security
- Design Kafka and Flink workloads for AWS and/or GCP environments.
- Support private connectivity using VPN, Private Link, Direct Connect or equivalent services.
- Apply IAM, RBAC, encryption and security requirements to solution designs.
- Work with security and network teams to resolve connectivity and access requirements.
Support to the Vodacom Team: Automation & DevOps
- Develop or support Infrastructure as Code using Terraform.
- Support automated platform provisioning and configuration.
- Define or contribute to CI/CD pipelines for platform and streaming application deployment.
- Ensure configuration and infrastructure changes can be controlled through appropriate version-management and deployment processes.
Support to the Vodacom Team: Monitoring & Production Readiness
- Design monitoring and observability requirements using Grafana, Prometheus and Instana.
- Define monitoring requirements for Kafka brokers, topics, producers, consumers, Flink jobs and platform infrastructure.
- Support troubleshooting of Kafka, Flink and Confluent platform issues.
- Contribute to capacity planning and performance optimization.
- Support high-availability and disaster-recovery implementation and testing.
Migration Support
- Assess existing Kafka workloads and dependencies.
- Assist with migration planning and workload classification.
- Produce detailed migration designs and technical migration runbooks.
- Support migration testing, validation and cutover activities.
- Identify migration risks, dependencies and compatibility issues.
- Support post-migration validation and operational handover.
Required Technical Competencies
- Strong hands-on knowledge of Apache Kafka.
- Strong experience with Confluent Platform and/or Confluent Cloud.
- Good knowledge of Apache Flink and stream-processing architecture.
- Kafka migration experience using Cluster Linking and/or MirrorMaker 2.
- Working knowledge of consumer offset migration.
- Schema Registry.
- Kafka Connect.
- ksqlDB.
- Event-Driven Architecture.
- AWS and/or GCP.
- Cloud networking and private connectivity.
- IAM, RBAC and encryption.
- Terraform.
- CI/CD.
- Grafana, Prometheus and Instana.
- General Kubernetes and cloud-native platform knowledge.
- Strong hands-on expertise in Apache Kafka, including enterprise-scale production environments.
- Strong hands-on experience with Confluent Platform and/or Confluent Cloud, including migration or modernization initiatives.
Experience
- Proven experience designing or supporting enterprise Kafka environments.
- Experience supporting production Kafka platforms.
- Experience with Kafka/Confluent migration or modernization initiatives.
- Experience moving streaming workloads from on-premises infrastructure to cloud environments.
- Experience with Flink-based streaming workloads.
- Experience working within cross-functional architecture, cloud, security, DevOps and data engineering teams.
- Strong troubleshooting and technical documentation capability.
- Proven experience delivering or supporting enterprise-scale Kafka and/or Confluent Cloud environments and migration programmes.
Preferred Certifications
- Confluent Certified Developer for Apache Kafka.
- Confluent Certified Administrator for Apache Kafka.
- AWS or GCP cloud certification.
- HashiCorp Terraform Associate.
- Kubernetes or cloud-native certification.
Skills
Sign in to read and join the discussion.