Senior Software Engineer
Summary
A senior Kafka platform engineer role in Dubai: day-to-day you operate and scale Confluent Kafka clusters, manage topics, partitions, ACLs and disaster recovery, and run upgrades for Confluent, Flink, Kafka Connect and Schema Registry using Bash, Ansible and Python while supporting streaming data pipelines.
Monitor cluster performance, availability, and capacity; implement scaling and tuning as needed.
Manange confluent Platfrom upgrade, Installations, Flink installation upgrades.
Manage topics, partitions, consumer groups, and data retention policies.
Ensure high availability, disaster recovery, and fault tolerance of Kafka clusters.
Implement security policies including ACLs, authentication, and encryption.
Troubleshoot and resolve issues with producers, consumers, and brokers.
Collaborate with development and DevOps teams to support streaming data pipelines.
Apply patches, upgrades, and manage version control of the Confluent Kafka, Bash, Ansible, Python, KSQL, Kafka Connect, Flink, Schema Registry.
Bachelor's or master’s degree in computer science/information technology, Engineering (Computer/Telecommunication) or equivalent.
- 4+ years of experience in in managing Confluent Kafka Platform.
- Good understanding of Confluent Kafka Components .
- Design, develop and manage Kafka-based data pipeline..
- Good Experience in Confluent Connectors implementation with multiple sources.
- Good understanding of distributed systems.
- Familiarity with cloud environments (AWS, Azure) is a plus.
- Strong problem-solving skills and ability to troubleshoot complex systems.
- Excellent communication skills and ability to work in a collaborative team environment.