Kafka Architect
Summary
Senior Kafka Architect in Hyderabad who designs and operates enterprise Apache Kafka platforms: defining architecture standards, building event-driven streaming solutions with Kafka Streams, Kafka Connect and ksqlDB, and integrating Kafka with microservices, cloud (AWS/Azure/GCP), CI/CD and data systems, plus automation, capacity planning and disaster recovery.
About the Role :
We are looking for an experienced Kafka Architect to design and implement scalable, highly available, and resilient enterprise Kafka platforms. The role involves defining Kafka architecture standards, building event-driven and real-time streaming solutions, and integrating Kafka with microservices, APIs, cloud platforms, and enterprise data systems. The candidate will drive platform automation, capacity planning, disaster recovery, and architectural best practices while collaborating with engineering, DevOps, cloud, and data teams.
Responsibilities :
- Architect, implement, and manage enterprise Kafka clusters with high availability, disaster recovery, and multi-region capabilities.
- Define Kafka architecture standards covering topic design, partitioning strategies, retention policies, replication, and cluster capacity planning.
- Design end-to-end event-driven architectures and streaming solutions using Kafka Connect, Kafka Streams, and ksqlDB.
- Develop reference architectures for integrating Kafka with microservices, APIs, and enterprise data platforms.
- Design and implement automation for cluster provisioning, topic management, policy enforcement, and scaling workflows.
- Integrate Kafka platforms with CI/CD pipelines, infrastructure-as-code tools such as Terraform and Ansible, and Kubernetes environments.
- Ensure platform scalability, reliability, performance, security, and operational resilience.
- Collaborate with engineering, cloud, DevOps, and data teams to deliver reliable enterprise streaming solutions.
- Lead architectural reviews, technology evaluations, roadmap planning, and the adoption of Kafka best practices.
Requirements :
- 10 - 12 years of overall experience in software engineering, distributed systems, data streaming, or platform architecture.
- Strong hands-on experience with Apache Kafka architecture, cluster management, and enterprise implementations.
- Deep understanding of Kafka Streams, Kafka Connect, and ksqlDB.
- Experience designing highly available Kafka clusters, disaster recovery solutions, multi-region architectures, and capacity planning strategies.
- Strong understanding of event-driven architecture, microservices, distributed systems, and real-time data streaming.
- Hands-on experience with Docker, Kubernetes, and containerised deployment environments.
- Experience with AWS, Azure, or GCP cloud infrastructure.
- Working knowledge of Terraform, Ansible, CI/CD pipelines, and infrastructure automation.
- Strong understanding of cluster provisioning, monitoring, performance optimisation, and platform scalability.
- Excellent architectural, analytical, problem-solving, and stakeholder collaboration skills.
- Kafka or cloud platform certifications would be an advantage.
- Retail domain experience is preferred.