Kafka Developer
Posted Updated
Description
Key Responsibilities
- Design, build, and optimize Kafka-based messaging and streaming pipelines for large-scale, high-throughput applications.
- Troubleshoot Kafka messaging issues, analyzing broker, producer, and consumer logs to identify and resolve performance bottlenecks.
- Manage and fine-tune Kafka clusters, including brokers, topics, partitions, replication factors, and consumer groups.
- Implement and maintain Kafka security features (SSL, SASL, ACLs, Kerberos) and ensure compliance with organizational standards.
- Integrate Kafka with downstream systems using Kafka Connect, Schema Registry, and REST APIs.
- Monitor Kafka clusters and pipelines using tools such as Confluent Control Center, Prometheus, Grafana, or Datadog to track consumer lag and key performance metrics.
- Collaborate with architects and developers to design scalable, fault-tolerant, and resilient Kafka solutions.
- Contribute to system design and architecture discussions for enterprise data streaming solutions.
Must-Have Skills
- 8–10 years of overall IT experience with 5+ years in Kafka ecosystem.
- Strong expertise in Kafka core components (brokers, producers, consumers, topics, partitions, replication, offsets).
- Hands-on experience with Confluent Kafka, Kafka Streams, and Kafka Connect.
- Proficiency in Java/Scala (Python is a plus).
- Strong knowledge of log analysis, debugging, and performance tuning in distributed systems.
- Experience with monitoring and metrics tools (Grafana, Prometheus, Datadog, or similar).
- Deep understanding of distributed system design principles and event-driven architecture.
Skills
Sign in to read and join the discussion.