Senior Software Engineer
Summary
Kafka platform engineering role: manage Apache Kafka cluster lifecycle on Kubernetes using Confluent for Kubernetes — provisioning, tuning, upgrades, monitoring/alerting, disaster recovery and multi-region replication, security (TLS/mTLS, SASL, ACLs) and troubleshooting. Requires Java or Scala, Python scripting, and observability with Prometheus, Grafana, OpenTelemetry.
Job Description.
Key Responsibilities
• Manage Kafka cluster lifecycle: provisioning, configuration tuning, upgrades, and decommissioning
• Deploy, operate, and maintain Kafka on Kubernetes using Confluent for Kubernetes (CFK)
• Monitor cluster health and set up alerting across consumer lag, broker health, and topic throughput
• Design and operate disaster recovery and multi-region replication strategies
• Enforce Kafka security policies including access reviews and ACL management
• Troubleshoot performance bottlenecks, consumer lag issues, and broker failures
Must Have
• 8+ years in messaging/streaming platform engineering, with 5+ years on Apache Kafka platform engineering
• Hands-on administration of self-managed on-premises Kafka clusters in KRaft mode
• Confluent for Kubernetes (CFK): operator management and custom resource configuration
• Kafka Connect: connector lifecycle, SMTs, and error handling
• Confluent Schema Registry; schema evolution and compatibility modes
• Kafka security: TLS/mTLS, SASL, and ACL configuration and troubleshooting
• Proficiency in Java or Scala; Python for scripting and tooling
• Observability: Prometheus, Grafana, and OpenTelemetry
Good to Have
• Apache Flink for complex event processing beyond Kafka Streams
• CDC pipelines into Kafka
• Operational experience with Mirror Maker 2 or Confluent Replicator in multi-DC setups
• Knowledge of event-driven architecture or data mesh principles
• Build and maintain infrastructure-as-code for Kafka resources
• Integrate Kafka platform changes with CI/CD pipelines for topic and schema deployments
Skills
Sign in to read and join the discussion.