Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

InMobi

SDE IV

Posted Updated

Description

Responsibilities:

  • Design and Development: Bridge OSS tools (Spark, Flink, Airflow, Iceberg), internal services, and cloud offerings into a cohesive data platform infrastructure. Build intuitive platform integrations enabling push-button data workflows.
  • Scale Engineering: Operate distributed systems processing petabytes of data daily. Own multi-region Kubernetes infrastructure with elastic scalability and fault tolerance.
  • Performance Optimisation: Optimise compute utilisation (Spark/Flink clusters, Velox/Gluten acceleration) for large-scale batch and real-time streaming with sub-second latency.
  • Observability and Data Quality: Build comprehensive telemetry (metrics, logs, traces) and data quality frameworks for 24/7 uptime. Enforce SLAs/SLOs with automated incident response and data validation.


Requirements:

  • 7-10 years building, optimising, and operating production data platforms.
  • Deep data engineering fundamentals: data modelling, partitioning strategies, query optimisation.
  • Distributed computing: Spark (PySpark/Scala), Flink streaming, performance tuning at petabyte scale.
  • Data lake architecture: Iceberg table format, Polaris catalogue, schema evolution, time travel.
  • Orchestration: Airflow DAG development, dependency management, and SLA monitoring.
  • Data transformation: DBT modelling, testing, documentation, incremental builds.
  • Data quality: Great Expectations, dqueue validation frameworks, and drift detection.
  • Query acceleration: Velox, Gluten integration, columnar formats (Parquet, ORC).
  • Data governance: OpenMetadata catalogue, lineage tracking, and access control.
  • Kubernetes platform development: operators (Spark/Flink), Yunikorn scheduler, multi-tenancy, and autoscaling.
  • Cloud infrastructure: GKE multi-region clusters, GCS object storage, hybrid cloud/on-prem architecture.
  • Programming: Python, PySpark, Scala for data pipelines and platform tooling.
  • IaC: Terraform, Helm, GitOps for reproducible deployments.
  • CI/CD: Automated testing, deployment pipelines for data platform components.


Good-to-Have:

  • Experience in building a cloud data platform/control plane development.
  • Advanced observability: Prometheus/Grafana, Loki, Firehydrant integration.
  • Real-time streaming: Kafka integration, exactly-once semantics, backpressure handling.
  • Cost optimisation: Resource allocation, query optimisation, storage tiering strategies.

Skills

Apply

See also