Cloud Lakehouse Platform Engineer
Posted Updated
Description
Hyve Solutions is seeking an experienced Data Warehouse SE to evolve our data platform from Hadoop to a cloud-native Lakehouse built on Apache Iceberg. You will focus on platform engineering, running large-scale data infrastructure, not just pipelines, with emphasis on Spark tuning, metadata governance, and multi-engine data access.
Join a team delivering PB-scale workloads across cloud-native storage and Kubernetes environments, with Azure, Databricks, and Snowflake integrations to enable
Design and operate an enterprise-grade Lakehouse on Iceberg for batch and streaming workloads. Orchestrate migration from Hadoop/HDFS to cloud-native storage with compute-storage separation. Develop core platform services: metadata, catalog, governance, and multi-engine access. Optimize Spark pipelines for PB-scale data processing and cost efficiency. Build cloud-native data platforms on Kubernetes and automate operations. Integrate with Azure, Databricks, and Snowflake for seamless data flow. Experience with Spark execution planning and memory management. Hands-on with Iceberg, Hudi, or Delta Lake metadata, snapshots, and schema evolution. Kubernetes deployment of Spark workloads and resource scheduling. Cloud object storage experience (Azure/S3/MinIO) and performance considerations. Experience with Azure Databricks or Snowflake as a data platform. Strong programming skills in Java, Scala, or Python.Skills
Sign in to read and join the discussion.