Data Engineer (Apache Iceberg & Databricks)
Summary
A data engineer role focused on designing, building, and maintaining scalable batch and streaming data pipelines on lakehouse architectures. Core technologies are Databricks, Apache Iceberg, Spark/PySpark, SQL, and Python on cloud platforms (AWS, Azure, or GCP).
Responsibilities
Design, build, and maintain scalable data pipelines and data platforms.
Develop and optimize data processing workflows using Databricks and Apache Iceberg.
Build and manage batch and streaming data pipelines.
Work with data lake/lakehouse architectures and ensure data quality, reliability, and performance.
Integrate data from multiple sources and transform it for analytics and business use.
Implement data governance, security, monitoring, and optimization practices.
Collaborate with Data Architects, Analysts, and other engineering teams to deliver data solutions.
Requirements
3+ years of experience in Data Engineering or a related role.
Hands-on experience with Databricks and Apache Iceberg.
Strong SQL and Python skills.
Experience with Spark/PySpark and distributed data processing.
Strong understanding of Data Lake/Lakehouse architecture.
Experience with cloud platforms such as AWS, Azure, or GCP.
Knowledge of ETL/ELT, data modeling, and data pipeline development.
Experience with Delta Lake, Kafka, or other data technologies is an advantage.