Data Engineer Apache Iceberg And Databricks
Posted 9
views
Summary
Data engineer who designs, builds, and maintains scalable batch and streaming data pipelines on a lakehouse stack, centered on Databricks, Apache Iceberg, Spark/PySpark, SQL, and Python on cloud platforms (AWS/Azure/GCP), with attention to data quality, governance, and performance.
Job Description:
Requirements
- 3+ years of experience in Data Engineering or a related role.
- Hands-on experience with Databricks and Apache Iceberg.
- Strong SQL and Python skills.
- Experience with Spark/PySpark and distributed data processing.
- Strong understanding of Data Lake/Lakehouse architecture.
- Experience with cloud platforms such as AWS, Azure, or GCP.
- Knowledge of ETL/ELT, data modeling, and data pipeline development.
- Experience with Delta Lake, Kafka, or other data technologies is an advantage.
Responsibilities
Minimum Qualifications:
- Design, build, and maintain scalable data pipelines and data platforms.
- Develop and optimize data processing workflows using Databricks and Apache Iceberg.
- Build and manage batch and streaming data pipelines.
- Work with data lake/lakehouse architectures and ensure data quality, reliability, and performance.
- Integrate data from multiple sources and transform it for analytics and business use.
- Implement data governance, security, monitoring, and optimization practices.
- Collaborate with Data Architects, Analysts, and other engineering teams to deliver data solutions.