.Data Engineer [Apache Iceberg & Databricks]
Posted 10
views
Summary
Builds and maintains scalable data pipelines and lakehouse platforms on Databricks and Apache Iceberg, developing batch and streaming workflows with Spark/PySpark, SQL, and Python on cloud platforms like AWS, Azure, or GCP, while collaborating with architects and analysts on data quality and governance.
Requirements
- 3+ years of experience in Data Engineering or a related role.
- Hands-on experience with Databricks and Apache Iceberg.
- Strong SQL and Python skills.
- Experience with Spark/PySpark and distributed data processing.
- Strong understanding of Data Lake/Lakehouse architecture.
- Experience with cloud platforms such as AWS, Azure, or GCP.
- Knowledge of ETL/ELT, data modeling, and data pipeline development.
- Experience with Delta Lake, Kafka, or other data technologies is an advantage.
Requirements
- 3+ years of experience in Data Engineering or a related role.
- Hands-on experience with Databricks and Apache Iceberg.
- Strong SQL and Python skills.
- Experience with Spark/PySpark and distributed data processing.
- Strong understanding of Data Lake/Lakehouse architecture.
- Experience with cloud platforms such as AWS, Azure, or GCP.
- Knowledge of ETL/ELT, data modeling, and data pipeline development.
- Experience with Delta Lake, Kafka, or other data technologies is an advantage.
Responsibilities
Minimum Qualifications
- Design, build, and maintain scalable data pipelines and data platforms.
- Develop and optimize data processing workflows using Databricks and Apache Iceberg.
- Build and manage batch and streaming data pipelines.
- Work with data lake/lakehouse architectures and ensure data quality, reliability, and performance.
- Integrate data from multiple sources and transform it for analytics and business use.
- Implement data governance, security, monitoring, and optimization practices.
- Collaborate with Data Architects, Analysts, and other engineering teams to deliver data solutions.