Data Engineer - PySpark & Databrick
Posted 28
views
Description
Key Primary Skills
- Bachelors degree in computer science, Engineering, Information Systems, or related field.
- 3+ years of experience in data engineering
- Strong SQL, PySpark/Scala Spark development experience.
- Design ETL/ELT pipelines using PySpark, SQL, Spark Structred Streaming, and Databricks workflows - Lakeflow.
- Capable of developing streaming data pipelines using PySpark APIs / Scala Spark APIs (version 3+) , Lakeflow concepts, Unity Catalog features
- Capable of coding with or without AI assistants
- Experience designing and supporting data pipelines; working in cross-functional or matrix organizations.
- Ability to communicate technical concepts effectively.
- Willingness to travel up to 10% as needed.
Secondary Skills
- SQL
- Python / Scala
- ETL Development
- Databricks
- Azure Cloud Technologies
- Data Modeling
- Data Warehousing
- Big Data Platforms
- REST APIs