Senior Data Engineer: Spark, PySpark & Databricks
Summary
Design and build scalable ETL/ELT pipelines with Spark SQL, PySpark, and Databricks Medallion architecture, focusing on data quality, governance, and cost optimization.
Livewell is seeking a senior data engineer to design and implement scalable ETL/ELT pipelines using Spark SQL and Python within the Databricks Medallion architecture. You will develop dimensional models (star schema) and optimize performance while controlling costs, with a strong focus on data quality and governance.
The role requires expertise in CI/CD pipelines, GitHub workflows, and cloud data services (AWS/Azure/GCP).