Principal Data Engineer Databricks
Summary
Design and build scalable ETL pipelines on Databricks, using PySpark, Delta Lake, and Medallion Architecture to deliver clean, production-grade data workflows.
Required CertificationsDatabricks Certified Data Engineer Professional
Role requirements7+ years of experience building highly reliable ETL pipelines and managing complex data workflows. Elite-level proficiency with Apache Spark, Delta Lake architecture, and deploying optimized data infrastructure in production environments.
Technical requirements- Core Architecture: Delta Lake, Medallion Architecture (Bronze/Silver/Gold), Unity Catalog
- Engines: PySpark, Spark SQL, Photon Engine, Delta Live Tables (DLT)
- Orchestration: Databricks Workflows, dbt (Data Build Tool), Airflow
- Performance: Z-Ordering, Liquid Clustering, Data Skipping, and Spark UI Debugging
DatabricksSparkDelta Lake