Lead Data Engineer
Description
Responsibilities
Lead Databricks architecture and delivery
Drive Synapse-to-Databricks migration
Establish engineering standards and reusable frameworks
Mentor engineers and conduct design reviews
Own technical delivery across major workstreams
Drive performance optimization and scalability
Required Skills
Databricks (Highest Priority)
Databricks
PySpark
Spark SQL
Delta Lake
Unity Catalog
Databricks Workflows
Databricks Asset Bundles
Lakehouse Architecture
Azure Platform
Azure Data Factory (ADF)
ADLS Gen2
Azure SQL
Azure Security Patterns
Migration Experience
Synapse Pipelines
Synapse Spark
Synapse SQL Pools
Synapse Notebooks
Synapse Data Flows
Modernization of legacy ETL workloads
Engineering
Advanced SQL
Data Modeling
GitLab CI/CD
Performance Tuning
Code Reviews
Preferred
Apache Airflow
Snowflake
Data Governance
Enterprise-scale Azure platform implementation