Remote Data Engineer: Databricks Lakehouse & Pipelines
Summary
A middle-level data engineer modernizes a 15-year-old data warehouse into a governed Databricks Lakehouse, building batch and streaming pipelines with PySpark and Delta Lake for analytics, reporting, and ML consumers. The role also covers data quality, governance, and observability work with DevOps and analytics engineers, using Claude and GitHub Copilot.
AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, supporting analytics, reporting, and ML consumers.
You will work with Claude and GitHub Copilot to accelerate development, implement data quality and governance, and collaborate with DevOps and analytics engineers on observability, security, and compliance.