Remote Data Engineer: Databricks Lakehouse & AI Pipelines
Summary
Middle Data Engineer (via AgileEngine) modernizing a 15-year-old data warehouse into a governed Databricks Lakehouse — building batch and streaming pipelines with PySpark and Delta Lake using bronze/silver/gold medallion architecture for analytics and ML. Remote work supported; AI assistants (Claude, GitHub Copilot) are part of the toolchain.
AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You’ll build batch and streaming pipelines with PySpark/Delta Lake, using a medallion architecture (bronze/silver/gold) for analytics and ML use cases.
You’ll leverage Claude and GitHub Copilot to accelerate development, ensure data quality, governance, and scalable data platforms for enterprise customers worldwide. Remote work is supported.