Senior Data Engineer – Databricks | Azure | Data Engineering
Posted Updated
Summary
Senior Data Engineer building cloud data platforms for enterprise customers: translating business requirements into scalable designs, and building batch/streaming ETL/ELT ingestion pipelines on Databricks and Azure, with CI/CD, performance tuning, and mentoring. Core stack: Databricks, Apache Spark, Scala, Azure Data Factory, SQL Server.
This is remote for US applicants only.
Role Overview
As a Senior Databricks Data Engineer, you will work directly with enterprise customers to design, develop, and implement modern cloud data solutions. You'll translate business requirements into scalable technical solutions, build high-performance data ingestion pipelines, and lead end-to-end delivery across multiple client engagements.
You'll collaborate with architects, business stakeholders, AI engineers, analytics teams, and cloud platform specialists to deliver production-ready data platforms that power reporting, analytics, machine learning, and AI applications.
Key Responsibilities
- Translate business requirements into scalable technical specifications and implementation plans.
- Design, build, and optimize enterprise data ingestion pipelines.
- Develop scalable ETL/ELT frameworks using Azure and Databricks.
- Build reliable batch and streaming data pipelines.
- Lead end-to-end implementation of enterprise data engineering projects.
- Develop reusable frameworks, accelerators, and automation components.
- Optimize data quality, performance, scalability, and cost.
- Collaborate with solution architects, product owners, and business stakeholders.
- Implement CI/CD and DevOps best practices for data engineering.
- Support production deployments and troubleshoot complex data issues.
- Mentor junior engineers and participate in architecture and design reviews.
Required Qualifications
- Bachelor's or Master's degree in Computer Science, Information Systems, Engineering, or a related field.
- 5+ years of hands-on experience in Data Engineering.
- 5+ years building enterprise data ingestion and integration solutions.
- Strong experience with Databricks and Apache Spark.
- Strong Scala programming experience.
- Hands-on experience with Azure Data Factory (ADF).
- Strong SQL development skills.
- 5+ years working with Microsoft SQL Server.
- Experience designing enterprise data models and data warehouses.
- Experience developing REST API integrations and working with structured and semi-structured data.
- Strong troubleshooting and performance tuning skills.
Preferred Qualifications
- Delta Lake
- Unity Catalog
- Azure Data Lake Storage
- Azure DevOps
- GitHub Actions
- Databricks Certifications
What Success Looks Like
- Deliver enterprise-grade cloud data platforms on time and within budget.
- Build scalable, high-quality data pipelines with strong reliability on Databricks
- Enable trusted, AI-ready data for enterprise customers.
- Contribute reusable frameworks that accelerate future implementations.
- Act as a trusted technical advisor to customers and internal teams.