Senior Data Engineer, Databricks
We are looking for a senior data engineer with deep Databricks experience to own the data platform behind one of our largest enterprise accounts and lead its migration from AWS to Databricks.
About the role
Data infrastructure is the foundation of everything we deliver. Our models, dashboards, and the decisions our clients make on top of them are only as reliable as the pipelines underneath, which is why we put the platform first.
You will join a small senior team working inside the client's environment. Through the rest of 2026 you will take ownership of the existing pipelines and the architecture around them. From 2027 you will lead the migration to Databricks: designing the target architecture, planning the cutover, and doing the implementation. This is a hands-on engineering role with real architectural authority. You decide how the platform gets built, and then you build it.
You will be the person the team turns to on anything Databricks, working alongside senior machine learning engineers and data scientists who depend on what you deliver.
What you will do:
Design and build the target Databricks architecture, and plan the migration from the current AWS stack
Build and maintain production data pipelines: ingestion, transformation, and orchestration
Own data quality, including the checks and alerts that catch problems before the client does
Design the data models the analytics and machine learning work sits on
Support the pipelines you build: monitoring, alerting, and the on-call reality of production data
Work directly with the client's engineers, and explain technical decisions to people who are not engineers
Requirements:
Five or more years in data engineering or software development
At least three years of production Databricks work
Deep experience with Spark, Delta Lake, and Unity Catalog
Has planned and executed at least one migration onto Databricks from another platform, and can walk through the challenges
Expert Python: clean modular code, comfortable with optimization and profiling
Advanced SQL and relational database work: raw SQL, data modeling, scripted migrations
Systems design: how storage and compute fit together, end to end pipelines, containerized applications
Cloud operations on AWS or equivalent: networking, identity and access management, monitoring and alerting, and the domain knowledge for why things break
Comfortable working inside a client's environment, with their data and their constraints
Strong written English, and spoken English good enough to present your work
Available for some overlap with United States working hours
Nice to have:
Databricks certification
Deep AWS experience specifically, since that is the platform we are migrating from
dbt, Airflow, or a similar transformation and orchestration framework
Machine learning pipeline experience, since models run on top of what you build
Background in utilities, energy, telecoms, or another regulated industry
Experience being the senior engineer a client relies on directly
You are:
Careful with data quality, and you ask when something looks wrong instead of guessing
A self starter who can turn a general requirement into a specific estimate and a deliverable
Someone who documents. We write things down, and every number we deliver carries its source
Comfortable in a small remote team where nobody is checking your hours
About Bennett Data Science
We have spent ten years building predictive analytics and data science for enterprise clients, from some of the largest companies in the world to startups in Silicon Valley. Our engineers sit inside the client's systems and build critical systems. The work ships to production and lands at the bottom line.
What we offer:
Competitive compensation, reviewed as you grow
Work from anywhere, with a flexible schedule
A small senior team, so your work is visible and your decisions are important
Direct exposure to enterprise clients and the problems they actually have
Mentorship from senior staff, and the space to grow and mentor others