Principal Data Engineer
Summary
Lead data engineering efforts to build and migrate a Databricks-based data lake and ML pipelines for an automotive client, using PySpark, Python, and CI/CD.
Swedium Global Services are looking for Principal Data Engineer for one of our Automotive Customer.
Period:2026-01-12 to 2026-12-31
Description:
Principal Data Engineer
The Position: We are looking for a Principal Data Engineer to join our Data & Analytics Operations Team within the Data, Analytics & AI Organization. You will work together with Data Scientists, ML Engineers, Data Analysts, and Data Engineers to lead the transition into a data-driven culture, where Data, Analytics & AI/ML are an integral part of our services and internal processes. By having the responsibility all the way from raw data to actionable insights, we are at the forefront of creating value.
We are in the final stages of migrating our current AWS native data lake solution to Databricks, and are looking for someone that can support in the final stretch of the delivery, as well as supporting in creating our next generation data assets and models suitable for wide consumption and self-service analytics.
The Profile: We are looking for a Senior Consultant to bring in expert knowledge in the field of Machine Learning Engineering and Data Engineering. You should be well vested in building high-performance Data Pipelines and designing analytics solutions for production. Our primary tool stack is Databricks on AWS, coupled with Github Actions as our CI/CD framework. As a senior member of our team you are able to adapt to our way of working quickly, while bringing in state-of-the-art ideas and coaching and supporting other members of the team in all parts of the data engineering process.
Expected skills:
- Databricks Expertise, including but not limited to:
- Data Engineering toolstack (Lakeflow, Lakeflow Declarative Pipelines, Unity Catalog)
- ML Engineer toolstack (MLflow, Model Training, Model Serving, Agent Bricks, AI Gateway & Agent Framework and Evaluation)
- General knowledge (Unity Catalog, basic AWS knowledge)
- Primary Languages
- Python
- PySpark
- Bash
- DAX
- Software Engineering
- Confident in working with GitHub, CI/CD and writing high quality code
- Good knowledge of architectural patterns around ML Engineering and Data Engineering
As you will be working closely in a team with other Data Engineers, ML Engineers & Data Analysts communicative and cooperative skills are highly valued.
Job Overview
- Vacancy : 1