Senior Data Engineer - Databricks
We're looking for a Senior Data Engineer to take a hands-on technical leadership role in the design and development of a new enterprise data capability.
This is an opportunity to help shape a future-state data architecture while still remaining close to the technology. You'll design and build secure, scalable and modular solutions using Azure Databricks across a complex environment spanning multiple businesses, ERP platforms and operational systems.
Key responsibilities:
This is an opportunity to help shape a future-state data architecture while still remaining close to the technology. You'll design and build secure, scalable and modular solutions using Azure Databricks across a complex environment spanning multiple businesses, ERP platforms and operational systems.
Key responsibilities:
- Helping shape the future data architecture and providing hands-on technical leadership
- Designing modular and adaptable data solutions
- Designing, building and optimising ETL and ELT pipelines in Azure Databricks
- Integrating ERP platforms, SQL databases and operational systems
- Defining source-to-target mappings, transformation logic and data structures
- Developing scalable data models and curated datasets for reporting, analytics and future AI use cases
- Building efficient backend datasets for Databricks and Power BI
- Making appropriate technical decisions across Delta tables, Parquet files and SQL transactional formats
- Implementing data governance, security and access controls
- Leading automation of manual financial and reporting processes
- Applying practical CI/CD, version control, testing, deployment and release management practices
- Working through technical ambiguity and establishing solutions where existing patterns do not yet exist
- Collaborating closely with business stakeholders, analysts, technology teams and external partners.
- At least 2-4 years of hands-on Azure Databricks experience in a production environment
- Strong experience independently designing, building and optimising Databricks pipelines and complex data models
- Strong ETL/ELT, data integration, warehousing and schema-design experience
- Experience integrating ERP systems, SQL databases and other operational platforms
- Strong SQL capability
- Experience building semantic-ready datasets for Power BI
- Practical experience applying CI/CD to data engineering or analytics environments
- Knowledge of Delta tables, Parquet and SQL transactional formats
- The ability to work autonomously through ambiguity and solve complex technical problems
- Strong stakeholder management and delivery discipline
- Exposure to Pronto Xi, SAP Business One or Infor SyteLine would be advantageous
- Experience with Master Data Management, GitHub integration, reporting automation or complex multi-entity environments would also be highly regarded