Data Engineer – Azure / Databricks / Remote
Summary
Build and maintain Azure-based data pipelines using Databricks and Python to ingest, process, and govern financial data in a layered data lake architecture.
Data Engineer – Azure / Databricks / 3-month contract / Remote
Data Engineer / Data Engineering / Azure / Databricks / Ingestion / Pipelines / API / Azure API Management (APIM) / Python / 3-month contract / Outside IR35 / Fully remote / £400 – 500 per day.
One of our leading clients is looking to recruit a Data Engineer.
Location – Remote / home based / WFH
Duration – Initial 3 month
Day Rate – £400-500 per day Outside IR35
Role / Responsibilities:
- Define and build a data lake architecture on Azure (ADLS + Databricks)
- Layered data model (bronze / silver / gold) for ingestion, processing, and reporting
- Develop ingestion patterns from API sources (batch and streaming)
- Establish data governance, metadata, and access controls
- Define storage structures, naming conventions, and schema design
- Develop API connectors and ingestion pipelines using Python
- Implement integration services feeding into the Databricks environment
- Build data transformation logic using PySpark / Python (data cleansing, standardisation, validation)
- Configure monitoring, logging, and alerting
- Execute unit and integration testing across pipelines
Core Skills:
- Azure
- Databricks
- Python
Desirable:
- Any exposure to Financial Services, Insurance or Bank/Banking would be beneficial but not essential