Senior Data Engineer
Summary
Senior Data Engineer builds and maintains an Azure-based cloud-native data platform using Python, PySpark, Databricks, and CI/CD pipelines to integrate legacy and modern systems for a bank.
About the Team
The Data Services Platform department is building a modern data ecosystem based on Data Mesh principles. The goal is to make data easily discoverable, shareable and reusable across the wider bank, enabling better customer experiences, automation and data-driven decision making.
The team works in an informal Agile environment where craftsmanship, collaboration, experimentation and continuous learning are highly valued.
Responsibilities
As a Senior Data Engineer, you will help design and build the next-generation Azure data platform while acting as a technical leader within the team.
Key responsibilities include:
- Design and build Azure-based cloud-native data platforms
- Develop and maintain CI/CD pipelines
- Integrate data from modern and legacy systems
- APIs
- Streaming platforms
- CSV batch integrations
- Support migration from legacy platforms to Azure
- Design reliable migration patterns for phased migrations
- Improve architecture where necessary together with architects
- Share technical knowledge and raise engineering standards
- Participate in business-hours on-call rotation
Required Skills
Technical
Must have:
- Python
- Object-Oriented Programming
- Pytest
- SQL and/or PySpark
- Databricks
- Git
- CI/CD