Senior Data Engineer
Summary
Designs and builds data foundations and end-to-end solutions using Python, PySpark, Databricks, Azure, and SQL. This role involves creating data-driven initiatives and mentoring junior engineers within a trading business.
Senior Data Engineer
Certain Advantage are recruiting for a Senior Data Engineer on a contract basis to start in September for 6-12 months, operating inside IR35.
3 days onsite in London is required per week.
This is a fantastic opportunity to work with a high performing team on a project supporting a major data and analytics initiative within a globally renowned Trading business in London.
The Senior Data Engineer is someone who designs and builds data foundations and end-to-end solutions for the business to maximize value from data. The role helps create a data-driven thinking within the organization, not just within IT teams, but also in the wider Business stakeholder community.
You’ll be expected to be a subject matter expert who designs & builds data solutions and mentor junior engineers.
We’re looking for someone who can offer the following skills & experience.
- Strong experience in Python, PySpark/Spark, Databricks, Azure Data Factory, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest.
- Experience designing, building and supporting scalable data pipelines and data products in cloud environments.
- Experience developing and maintaining REST APIs using frameworks such as FastAPI.
- Strong knowledge of data modelling, data quality, and data governance.
- Experience with Git, GitHub Actions, CI/CD pipelines and automated testing.
- Experience working with containerised applications using Docker and Kubernetes
- Strong understanding of cloud-native application design, observability, monitoring and operational support.
- Experience applying software development best practices, including code reviews, automation, testing and secure coding standards.