Data Engineer (Energy Trading)
Summary
6-month contract data engineer embedded in a major energy trading organisation's data transformation: building end-to-end data products with Python, PySpark, Databricks and SQL Server, developing FastAPI REST services, and running CI/CD via GitHub Actions with Docker/Kubernetes on Azure.
Contract: Initial 6-month contract (long-term transformation programme)
X4 Technology are partnered with a major energy trading organisation on a strategic programme to unlock the power of their data across Trading, driving sharper decision-making and new cross-commodity trading opportunities.
Key focuses are pulling analysis out of Excel, unlocking modern visualisation tools and building out a proper Databricks-powered data platform to give trading desks real, near-term value from their data.
Joining as Data Engineer, you'll own data products end-to-end - from design and build through to deployment and live support - combining serious data engineering experience with strong backend/API development.
Responsibilities for the Data Engineer
- Design, build and maintain scalable data pipelines and data products using Python, PySpark, Databricks and SQL Server
- Develop and support modern REST APIs and backend services using FastAPI - a genuine hands-on API-building role, not just pipelines
- Own engineering end-to-end across the full SDLC - design, build, test, deploy, monitor and support - working closely with stakeholders, product owners, architects and engineers to turn requirements into robust solutions
- Build and maintain CI/CD pipelines, automated testing and release automation via GitHub Actions, plus containerised deployments using Docker and Kubernetes
- Champion engineering best practice - code reviews, automated testing, security, observability and operational excellence
Requirements for the Data Engineer
- Strong hands-on Python, PySpark, Databricks and SQL Server experience
- Strong, proven FastAPI experience - this is essential, not a nice-to-have
- Solid Azure cloud experience, plus Docker, Kubernetes and GitOps
- Strong understanding of data architecture, data modelling and data governance principles
- Excellent stakeholder engagement - comfortable with both technical and non-technical audiences
A brilliant opportunity to build modern, cloud-native data products at the heart of a major trading business's data transformation - full ownership from design to production, with real influence over how the platform evolves.