Senior Data Engineer : Azure
Summary
Senior Data Engineer joining a world-leading energy trading company's Data Flow project for its Front Office in London: building data pipelines, a governed data lake and trading analytics, primarily on Azure (Data Factory, Databricks, Synapse, Stream Analytics) with AWS tooling and CI/CD via GitHub Actions and Azure DevOps.
Our latest opportunity is for a Senior Data Engineer with Azure experience to join a world-leading Energy Trading company as part of a Data Flow project for their Front Office.
Key Characteristics
- Technology expert who constantly pursues knowledge enhancement and has inherent curiosity to understand work from multiple dimensions.
- Deep data focus with expertise in technology domain
- A skilled communicator capable of speaking to both technical developers and business managers. Respected and trusted by leaders and staff.
- Actively delivers the roll‑out and embedding of Data Foundation initiatives in support of the key business programmers.
- Coordinate the change management process, incident management and problem management process.
- Present reports and findings to key stakeholders and be the subject matter expert on DataAnalysis & Design.
- Drive implementation efficiency and effectiveness across the pilots and future projects to minimize cost, increase speed of implementation and maximize value delivery.
- Contributes to community building initiatives like CoE, CoP.
Mandatory skills
- AWS/Azure/SAP Master
- ELT Master
- Data Modeling Master
- Data Integration & Ingestion Skill
- Data Manipulation and Processing Skill
- GITHUB, Action, Azure DevOps Skill
- Data factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest Skill
Optional skills
- Experience in project management, running a scrum team.
- Experience working with BPC, Planning.
- Exposure to working with external technical ecosystem.
- MKDocs documentation
Project Initiatives
To leverage trading information flow advantage across RDS to improve decision making and trading bias based on data insights.The project will increase value creation through collaboration, creating new cross commodity trading opportunities (in both Crude and Products). It will provide better investment in analysis capabilities, tools and infrastructure and more rapidly deploy near term value creation for trading desks.
Deliverables
- Aim to tackle the low hanging fruit, getting things out of Excel, unlocking the power of new visualisation tools, restructuring and cleaning up existing models.
- Support better analysis of Refinery maintenance, Crude supply & demand and Gasoline supply & demand amongst others.
- Creation of a Data Lake and not a Data Swamp and will improve decision making & trading bias via insights from data
- An Asset Capacity & Maintenance deployment will give T&S the summary of planned and unplanned Refinery Maintenance plus Capacity information