Senior Data Engineer

We are seeking an experienced Senior Data Engineer to support a data engineering programme within the energy and renewables sector. The successful candidate will design, develop and optimise scalable data pipelines and cloud-based data platforms, supporting the processing of large-scale IoT, operational and real-time energy data.

Key Responsibilities

• Design, build and maintain scalable data pipelines using Python, SQL, Azure Databricks and Apache Spark.
• Develop real-time and batch data processing solutions for energy and industrial IoT data.
• Implement and optimise Delta Lake architecture for efficient data storage and processing.
• Design and maintain cloud-based data platforms within Microsoft Azure.
• Integrate data from IoT sensors, operational systems and external data sources.
• Ensure data quality, reliability, performance and security across data pipelines.
• Collaborate with data architects, analysts and engineering teams to deliver robust data solutions.
• Support performance optimisation, troubleshooting and continuous improvement of data infrastructure.

Required Experience

• Strong hands-on experience with Python and SQL.
• Advanced expertise in Azure Databricks and Apache Spark.
• Proven experience developing real-time and batch data pipelines.
• Strong knowledge of Delta Lake and modern data platform architecture.
• Experience working with IoT data, streaming technologies and large-scale datasets.
• Understanding of Azure data services and cloud-based data engineering.
• Experience with data modelling, ETL/ELT processes and pipeline optimisation.
• Strong problem-solving and stakeholder communication skills.

Preferred Qualifications

• Previous experience within energy, utilities, renewable energy or industrial IoT environments.
• Experience with Azure Event Hubs, Azure Data Factory or Structured Streaming.
• Familiarity with time-series data, sensor telemetry and operational analytics.
• Knowledge of CI/CD, infrastructure automation and data platform governance.