Senior Data Engineer
Summary
Designs and builds scalable data pipelines and cloud-native platforms to support analytics, AI/ML, and smart manufacturing, using Python, Spark, Kafka, and cloud data tools.
We're looking for a Senior Data Engineer to design, build, and optimise scalable data platforms that support analytics, AI/ML, and smart manufacturing initiatives. You'll lead the development of batch, streaming, and event-driven data pipelines while working closely with business stakeholders and engineering teams to deliver reliable, cloud-native data solutions.
Responsibilities
- Design and develop scalable ETL/ELT, streaming, and event-driven data pipelines.
- Build and maintain cloud-based data platforms, data warehouses, and Lakehouse architectures.
- Develop data solutions using Python, SQL, Apache Spark, Airflow, Kafka, and modern cloud technologies.
- Implement DataOps best practices, including CI/CD, monitoring, testing, and pipeline optimisation.
- Collaborate with cross-functional teams to translate business requirements into technical solutions and mentor junior engineers.
Requirements
- 8+ years of experience in Data Engineering, Data Warehousing, or large-scale data platform development.
- Strong hands-on experience with Python, SQL, Apache Spark, Airflow, Kafka, and modern cloud data platforms such as Databricks, Snowflake, BigQuery, Redshift, or Azure Synapse.
- Experience with AWS, Azure, or GCP, Lakehouse architectures, and DataOps/CI/CD practices.
- Strong understanding of data modelling, database design, and performance optimisation.
- Experience with Industrial IoT (IIoT), MES, PLC, SCADA, or manufacturing data systems is highly preferred.