Senior Data Engineer - BigData Development
Summary
Senior Data Engineer building production-grade data pipelines, transformations, and analysis-ready datasets for observability and embedded analytics on a project for a leading automotive client. Core work spans Python, SQL, cloud data solutions, orchestration, and processing high-volume logs, events, and telemetry.
Project description
We are looking for an experienced Senior Data Engineer to join a project within observability and embedded analytics for a leading automotive Client.The project focuses on transforming logs, events, measurements and telemetry from embedded, cloud, test and application systems into trusted and reusable datasets and engineering insights.You will have a hands-on role in building production data pipelines, transformations and analysis-ready datasets that support faster troubleshooting, reliability analysis, product-quality follow-up and improved understanding of platform performance in real-world conditions.
Responsibilities
- Design, build and maintain production-grade data pipelines and transformations.
- Develop data models, schemas and curated datasets for different engineering use cases.
- Process logs, events, telemetry and high-volume time-series data.
- Integrate data from embedded, cloud, test and application sources using APIs and other data interfaces.
- Implement automated data-quality checks and pipeline monitoring.
- Build analysis-ready datasets supporting troubleshooting, reliability and product-quality analysis.
- Improve pipeline reliability, performance and maintainability.
- Support dashboards and analytical use cases.
- Document delivered solutions and support knowledge transfer.
SKILLS
Must have
- Strong experience developing and operating production-grade data pipelines.
- Strong hands-on Python and SQL skills.
- Strong knowledge of data transformation and data modelling.
- Experience with cloud-based data solutions.
- Experience with data orchestration and automated workflows.
- Experience working with APIs, event data and logs.
- Experience with data quality and pipeline monitoring.
- Experience working with high-volume time-series and/or telemetry data.
Nice to have
- Automotive or connected-car experience.
- Experience working with data from embedded systems or devices.
- Experience supporting troubleshooting, reliability or product-quality analytics.
- Experience with data visualisation and dashboards.
- Mobile analytics experience.
- Knowledge of data lineage and data governance.