data engineer real-time data

Описание: The independent technology consultancy works across data, cloud and technology transformation, helping clients modernise their platforms and make better use of their data.

Задачи:

• Design, build and maintain scalable ELT/ETL data pipelines within Snowflake.
• Implement data ingestion from multiple sources, including real-time streams using Kafka.
• Develop and deploy KSQL and KStreams pipelines for real-time data processing.
• Develop efficient and scalable data models within Snowflake for analytics and business intelligence.
• Ensure data quality, integrity and reliability throughout the data lifecycle.
• Optimise data pipelines and Snowflake workloads for performance and scalability.
• Collaborate with architects, analysts and data scientists to understand requirements and deliver effective data solutions.
• Contribute to data engineering standards, best practices and continuous improvement across the platform.

Требования:

• Solid commercial experience as a Data Engineer delivering cloud-based data solutions.
• Previous experience within insurance or financial services.
• Strong knowledge of Snowflake, including data modelling and performance optimisation.
• Hands-on experience with Kafka for data ingestion and stream processing.
• Practical experience developing and deploying KStreams and KSQL pipelines.
• Strong SQL skills and experience with Python or another scripting language.
• Experience with data orchestration tools such as Airflow.
• Experience with at least one major cloud platform, such as AWS, Azure or GCP.
• Strong understanding of data ingestion, transformation, modelling and integration.
• Ability to collaborate with technical and non-technical stakeholders.
• Nice to have: Experience building data pipelines for Policy Administration Systems, particularly Guidewire.
• understanding of data schemas and Confluent Schema Registry.
• experience with CI/CD practices and DevOps teams.
• broader experience across AWS, Azure or GCP.

Условия:

• Salary up to €60,000
• Health insurance and mental health support
• Financial support for ongoing professional training and development
• Flexible working
• Opportunities to work on modern data and cloud transformation programmes
• Career development opportunities within a growing technology consultancy