Senior Data Engineer

Job Description

We are looking for Senior Data Engineers for the team of our Fortune 500 client, working for their banking client.

This is a remote-first position with a required location within EEA (European Economic Area).

Responsibilities

Data Modeling, Migration & ETL Development
Implement ETL processes and real-time data pipelines for efficient data handling and integration.
Develop and maintain scalable, cloud-based data architecture using Azure especially Azure Event Hub and Kafka.
Implement CI/CD practices to streamline development and deployment processes.
Employ Docker for containerization and efficient resource management across development environments.

Collaboration & Agile Development

Gather requirements, set targets, define interface specifications, and conduct design sessions.
Work closely with data consumers to ensure proper integration.
Adapt and learn in a fast-paced project environment.

Work Conditions

Job Requirements

Strong SQL skills for ETL, data modeling, and performance tuning.
Experience in Databricks and Kafka
Proficiency in Python, especially for handling and flattening complex JSON structures.
Extensive experience with Azure cloud services and services like: Azure Event Hub, as well as real-time data processing technologies including Apache Druid and Apache Storm.
Understanding of software engineering and testing practices within an Agile environment.
Experience with Data as Code; version control, small and regular commits, unit tests, CI/CD, packaging, familiarity with containerization tools such as Docker (must have) and Kubernetes (plus).
Excellent teamwork and communication skills.
Proficiency in English, with strong written and verbal communication skills.
Efficient, high-performance data pipelines for real-time and batch data processing.

See also