Data Engineer

Must-have technical skills AWS, SQL, PySpark, Apache Iceberg, dbt and Apache Airflow.

Key Responsibilities

• Build and maintain data pipelines using cloud technologies and distributed systems.
• Work with data transformation frameworks and pipeline orchestration tools.
• Manage large volumes of data and optimise data processing.
• Design and implement effective tests, including unit, component, integration, end-to-end and performance tests.
• Coach team members on coding practices, design principles and implementation patterns that produce high-quality, maintainable solutions.
• Manage stakeholders and prioritise work effectively across multiple workstreams.

Requirements

• Formal training or certification in data engineering concepts, together with applied experience.
• Recent hands-on professional experience as a data engineer.
• Experience coding in a recent version of Python.
• Hands-on experience with AWS, SQL, PySpark, Apache Iceberg, dbt and Apache Airflow.
• Experience with cloud technologies and distributed systems.
• Experience with data transformation frameworks and data pipeline orchestration tools.
• Experience managing large volumes of data and optimising data processing.
• Understanding of event-based architecture, data streaming and messaging frameworks.
• Experience designing and implementing effective software tests.