Data Engineer
Posted Updated
Summary
Data engineer based in Singapore who builds and maintains cloud data pipelines and large-scale data processing, using AWS, SQL, PySpark, Apache Iceberg, dbt and Apache Airflow, plus designing software tests and coaching teammates.
Must-have technical skills AWS, SQL, PySpark, Apache Iceberg, dbt and Apache Airflow.
Key Responsibilities- Build and maintain data pipelines using cloud technologies and distributed systems.
- Work with data transformation frameworks and pipeline orchestration tools.
- Manage large volumes of data and optimise data processing.
- Design and implement effective tests, including unit, component, integration, end-to-end and performance tests.
- Coach team members on coding practices, design principles and implementation patterns that produce high-quality, maintainable solutions.
- Manage stakeholders and prioritise work effectively across multiple workstreams.
- Formal training or certification in data engineering concepts, together with applied experience.
- Recent hands-on professional experience as a data engineer.
- Experience coding in a recent version of Python.
- Hands-on experience with AWS, SQL, PySpark, Apache Iceberg, dbt and Apache Airflow.
- Experience with cloud technologies and distributed systems.
- Experience with data transformation frameworks and data pipeline orchestration tools.
- Experience managing large volumes of data and optimising data processing.
- Understanding of event-based architecture, data streaming and messaging frameworks.
- Experience designing and implementing effective software tests.