Data Engineer

We are looking for a Data Engineer to develop, maintain, and improve data pipelines and data processing systems. The role focuses on building reliable data solutions, optimising existing processes, and ensuring data quality, system performance, and operational reliability.

You will contribute to system improvements, automate data workflows, and apply good software engineering practices to support reliable data operations.

Key Responsibilities

• Design, develop, and maintain ETL processes and data pipelines to collect, transform, and integrate data from multiple sources.
• Improve existing data processing systems while ensuring data accuracy and consistency.
• Design and maintain database schemas, data models, and storage structures for large datasets.
• Develop reusable code, libraries, and tools to improve efficiency and maintainability.
• Implement automated testing and deployment using CI/CD pipelines.
• Configure and manage data workflow scheduling, dependencies, monitoring, and error recovery.
• Manage source code using Git/GitLab and maintain technical documentation.
• Monitor data pipelines, troubleshoot issues, and improve system performance and reliability.
• Work with technical teams and stakeholders to translate requirements into practical data solutions.

Requirements

• 4–7 years of experience in data engineering, data platform development, or software engineering.
• Strong proficiency in Python and SQL.
• Hands-on experience with data processing libraries such as pandas or Polars.
• Experience in database management, data warehousing, ETL, and data integration.
• Familiarity with Git/GitLab, version control, code reviews, and CI/CD pipelines.
• Working knowledge of Linux/Unix environments and shell scripting.
• Understanding of software engineering practices, including modular coding, testing, logging, error handling, and documentation.
• Knowledge of data governance, access control, and security best practices.
• Experience with cloud technologies and workflow orchestration tools would be an advantage.
• Strong analytical, problem-solving, and communication skills, with the ability to work independently and in a team.

Interested candidates are encouraged to submit their resumes outlining their relevant experience and achievements to sherriyang(@)talentvis.com or click apply!

**We regret to inform that only shortlisted candidates would be notified**

EA License No: 04C3537

EA Personnel No: R22106683

EA Personnel Name: Yang Hui Shan, Sherri