Data Engineer
Posted Updated 9
views
Description
Talentvis Singapore Pte Ltd. invites a skilled Data Engineer to build and optimize data pipelines, ensuring data quality and system reliability.
You will contribute to automated workflows, maintain scalable ETL processes, and collaborate with cross-functional teams to translate requirements into robust data solutions. The role emphasizes Python and SQL proficiency, experience with pandas or Polars, data warehousing, and CI/CD practices in a cloud-enabled environment.
Design, develop, and maintain ETL processes and data pipelines to collect, transform, and integrate data from multiple sources. Improve existing data processing systems while ensuring data accuracy and consistency. Design and maintain database schemas, data models, and storage structures for large datasets. Develop reusable code, libraries, and tools to improve efficiency and maintainability. Implement automated testing and deployment using CI/CD pipelines. Configure and manage data workflow scheduling, dependencies, monitoring, and error recovery. Manage source code using Git/GitLab and maintain technical documentation. Monitor data pipelines, troubleshoot issues, and improve system performance and reliability. Work with technical teams and stakeholders to translate requirements into practical data solutions. 4-7 years of experience in data engineering, data platform development, or software engineering. Strong proficiency in Python and SQL. Hands-on experience with data processing libraries such as pandas or Polars. Experience in database management, data warehousing, ETL, and data integration. Familiarity with Git/GitLab, version control, code reviews, and CI/CD pipelines. Working knowledge of Linux/Unix environments and shell scripting. Understanding of software engineering practices, including modular coding, testing, logging, error handling, and documentation. Knowledge of data governance, access control, and security best practices. Experience with cloud technologies and workflow orchestration tools would be an advantage. Strong analytical, problem-solving, and communication skills, with the ability to work independently and in a team.Skills
Sign in to read and join the discussion.