Data Engineer
Posted 21
views
Summary
Data Engineer in Singapore who analyses data needs, designs and builds big-data ETL jobs and reporting/warehouse solutions, develops tests, supports deployments, and handles production issues. Core stack: Python, PySpark, SQL, Linux, Hive/Impala on Cloudera, Informatica, with Tableau and SAP BO for reporting.
Analyse the data needs and document the requirements
Refine data collection/consumption by migrating data collection to more efficient channels
Plan, design and implement data engineering jobs and reporting solutions to meet the analytical needs
Develop test plan and scripts for system testing, support user acceptance testing
Work with the technical teams to ensure smooth deployment and adoption of new solution
Ensure the smooth operations and service level of IT solutions
Support production issues
Requirements
Good understanding and completion of projects using waterfall/Agile methodology Hands-on experience in DevOps deployment and data virtualisation tools like Denodo will be preferred Understanding of reporting or visualization tool like SAP BO and Tableau is important Track record in implementing systems using Hive, Impala and Cloudera Data Platform will be preferred Hands-on experience in big data engineering jobs using Python, Pyspark, Linux, and ETL tools like Informatica Strong SQL and data modelling and data analysis skills Good understanding of analytics and data warehouse implementations Ability to troubleshoot complex issues ranging from system resource to application stack traces Track record in implementing systems with high availability, high performance, high security hosted at various data
centres or hybrid cloud environments will be an added advantage.
Requirements
Good understanding and completion of projects using waterfall/Agile methodology Hands-on experience in DevOps deployment and data virtualisation tools like Denodo will be preferred Understanding of reporting or visualization tool like SAP BO and Tableau is important Track record in implementing systems using Hive, Impala and Cloudera Data Platform will be preferred Hands-on experience in big data engineering jobs using Python, Pyspark, Linux, and ETL tools like Informatica Strong SQL and data modelling and data analysis skills Good understanding of analytics and data warehouse implementations Ability to troubleshoot complex issues ranging from system resource to application stack traces Track record in implementing systems with high availability, high performance, high security hosted at various data
centres or hybrid cloud environments will be an added advantage.