Junior & Mid Data Engineer

Optimise solution performance

• Develop tools to improve data flows between internal and/or external systems and the data warehouse
• Test data system configurations to increase efficiency
• Automate the data collection and analysis processes, data releasing and reporting tools
• Assist in the integration of data systems with existing infrastructure

Identify business needs

• Determine technical system requirements based on data needs
• Identify suitable data structures based on business needs to ensure availability and accessibility of data
• Keep abreast of latest technologies and products in database and data processing software, and technologies

Build and maintain data pipeline

• Develop backup data archiving systems to ensure system continuity
• Consolidate and create data storage solutions for storage and retrieval of information
• Support the handling and logging of errors
• Develop codes and scripts to process structured and unstructured data in real-time from a variety of data sources
• Monitor data system performance
• Implement and monitor data security and privacy measures on existing data solutions
• Assist in building scalable data pipelines to extract, transform, load and integrate data
• Test data pipelines for scalability and reliability to process high data volume, variety and velocity
• Develop prototypes and Proof-of-Concepts for data solutions

• With 1-3 years of experience supports designing, building, and maintaining data systems that handle data collection, storage, and processing (both batch and real-time). To create efficient, secure, and scalable solutions to turn data into insights. Develop, test, and implement data tools within the company’s infrastructure, working with large datasets to analyze and visualize information across multiple platforms.
• Must be skilled in databases, scripting, and programming languages used by their organization, and understand the deployment platforms.
• Passionate about data, focus on understanding business needs and solving problems through well-organized and interconnected data systems.