Jr. & Sr. Data Engineer
Description
Salary: RM 7,600 – RM 11,000 per month
Junior Data Engineer: 3-5 Years of relevant experience in Data Engineering fields and good knowledge of Data Warehousing/Data Lake modeling and implementation.
Senior Dara Engineer: 7 Years of relevant experience in Data Engineering fields and good knowledge of Data Warehousing/Data Lake modeling, ETL and implementation.
Location: Bangi, Selangor
9 Months contract role only
About the role
Responsible for data mapping, designing, development, testing, implementing for Enterprise Data Lake Data Pipelines (Bronze, Silver, and Gold) data delivery within timeline.
Key responsibilities
Design, build, and maintain ETL/ELT pipelines (end-to-end) from sourcing to final data product delivery for structured and unstructured data
Design schemas for warehouses and analytical model concept for implementation
Clean and preprocess data to make it suitable for analysis, implementing data transformation processes to extract data for the creation and loading of Data Products
Integrate data from multiple sources into centralized platforms for analytics and reporting, including API data integration to EDL pipelines with Internal and External applications
Build frameworks/program for validation, monitoring, and error handling
Provide documentation on Data Mapping, Technical Specification, and SIT plan & test cases
Provide strategic recommendations that enhance productivity and business performance
About you
Minimum 3 to 7+ years of relevant experience in Data Engineering fields and good knowledge of Data Warehousing/Data Lake modeling and implementation
Proficiency in Python, Powershell, YAML, bash
Strong proficiency in Linux systems and networking for integration and ELK stack management
Familiar with data streaming technologies such as Kafka
Experience with batch scheduling tools such as Nifi, Airflows
Experience with Apache Iceburg framework for SCD data modeling
Bachelor's or Master Degree in Engineering, Computer Science, IT, or related technical field
Good problem solving skill
Knowledge of cloud platforms (AWS, Azure, or GCP) and containerization (Docker/Kubernetes)
Knowledge of other Databases such as MongoDB, HBase, Postgress