Senior Data Engineer

We are hiring for one of our client in Kuala Lumpur, Malaysia, seeking a highly skilled and experienced Senior Data Engineer to join our Data & Analytics team within the Insurance domain. The successful candidate will be responsible for designing, developing, and maintaining scalable, secure, and high-performing data platforms that support enterprise reporting, analytics, regulatory requirements, and business decision-making.

This role requires strong expertise in AWS cloud technologies, data lake and data warehouse architectures, ETL/ELT development, Infrastructure as Code (IaC), data governance, disaster recovery, and modern AI-enabled software development practices. The ideal candidate will be passionate about delivering reliable data solutions, automating processes, enforcing data quality standards, and driving innovation through emerging technologies.

Key Responsibilities

• Design, develop, and maintain scalable, reliable, and secure data pipelines to support enterprise analytics and reporting initiatives.
• Build and optimize ETL/ELT processes to ingest, transform, and deliver data from various source systems into Data Lake and Data Warehouse environments.
• Design, implement, and maintain cloud-native data solutions on AWS.
• Develop scalable batch and near real-time data processing solutions.
• Develop and maintain Apache Iceberg datasets to support enterprise analytical workloads.
• Ensure data from source systems is successfully loaded into the enterprise Data Lake daily with accuracy, completeness, and timeliness.
• Monitor and troubleshoot data ingestion processes and resolve data-related issues proactively.
• Conduct root cause analysis and implement sustainable solutions for recurring incidents.
• Optimize data processing performance, scalability, and operational efficiency.
• Support enterprise reporting, analytics, and regulatory data requirements.
• Design, build, and support AWS-based data platforms using:
• AWS Glue
• Amazon Redshift
• Amazon S3
• Amazon RDS
• Amazon SNS
• AWS Step Functions
• Amazon CloudWatch
• AWS IAM
• Amazon EventBridge
• AWS DataZone
• Develop and maintain Infrastructure as Code (IaC) solutions using Terraform.
• Automate environment provisioning, deployment, and operational processes.
• Implement monitoring, alerting, and operational support frameworks to ensure platform stability and high availability.
• Optimize cloud infrastructure for performance, security, reliability, and cost efficiency.
• Support and enhance CI/CD deployment pipelines and DevOps practices.

Disaster Recovery & Business Continuity

• Participate in Disaster Recovery (DR) planning, validation, and execution activities for critical data platforms and services.
• Perform Disaster Recovery (DR) drills, failover, failback, and recovery procedures within established Recovery Time Objective (RTO) and Recovery Point Objective (RPO) requirements.
• Ensure data pipelines, Data Lake, and Data Warehouse platforms can be recovered and restored during disaster scenarios.
• Maintain and regularly update Disaster Recovery documentation, runbooks, and recovery procedures.
• Collaborate with infrastructure, security, and application teams to ensure business continuity readiness.

Data Governance & Quality Management

• Establish and maintain data quality validation, reconciliation, and monitoring controls.
• Ensure data accuracy, consistency, completeness, and reliability across the data platform.
• Support enterprise data governance initiatives through AWS DataZone, including metadata management, data ownership, and data discoverability.
• Collaborate with stakeholders to define and implement data standards, governance policies, and best practices.
• Ensure compliance with enterprise security, privacy, regulatory, and audit requirements.

Stakeholder Collaboration

• Collaborate closely with Business Analysts, Data Analysts, BI Developers, Architects, Product Owners, and business stakeholders to understand and deliver data requirements.
• Translate business requirements into scalable technical designs and data models.
• Provide technical leadership and guidance on data engineering best practices.
• Partner with cross-functional teams to continuously improve data platform capabilities.

Documentation & Knowledge Transfer

• Create and maintain comprehensive technical documentation, including:
• Solution Design Documents
• Technical Specifications
• Data Flow Diagrams
• Runbooks
• Support Guides
• Operational Procedures
• Disaster Recovery Procedures
• Conduct Knowledge Transfer (KT) sessions to ensure team members understand implemented solutions, processes, and support procedures.
• Promote knowledge sharing, engineering standards, and best practices across the team.
• Mentor junior engineers and support team capability development.

Required Qualifications

• Bachelor's Degree in Computer Science, Information Technology, Data Engineering, Engineering, Information Systems, or a related discipline.
• Minimum 3+ years of experience in Data Engineering, Data Warehousing, Data Platform Engineering, or related roles.

Strong hands-on experience in:

• SQL
• Python
• PySpark
• Terraform
• Strong experience with AWS services including:
• AWS Glue
• Amazon Redshift
• Amazon S3
• Amazon RDS
• Amazon SNS
• AWS Step Functions
• Amazon CloudWatch
• AWS IAM
• Amazon EventBridge
• AWS DataZone
• Experience designing and supporting enterprise Data Lake and Data Warehouse solutions.
• Experience working with Apache Iceberg tables and large-scale data processing platforms.
• Strong understanding of ETL/ELT methodologies, data modeling, and data integration techniques.
• Experience implementing Infrastructure as Code (IaC) using Terraform.
• Strong understanding of cloud security, IAM permissions, and access control principles.
• Experience planning, executing, and supporting Disaster Recovery (DR) activities, including failover testing, recovery validation, and business continuity processes.
• Knowledge of Agentic AI, Generative AI, and AI-assisted software development practices.
• Experience utilizing AI-powered coding assistants and modern AI development workflows to improve engineering productivity, accelerate delivery, and enhance code quality.
• Strong analytical, problem-solving, troubleshooting, and communication skills.
• Ability to work independently and collaboratively in a fast-paced environment.

Preferred Qualifications

• AWS Certifications, such as:
• AWS Certified Solutions Architect
• AWS Certified Developer
• Experience with Power BI.
• Knowledge of SAP BusinessObjects (SAP BO).
• Experience implementing AWS DataZone and enterprise data governance frameworks.
• Experience with Data Lakehouse architecture.
• Familiarity with DevOps, CI/CD, and cloud automation practices.
• Experience within the Insurance or Financial Services industry.