Data Engineer (Python Backend, IT)47k
Posted
Summary
A data engineer designs and maintains scalable data pipelines and Apache Spark workloads on an AWS-based data platform (EMR, S3, Glue, Athena, Redshift), supporting analytics, MLOps, and AI/GenAI initiatives. The role is in Hong Kong through recruiter Michael Page for an unnamed client, requiring Python, advanced SQL, and Cantonese/English fluency.
- Opportunity to work on a modern AWS-based data and AI platform.
- Exposure to Spark, MLOps, and emerging AI/GenAI initiatives.
About Our Client
Our client is an organisation investing in modern data platform capabilities to support analytics, machine learning, and AI-driven business initiatives. The environment emphasizes cloud technologies, data governance, and engineering best practices.
Job Description
- Design, develop, and maintain scalable data pipelines and data products within a cloud-based data platform.
- Build, enhance, and troubleshoot Apache Spark workloads on AWS EMR for both batch and real-time data processing.
- Enable self-service analytics and data science capabilities through well-structured, governed, and high-quality datasets.
- Contribute to MLOps processes, including support for model deployment, monitoring, and data preparation activities for production ML solutions.
- Assist with AI and Generative AI initiatives by delivering the necessary data foundations and platform integrations.
- Maintain data quality, governance, lineage, security, and consistency across the end-to-end data lifecycle.
- Collaborate with business users, data specialists, and platform engineering teams to gather requirements and implement data-driven solutions.
- Adhere to established architecture guidelines, engineering best practices, and development standards.
The Successful Applicant
- 3-5 years of hands-on experience in data engineering.
- Strong practical experience with Apache Spark running on AWS EMR.
- Experience working with AWS data services, including S3, Glue, Athena, Redshift, Step Functions, and Lambda; exposure to Lake Formation and SageMaker is advantageous.
- Proficiency in Python and advanced SQL, with experience in ETL/ELT development and data modelling.
- Familiarity with big data and streaming technologies such as Hive, Presto, Kafka, and Spark Streaming.
- Knowledge of MLOps principles and experience supporting machine learning models in production environments is beneficial.
- Exposure to AI and Generative AI-related projects is an advantage.
- Strong collaboration skills and effective communication abilities.
- Fluent in Cantonese and English; Mandarin proficiency is a plus.
- Prior experience gained within an IT consulting environment or technology services organisation is preferred.
What's on Offer
- Competitive monthly salary.
- Access to the usual benefits provided by the employer.