Data Engineer (Python Backend, IT)47k
Posted
Summary
Data engineer in Hong Kong building scalable data pipelines and Apache Spark workloads on AWS (EMR, S3, Glue, Redshift) while supporting MLOps and AI/GenAI data foundations. Requires 3-5 years of data engineering experience, strong Python and SQL, and fluency in Cantonese and English.
$42300 - $47k p.m. + Medical+10%Gratuity
Description
- Design, develop, and maintain scalable data pipelines and data products within a cloud-based data platform.
- Build, enhance, and troubleshoot Apache Spark workloads on AWS EMR for both batch and real-time data processing.
- Enable self-service analytics and data science capabilities through well-structured, governed, and high-quality datasets.
- Contribute to MLOps processes, including support for model deployment, monitoring, and data preparation activities for production ML solutions.
- Assist with AI and Generative AI initiatives by delivering the necessary data foundations and platform integrations.
- Maintain data quality, governance, lineage, security, and consistency across the end-to-end data lifecycle.
- Collaborate with business users, data specialists, and platform engineering teams to gather requirements and implement data-driven solutions.
- Adhere to established architecture guidelines, engineering best practices, and development standards.
Profile
- 3-5 years of hands-on experience in data engineering.
- Strong practical experience with Apache Spark running on AWS EMR.
- Experience working with AWS data services, including S3, Glue, Athena, Redshift, Step Functions, and Lambda; exposure to Lake Formation and SageMaker is advantageous.
- Proficiency in Python and advanced SQL, with experience in ETL/ELT development and data modelling.
- Familiarity with big data and streaming technologies such as Hive, Presto, Kafka, and Spark Streaming.
- Knowledge of MLOps principles and experience supporting machine learning models in production environments is beneficial.
- Exposure to AI and Generative AI-related projects is an advantage.
- Strong collaboration skills and effective communication abilities.
- Fluent in Cantonese and English; Mandarin proficiency is a plus.
- Prior experience gained within an IT consulting environment or technology services organisation is preferred.
Job Offer
- Competitive monthly salary.
- Access to the usual benefits provided by the employer.