Data Engineer
Summary
Data Engineer at the Hong Kong Jockey Club designing, building, and operating large-scale ETL/ELT pipelines with PySpark and SQL on AWS, managing a cloud data lake with medallion architecture, real-time streaming, and DevOps/DataOps practices to support the Club's digital transformation.
About the Role
The role contributes to shaping the future of sports and entertainment at the Club through a strategic mission to transform digital capabilities. It sits within the AOP programme, which aims to accelerate platform, product, people, performance, and potential through innovation and delivery excellence. The work spans product, technology, operations, delivery, and change, creating next-generation experiences and strengthening the capabilities that support the Club's future growth. The team is diverse and passionate about solving complex challenges, delivering meaningful impact, and creating value for customers, members, the Club, and the wider community.
Key Responsibilities
- Design, develop, test, and operationalise reliable and scalable ETL/ELT data pipelines using PySpark and SQL on the AWS platform.
- Implement and manage a cloud data lake and orchestrate data flows following the medallion architecture.
- Optimise data pipelines and storage solutions for performance, cost, and reliability while handling huge volumes of data.
- Communicate with business users and product owners as required.
Required Experience & Qualifications
- Proven expertise in PySpark for large-scale data processing at terabyte to petabyte scale.
- Strong experience building and managing production‑level ETL workflows and frameworks.
- Hands‑on experience with AWS or other cloud data analytics services.
- Hands‑on experience with data visualisation tools such as Power BI or Tableau.
- 5–8 years of hands‑on experience in data engineering, focusing on designing, building, and maintaining scalable cloud ETL/ELT pipelines.
- 5–8 years of production experience using PySpark and SQL for large-scale data processing.
- 3+ years of experience with end‑to‑end data pipeline lifecycle management, including monitoring, observability, and production support.
- 3+ years of experience architecting, implementing, and managing cloud data platforms on AWS.
- Experience with data governance and data security best practices and compliance requirements.
- Experience with real‑time data streaming technologies such as Kafka or Solace.
- Knowledge of Medallion architecture principles, design patterns, and data modelling.
- Experience with infrastructure‑as‑code using Terraform or CloudFormation.
- Experience with DevOps/DataOps practices including CI/CD, automated testing, and pipeline monitoring.
- Background working with on‑premises data platforms and migration to the cloud.
- AWS certifications such as Data Analytics Speciality, Solutions Architect Associate/Professional.
Salary
Competitive salary
Benefits & Perks
- Competitive benefits package
- Dynamic working environment
- Development opportunities
About the Company
The Hong Kong Jockey Club is a world‑class racing club founded in 1884 that acts continuously for the betterment of society. It has a unique integrated business model comprising racing and racecourse entertainment, a membership club, responsible sports wagering and lottery, and charities and community contribution. Through this model, the Club generates economic and social value for the community and supports the HKSAR Government in combatting illegal gambling.