Big Data Engineer
Summary
A big data engineer at a Hong Kong fintech who designs and builds the enterprise data warehouse, ETL pipelines, and BI/reporting systems, and increasingly integrates LLMs, RAG, and vector databases for AI-driven analytics. Core stack: Spark, Flink, Hive, Kafka, ClickHouse, MySQL, Docker/K8s, and Java/Scala/Python.
Finloop Finance Technology Services Limited View all jobs
Design, model, and implement the enterprise data warehouse (EDW); drive data governance and data quality initiatives, unify metric standards, and enable metric automation with monitoring and alerting. Build and develop the BI system, data visualization, and reporting framework to support self-service analytics, decision dashboards, and data-driven operations. Combine LLMs / AI agents to build AI-driven data analysis capabilities, delivering scenarios such as data Q&A, anomaly detection, and intelligent analysis. Own big data ETL development and data integration, connecting the full data pipeline across big data and AI services (LLM / RAG / vector databases). Optimize data warehouse and compute task performance, ensure stability, and manage costs; resolve production-critical technical challenges. Track the convergence of big data and AI technologies, and work with the team to deliver integrated Data + AI + BI solutions that improve development and analytical efficiency.
Key responsibilities
Design, model, and implement the enterprise data warehouse (EDW); drive data governance and data quality initiatives, unify metric standards, and enable metric automation with monitoring and alerting
Build and develop the BI system, data visualization, and reporting framework to support self-service analytics, decision dashboards, and data-driven operations
Combine LLMs / AI agents to build AI-driven data analysis capabilities, delivering scenarios such as data Q&A, anomaly detection, and intelligent analysis
Own big data ETL development and data integration, connecting the full data pipeline across big data and AI services (LLM / RAG / vector databases)
Optimize data warehouse and compute task performance, ensure stability, and manage costs; resolve production-critical technical challenges
Track the convergence of big data and AI technologies, and work with the team to deliver integrated Data + AI + BI solutions that improve development and analytical efficiency
About you
Bachelor's degree or above in Computer Science, Big Data, Statistics, or a related field
3–5 years of big data development experience, with hands-on experience in 0-to-1 data warehouse construction, metrics framework development, and BI reporting
Systematic thinking and strong data modeling capabilities; sensitive to AI + data technologies; self-driven and skilled at cross-team collaboration
Proficient in the big data stack (Spark / Flink / Hive / Kafka / ClickHouse), with solid experience in offline and real-time data processing
Strong command of data warehouse layered modeling, dimensional modeling, data governance, metadata, and data quality management
Proficient in metric design and metric calculation development; experience in BI system development is a plus
Data visualization expertise, with the ability to build reporting and self-service analytics platforms
Understanding of LLM / RAG / agent principles, able to combine AI for data analysis and metric-driven applications
Proficient in relational databases such as MySQL; familiar with vector databases; experienced with Docker / K8s and job scheduling tools
Proficient in at least one of Java / Scala / Python, with system design and architecture optimization capabilities