Data Engineer - Traffic Mart
Summary
Data Engineer on Shopee's Marketplace team who builds and operates the Traffic Mart data warehouse end to end — batch and real-time ingestion, semantic layer design, and pipeline optimization — while leading data quality and governance. Core tech includes Spark, Flink, Hadoop/HBase, and Java/Scala/Python/SQL.
The Engineering and Technology team is at the core of the Shopee platform development. The team is made up of a group of passionate engineers from all over the world, striving to build the best systems with the most suitable technologies. Our engineers do not merely solve problems at hand; We build foundations for a long-lasting future. We don't limit ourselves on what we can or can't do; we take matters into our own hands even if it means drilling down to the bottom layer of the computing platform. Shopee's hyper-growing business scale has transformed most \"innocent\" problems into huge technical challenges, and there is no better place to experience it first-hand if you love technologies as much as we do.
Job Description:
- Build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers.
- Collaborate with product, business, and engineering teams to translate requirements into scalable data architectures that support BI reporting, self-serve analytics, and algorithmic applications.
- Design and optimize large-scale data processing pipelines, improving throughput, latency, and cost efficiency across offline and streaming platforms.
- Lead data quality and governance efforts, including schema standardization, lineage tracking, SLA monitoring, and best practices that increase reliability and user trust.
- Evaluate and adopt emerging data technologies such as streaming engines, lakehouse architectures, and AI/ML data stacks to address key Marketplace challenges and evolve the platform roadmap.
Requirements:
- Bachelor's Degree or above in Computer Science or related fields
- At least 3 years of data engineering experience
- Prior experience or knowledge working with Traffic Mart would be advantageous
- Experience with big data processing technologies such as Spark, Flink, Hadoop or HBase
- Proficiency in one or more programming languages, such as Java, Scala, Python, SQL, etc.
- Familiar with data warehouse architecture and principles