Data Engineer - Traffic Mart
Posted 1
view
Summary
Data engineer on Shopee's Marketplace team who builds and operates the Traffic Mart data warehouse end to end — batch and real-time ingestion, semantic layer, and large-scale pipelines for BI, analytics, and algorithmic applications using Spark, Flink, Hadoop/HBase, and languages like Java, Scala, Python, and SQL.
Job Description:
- Build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers.
- Collaborate with product, business, and engineering teams to translate requirements into scalable data architectures that support BI reporting, self-serve analytics, and algorithmic applications.
- Design and optimize large-scale data processing pipelines, improving throughput, latency, and cost efficiency across offline and streaming platforms.
- Lead data quality and governance efforts, including schema standardization, lineage tracking, SLA monitoring, and best practices that increase reliability and user trust.
- Evaluate and adopt emerging data technologies such as streaming engines, lakehouse architectures, and AI/ML data stacks to address key Marketplace challenges and evolve the platform roadmap.
Requirements:
- Bachelor's Degree or above in Computer Science or related fields
- At least 3 years of data engineering experience
- Prior experience or knowledge working with Traffic Mart would be advantageous
- Experience with big data processing technologies such as Spark, Flink, Hadoop or HBase
- Proficiency in one or more programming languages, such as Java, Scala, Python, SQL, etc.
- Familiar with data warehouse architecture and principles