Data Engineer - Traffic Mart
Posted Updated 3
views
Summary
Builds and operates Shopee Marketplace's data warehouse end to end — batch and real-time ingestion, semantic layer design, and data modeling for BI, self-serve analytics, and algorithmic applications — while optimizing large-scale pipelines and leading data quality and governance, using big data tech like Spark, Flink, Hadoop and HBase.
Job Description:
Build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers. Collaborate with product, business, and engineering teams to translate requirements into scalable data architectures that support BI reporting, self-serve analytics, and algorithmic applications. Design and optimize large-scale data processing pipelines, improving throughput, latency, and cost efficiency across offline and streaming platforms. Lead data quality and governance efforts, including schema standardization, lineage tracking, SLA monitoring, and best practices that increase reliability and user trust. Evaluate and adopt emerging data technologies such as streaming engines, lakehouse architectures, and AI/ML data stacks to address key Marketplace challenges and evolve the platform roadmap.
Requirements:
Bachelor's Degree or above in Computer Science or related fields At least 3 years of data engineering experience Prior experience or knowledge working with Traffic Mart would be advantageous Experience with big data processing technologies such as Spark, Flink, Hadoop or HBase Proficiency in one or more programming languages, such as Java, Scala, Python, SQL, etc. Familiar with data warehouse architecture and principles
Build and operate the Marketplace data warehouse end to end, covering batch and real-time ingestion, semantic layer design, and application-layer modeling for downstream consumers. Collaborate with product, business, and engineering teams to translate requirements into scalable data architectures that support BI reporting, self-serve analytics, and algorithmic applications. Design and optimize large-scale data processing pipelines, improving throughput, latency, and cost efficiency across offline and streaming platforms. Lead data quality and governance efforts, including schema standardization, lineage tracking, SLA monitoring, and best practices that increase reliability and user trust. Evaluate and adopt emerging data technologies such as streaming engines, lakehouse architectures, and AI/ML data stacks to address key Marketplace challenges and evolve the platform roadmap.
Requirements:
Bachelor's Degree or above in Computer Science or related fields At least 3 years of data engineering experience Prior experience or knowledge working with Traffic Mart would be advantageous Experience with big data processing technologies such as Spark, Flink, Hadoop or HBase Proficiency in one or more programming languages, such as Java, Scala, Python, SQL, etc. Familiar with data warehouse architecture and principles