Senior Data Engineer - MPID Team
Summary
Builds and optimizes Shopee’s Marketplace data warehouse, designs pipelines for real-time and batch data, and mentors junior engineers.
Job Description:
Participate in the development work related to the Marketplace data warehouse, including data collection in offline and real-time data warehouses, data public layer construction, data application layer construction, and data governance. Participate in supporting the data application of various business modules, cooperate with different teams to communicate the requirement, and design data architecture to provide data users with efficient data solutions, including supporting various scenarios such as BI analysis, data products, and algorithm applications. Participate in the exploration and breakthrough of Marketplace's key data technologies, and be responsible for optimizing and improving the existing data architecture, improving data quality and productivity, and including massive data processing, real-time data processing, and the application of various new technologies. Mentoring junior data engineers to do data pipelines.
Requirements:
Bachelor's degree or higher in Computer Science, Information technology or related fields. At least 2 years of full time experience in data engineering, data analytics, or software engineering with data-intensive systems. Preferably experience with data warehouse architecture and principles Preferably experience with one or more of the big data processing technologies such as Spark, Flink, Hadoop, HBase, Kafka, Druid, Clickhouse, etc. Proficiency in SQL and one or more programming languages, such as Java, Scala, Python, SQL, etc. Good thinking logic and communication skills, as well as good project management and coordination skills
Participate in the development work related to the Marketplace data warehouse, including data collection in offline and real-time data warehouses, data public layer construction, data application layer construction, and data governance. Participate in supporting the data application of various business modules, cooperate with different teams to communicate the requirement, and design data architecture to provide data users with efficient data solutions, including supporting various scenarios such as BI analysis, data products, and algorithm applications. Participate in the exploration and breakthrough of Marketplace's key data technologies, and be responsible for optimizing and improving the existing data architecture, improving data quality and productivity, and including massive data processing, real-time data processing, and the application of various new technologies. Mentoring junior data engineers to do data pipelines.
Requirements:
Bachelor's degree or higher in Computer Science, Information technology or related fields. At least 2 years of full time experience in data engineering, data analytics, or software engineering with data-intensive systems. Preferably experience with data warehouse architecture and principles Preferably experience with one or more of the big data processing technologies such as Spark, Flink, Hadoop, HBase, Kafka, Druid, Clickhouse, etc. Proficiency in SQL and one or more programming languages, such as Java, Scala, Python, SQL, etc. Good thinking logic and communication skills, as well as good project management and coordination skills
