Big Data Engineer (Libra) - Data Platform
Summary
Build and maintain a PB-scale data warehouse and ETL pipelines for a company-wide A/B testing platform, supporting offline and real-time metric processing.
About the team
Libra is a large-scale online one-stop A/B testing platform developed by Data Platform. Some of its features include:
- Provides experimental evaluation services for all product lines within the company, covering solutions for complex scenarios such as recommendation, algorithm, function, UI, marketing, advertising, operation, social isolation, causal inference, etc.
- Provides services throughout the entire experimental lifecycle from experimental design, experimental creation, indicator calculation, statistical analysis to final evaluation launch.
- Supports the entire company's business on the road of rapid iterative trial and error, boldly assuming and carefully verifying.
Responsibilities
- Responsible for data system of experimentation platform operation and maintenance.
- Construct PB-level data warehouses, participate in and be responsible for data warehouse design, modeling, and development, etc.
- Build ETL data pipelines and automated ETL data pipeline systems.
- Build an expert system for metric data processing that combines offline and real-time processing.