AI Data Platform Engineer
Posted Updated
Summary
Builds the data-asset platform behind MiniMax's LLM pretraining: ingestion, catalog/search, access control, quality gates, and APIs/SDKs that keep pretraining data discoverable, traceable, and reliable. Work centers on Python and large-scale data engineering with ML/LLM understanding.
You will build data-asset management and delivery capabilities for pretraining data. You will create catalog, search, access-control, quality, API, SDK, and self-service tools that make data discoverable, traceable, reusable, and reliable throughout its lifecycle.
Responsibilities
- Design and build pretraining-data asset-management capabilities
- Develop data-ingestion, development, processing, and publishing workflows
- Build data catalog, search, preview, access-control, and data-service capabilities
- Establish data-quality rules, validation, sampling checks, reports, and quality gates
- Design APIs, SDKs, and self-service tools for data algorithms and experiments
- Improve the data-asset lifecycle from production through consumption
Requirements
- Proficiency in Python for data analysis, algorithm implementation, and engineering development
- Strong foundation in data algorithms or large-scale data engineering
- Experience developing, running, and debugging large-scale data-processing tasks
- Knowledge of data quality assessment using statistics, sampling, and error cases
- Basic understanding of machine learning and large-language-model pretraining
- Ability to understand data controls, controlled variables, and model-evaluation results
- Analysis, communication, and collaboration skills
