【AI Lab】資深數據工程師 Senior Data Engineer

【Responsibilities】

• Design, build, and maintain scalable data pipelines that meet evolving product and analytics needs.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
• Create data tools that enable analytics and data scientist team members to build and ship faster.
• Build and maintain the services that deliver our data to product, analytics, and external consumers.
• Partner with data and analytics teams to evolve our data infrastructure.
• Support cross functional, cross BU data integration tasks.

【Must to Have】

• Bachelor's degree in Computer Science, Engineering, or a related field, or equivalent practical experience; Master's is a plus
• Strong proficiency in Python 3.10+ and shell scripting (Bash), with 5+ years of backend / data engineering experience and 3+ years owning production data pipelines
• Experience with workflow orchestration tools, e.g. Argo Workflow, Airflow, Dagster
• Experience with RDBMS, OLAP, and search databases, e.g. PostgreSQL, StarRocks (or BigQuery / ClickHouse / Snowflake / Druid), Elasticsearch
• Experience with backend API development and deployment, e.g. gRPC, REST (FastAPI)
• Experience deploying and operating services on Kubernetes with Docker
• Comfortable developing, deploying, and debugging in Linux environments

【Nice to Have】

• Basic knowledge of machine learning algorithms
• Basic knowledge of Data Lake and Data Warehouse Design
• Experience with streaming processing systems, e.g. Kafka, Apache Flink, Cloud Dataflow, Apache Beam, Pub/Sub
• Experience with Go programming language
• Experience with gRPC and Protocol Buffers
• Experience with production-grade operations — monitoring, debugging, and maintaining your own work after deployment