【TENG1402】Data Engineer

【 What You'll be Doing 】

• Pipeline Implementation: Develop, maintain, and optimize robust ETL and real-time data pipelines to ensure high-quality data collection, processing, and storage.
• System Monitoring: Actively monitor data pipeline health and troubleshoot data quality or performance issues to ensure high availability.
• Database Management: Assist in designing data schemas and implementing database indexing or partitioning strategies for performance optimization.
• API & Tooling: Build and support backend APIs and internal data tools to empower SHOPLINE merchants with actionable insights through visualization platforms.
• Collaboration: Work closely with Senior Engineers and Product Managers to translate business requirements into technical data solutions.

【 Who We are Looking for 】

• Core Programming: Proficient in Python and SQL; experience with API frameworks such as FastAPI, Flask, or Django.
• Data Engineering: Solid experience in building ETL/ELT workflows using orchestration tools (Apache Airflow is preferred).
• Streaming Knowledge: Practical experience with message brokers and stream-processing technologies (e.g., Kafka, RisingWave, or Flink).
• Storage & Analytics: Familiar with cloud data warehouses (e.g., BigQuery) and relational/NoSQL databases (e.g., TiDB, PostgreSQL, or MongoDB).
• Infrastructure: Experience with cloud platforms (AWS or GCP) and containerization tools like Docker and Kubernetes.
• Mindset: A proactive problem-solver with a strong sense of ownership and the ability to thrive in a fast-paced e-commerce environment.
• Plus: Previous experience in e-commerce or data analysis is a significant advantage.