Data Engineer

Monitor and maintain ETL/ELT pipelines and Apache Spark batch processing jobs to ensure stable and reliable data operations.

Monitor WeData or equivalent ETL platform execution, job schedulers, dependencies, execution logs, and system performance.

Investigate, troubleshoot, and resolve ETL/Spark job failures and production incidents.

Perform job rerun, recovery, performance tuning, and workaround implementation based on operational procedures.

Conduct data validation to ensure data quality and pipeline integrity.

Perform Root Cause Analysis (RCA) for recurring or critical production issues and recommend preventive actions.

Prepare daily monitoring reports, incident reports, and technical documentation.

Escalate complex issues to L3 Platform Engineers or Tencent Cloud support when required.

Collaborate with cross-functional teams to ensure production stability and continuous service improvement.

Provide technical guidance and support to junior team members (for Senior level).

Requirements

Bachelor's Degree in Computer Science, Information Technology, Engineering, or a related field.

Minimum 2 - 5 years of experience as a Data Engineer, ETL Support, Data Analyst, or Fresh Graduate with internship/project experience in Big Data or Data Engineering.

Strong understanding of ETL/ELT concepts and data pipeline architecture.

Experience with Apache Spark, including Spark SQL, batch processing, and job troubleshooting (advanced proficiency required for Senior level).

Familiarity with Tencent WeData or similar ETL platforms such as Airflow, Informatica, Talend, Ab Initio, AWS Glue, or Alibaba DataWorks.

Good SQL skills for data validation, troubleshooting, and query optimization (advanced SQL required for Senior level).

Familiarity with the Hadoop ecosystem (Hive, HDFS, YARN) is an advantage.

Basic knowledge of Linux commands and Shell scripting.

Understanding of job monitoring, execution logs, schedulers, dependencies, and recovery processes.

Strong analytical, troubleshooting, and problem-solving skills.

Good communication skills and the ability to work collaboratively in a production support environment.

Available to join as soon as possible (ASAP).

Information Technology Services 51-100 employees

IDstar is an IT Consultant that focused on Business Transformation Solution to any business on a global scale by providing IT Outsourcing, RPA Solutions and Data Extraction.

Our team of 250+ specialized IT professionals is dedicated to providing top-notch IT outsourcing, RPA solutions, and data extraction services. With over 205+ automation solutions and 10+ proven data extraction templates, we help more than 25+ companies adapt to the digital world.

Our Vision: We aspire to see a digital-ready generation that innovates and leads smart businesses across asia.

Our Mission: To Build, Accelerate, Support and Empower every organization business transformation.

IDstar is an IT Consultant that focused on Business Transformation Solution to any business on a global scale by providing IT Outsourcing, RPA Solutions and Data Extraction.

Our team of 250+ specialized IT professionals is dedicated to providing top-notch IT outsourcing, RPA solutions, and data extraction services. With over 205+ automation solutions and 10+ proven data extraction templates, we help more than 25+ companies adapt to the digital world.

Our Vision: We aspire to see a digital-ready generation that innovates and leads smart businesses across asia.

Our Mission: To Build, Accelerate, Support and Empower every organization business transformation.