Senior/Lead Data Engineer
Posted Updated
Summary
The Senior/Lead Data Engineer designs, develops, and optimizes scalable data pipelines and warehouses using Alibaba Cloud's big data ecosystem, including MaxCompute, Hologres, DataWorks, and Flink. They ensure data quality, translate business requirements into technical solutions, and mentor team members.
Job Overview:
The Senior/Lead Data Engineer is responsible for designing, developing, and optimizing scalable data solutions on Alibaba Cloud's big data platforms, including MaxCompute and Hologres. This role will take a hands-on approach in building robust data pipelines, ensuring data quality and integrity, and supporting the organization's data-driven initiatives. The ideal candidate has deep experience with Alibaba Cloud's big data ecosystem and is proficient in leading technical efforts in complex data warehousing and processing environments.
Responsibilities: 1. Data Pipeline Development and Optimization Design and implement efficient ETL/ELT pipelines on Alibaba Cloud to support ingestion, transformation, and loading of large-scale data Integrate data from multiple systems into a centralized data warehouse, ensuring consistency, accuracy, and accessibility for analytics and reporting Continuously optimize and monitor data pipelines to ensure high performance, scalability, and efficient resource usage within Alibaba Cloud environments
2. Data Warehouse Management Develop, manage, and optimize big data warehouses on Alibaba Cloud, using tools like Hologres, MaxCompute, and DataWorks Implement data modeling best practices to structure and organize data for optimal accessibility, flexibility, and query performance Establish data validation, monitoring, and quality assurance practices to maintain high data quality standards across the data warehouse
3. Collaboration and Team Management Work closely with data analysts, data scientists, and business stakeholders to understand data needs and provide tailored data solutions Translate business requirements into technical specifications for data pipelines, storage, and processing solutions Provide guidance on data engineering best practices and data warehouse solutions to team members and support their technical growth
4. Data Governance and Compliance Maintain clear and comprehensive documentation for data pipelines, data warehouse structures, and data governance practices Ensure compliance with data governance policies and standards for data security and privacy, working closely with security and governance teams
Requirements: Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or related field 5+ years in data engineering, with strong hands-on experience in Alibaba Cloud, particularly Hologres, MaxCompute, and related big data tools Proficiency in big data technologies (e.g., Hologres, MaxCompute, DataWorks, Flink), Strong SQL, Python, or Spark skills for data manipulation and processing Ability to work independently and take initiative on complex data engineering tasks Capacity to lead technical efforts, mentor junior engineers, and set standards for data engineering best practices Team player with strong collaboration skills, able to work effectively with both internal and external teams
The Senior/Lead Data Engineer is responsible for designing, developing, and optimizing scalable data solutions on Alibaba Cloud's big data platforms, including MaxCompute and Hologres. This role will take a hands-on approach in building robust data pipelines, ensuring data quality and integrity, and supporting the organization's data-driven initiatives. The ideal candidate has deep experience with Alibaba Cloud's big data ecosystem and is proficient in leading technical efforts in complex data warehousing and processing environments.
Responsibilities: 1. Data Pipeline Development and Optimization Design and implement efficient ETL/ELT pipelines on Alibaba Cloud to support ingestion, transformation, and loading of large-scale data Integrate data from multiple systems into a centralized data warehouse, ensuring consistency, accuracy, and accessibility for analytics and reporting Continuously optimize and monitor data pipelines to ensure high performance, scalability, and efficient resource usage within Alibaba Cloud environments
2. Data Warehouse Management Develop, manage, and optimize big data warehouses on Alibaba Cloud, using tools like Hologres, MaxCompute, and DataWorks Implement data modeling best practices to structure and organize data for optimal accessibility, flexibility, and query performance Establish data validation, monitoring, and quality assurance practices to maintain high data quality standards across the data warehouse
3. Collaboration and Team Management Work closely with data analysts, data scientists, and business stakeholders to understand data needs and provide tailored data solutions Translate business requirements into technical specifications for data pipelines, storage, and processing solutions Provide guidance on data engineering best practices and data warehouse solutions to team members and support their technical growth
4. Data Governance and Compliance Maintain clear and comprehensive documentation for data pipelines, data warehouse structures, and data governance practices Ensure compliance with data governance policies and standards for data security and privacy, working closely with security and governance teams
Requirements: Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or related field 5+ years in data engineering, with strong hands-on experience in Alibaba Cloud, particularly Hologres, MaxCompute, and related big data tools Proficiency in big data technologies (e.g., Hologres, MaxCompute, DataWorks, Flink), Strong SQL, Python, or Spark skills for data manipulation and processing Ability to work independently and take initiative on complex data engineering tasks Capacity to lead technical efforts, mentor junior engineers, and set standards for data engineering best practices Team player with strong collaboration skills, able to work effectively with both internal and external teams
What they ask for
Required
- Bachelor's or Master's degree in Computer Science, Data Engineering, IT or related field
- 5+ years in data engineering with hands-on Alibaba Cloud (Hologres, MaxCompute) experience
- Proficiency in Hologres, MaxCompute, DataWorks, Flink, SQL, Python, or Spark
- Ability to work independently and take initiative on complex tasks
- Capacity to lead technical efforts and mentor junior engineers
- Strong collaboration skills for working with internal and external teams