Data Engineer
Posted Updated 11
views
Summary
A lead data engineer role in Singapore managing day-to-day BAU data platform operations, production support, and a team of data engineers, while designing AWS-based data architecture. Core technologies include AWS (S3, Glue, DMS, MWAA, IAM), Snowflake, Airflow, SQL, Python, and PySpark.
Key Skills:
▪ Lead a team of data engineers in managing day-to-day BAU operations, production support, incident management, and platform stability. ▪ Strong AWS knowledge in terms of designing new architecture and providing optimized solutions for existing ones. (S3, Glue, DMS, MWAA, AMS, IAM). ▪ In-depth knowledge with respect to Snowflake and its architecture. ▪ Prefer prior Experience in BAU environment. ▪ Good knowledge on Airflow and MWAA. ▪ Hands-on experience in SQL/Python/Pyspark. ▪ Expertise in optimizing techniques in cloud environments. ▪ Should have the vision on data strategy and able to deliver the same. ▪ Should be able to lead the design and implementation of data management processes, including data sourcing, integration, and transformation. ▪ Able to manage and lead a team of data professionals, providing guidance, mentoring and foster a collaborative and innovative team culture focused on continuous improvement. ▪ To evaluate and recommend data-related technologies, tools, and platforms. ▪ Collaborate with IT teams to ensure seamless integration of data solutions. ▪ Should have experience in Implementing and enforcing data security protocols and ensure compliance with relevant regulations. Required Qualifications: ▪ At least 8+ years of Data Engineer Experience ▪ Bachelor qualification in a computer science or STEM (science, technology, engineering, or mathematics) related field. ▪ At least 5+ years of recent hands-on professional experience (actively coding) working as a Lead handling support & production issue. ▪ 2+ experience with large scale datasets, data lake and data warehouse technologies such as AWS Redshift, Google BigQuery, Snowflake. Snowflake is highly preferred.
▪ Lead a team of data engineers in managing day-to-day BAU operations, production support, incident management, and platform stability. ▪ Strong AWS knowledge in terms of designing new architecture and providing optimized solutions for existing ones. (S3, Glue, DMS, MWAA, AMS, IAM). ▪ In-depth knowledge with respect to Snowflake and its architecture. ▪ Prefer prior Experience in BAU environment. ▪ Good knowledge on Airflow and MWAA. ▪ Hands-on experience in SQL/Python/Pyspark. ▪ Expertise in optimizing techniques in cloud environments. ▪ Should have the vision on data strategy and able to deliver the same. ▪ Should be able to lead the design and implementation of data management processes, including data sourcing, integration, and transformation. ▪ Able to manage and lead a team of data professionals, providing guidance, mentoring and foster a collaborative and innovative team culture focused on continuous improvement. ▪ To evaluate and recommend data-related technologies, tools, and platforms. ▪ Collaborate with IT teams to ensure seamless integration of data solutions. ▪ Should have experience in Implementing and enforcing data security protocols and ensure compliance with relevant regulations. Required Qualifications: ▪ At least 8+ years of Data Engineer Experience ▪ Bachelor qualification in a computer science or STEM (science, technology, engineering, or mathematics) related field. ▪ At least 5+ years of recent hands-on professional experience (actively coding) working as a Lead handling support & production issue. ▪ 2+ experience with large scale datasets, data lake and data warehouse technologies such as AWS Redshift, Google BigQuery, Snowflake. Snowflake is highly preferred.