Data Engineer (IT Consulting)
Summary
Design and maintain ETL pipelines and data platforms using Hadoop, Spark, Kafka, and cloud services (AWS/GCP/Azure) to ensure reliable data flow and integrity.
About the job Data Engineer (IT Consulting)
On behalf of our client, an IT Consulting company that is looking to hire a Data Engineer position with details below
Responsibilities
- Design, develop, and maintain robust ETL processes for data transformation, quality, and reliability.
- Build and manage data platforms based on server clusters such as Hadoop, Spark, and Kafka.
- Utilize major cloud service providers (AWS, GCP, Azure) for data platform management.
- Deliver high-quality, well-tested, reusable, and secure code in compliance
- Collaborate with cross-functional teams to understand data requirements and translate them into technical specifications.
- Ensure data integrity and availability for various applications and stakeholders.
- Monitor and troubleshoot data pipelines to ensure seamless data flow.
- Continuously improve data engineering processes and practices.
Qualifications
- Bachelors degree in Computer Science, Information Technology, or a related major.
- Minimum of 2 years of experience as a Data Engineer.
- Proficiency in development languages such as Python, SQL, or other relevant languages.
- Extensive experience in building and managing ETL processes.
- Familiar with ETL tools such as SSIS, Pentaho or Talend
- Experinece in data visualisation such as PowerBI, Locker Studio or Bigquery
- Strong expertise in building data platforms based on server clusters (Hadoop, Spark, Kafka).
- Familiarity with major cloud service providers (AWS, GCP, Azure).
- Ability to deliver high-quality, well-tested, reusable, and secure code.
- Strong technical expertise with various data sources and data types.