Data Engineer
Summary
Designs and maintains scalable ETL pipelines and data integration processes using Python, SQL, Hadoop, and Spark to consolidate enterprise data.
Responsibilities
- Design, develop, and maintain scalable ETL pipelines to extract, transform, and load data from multiple sources into enterprise data platforms.
- Implement and manage data integration processes to consolidate data from various systems while ensuring data accuracy, consistency, and reliability.
- Develop and optimize data processing solutions using Python, SQL, and big data technologies such as Hadoop and Spark.
- Monitor, troubleshoot, and enhance ETL workflows to improve performance, data quality, and overall pipeline efficiency.
- Collaborate with cross-functional teams to understand business requirements, support data modelling initiatives, and deliver reliable data solutions
- Design, develop, and maintain scalable ETL pipelines to extract, transform, and load data from multiple sources into enterprise data platforms.
- Implement and manage data integration processes to consolidate data from various systems while ensuring data accuracy, consistency, and reliability.
- Develop and optimize data processing solutions using Python, SQL, and big data technologies such as Hadoop and Spark.
- Monitor, troubleshoot, and enhance ETL workflows to improve performance, data quality, and overall pipeline efficiency.
- Collaborate with cross-functional teams to understand business requirements, support data modelling initiatives, and deliver reliable data solutions
Requirements
- Proven experience as a Data Engineer with strong expertise in data integration, ETL development, and big data technologies.
- Proficiency in Python and SQL for developing, optimizing, and maintaining data pipelines and data processing solutions.
- Hands-on experience with big data frameworks such as Hadoop, Spark, or similar distributed data processing technologies.
- Solid understanding of data modelling concepts, database design principles, and ETL best practices.
- Strong analytical and problem-solving skills with the ability to ensure data quality, consistency, and scalability across enterprise data platforms