Data Engineer (ETL/Data Warehouse)
Our client is seeking a Senior Data Engineer to support the development and enhancement of a large-scale Weather Data Lakehouse platform. The platform serves as a central repository for meteorological, climate, and operational data, enabling analytics, AI/ML initiatives, research, and operational forecasting.
Responsibilities Design, develop, and maintain scalable data ingestion, transformation, and processing pipelines. Build and enhance cloud-based data lakehouse architectures for analytics and AI/ML workloads. Implement data engineering best practices including data governance, metadata management, monitoring, and automation. Optimize data platform performance, scalability, reliability, and operational efficiency. Support CI/CD, DevOps, and SRE practices to improve deployment and system resiliency. Troubleshoot and resolve complex data pipeline and platform issues. Ensure compliance with security requirements and enterprise architecture standards. Collaborate with cross-functional teams to deliver high-quality data solutions.
Requirements Degree in Computer Science, Data Engineering, Information Systems, or related disciplines. Strong experience in designing and implementing enterprise-scale data platforms, data lakes, or lakehouse solutions. Hands-on experience in data engineering, ETL/ELT development, cloud technologies, and distributed data processing frameworks. Experience with data pipeline orchestration, automation, monitoring, and CI/CD practices. Familiarity with DevOps and SRE principles in production environments. Strong analytical, problem-solving, and stakeholder management skills.
Responsibilities Design, develop, and maintain scalable data ingestion, transformation, and processing pipelines. Build and enhance cloud-based data lakehouse architectures for analytics and AI/ML workloads. Implement data engineering best practices including data governance, metadata management, monitoring, and automation. Optimize data platform performance, scalability, reliability, and operational efficiency. Support CI/CD, DevOps, and SRE practices to improve deployment and system resiliency. Troubleshoot and resolve complex data pipeline and platform issues. Ensure compliance with security requirements and enterprise architecture standards. Collaborate with cross-functional teams to deliver high-quality data solutions.
Requirements Degree in Computer Science, Data Engineering, Information Systems, or related disciplines. Strong experience in designing and implementing enterprise-scale data platforms, data lakes, or lakehouse solutions. Hands-on experience in data engineering, ETL/ELT development, cloud technologies, and distributed data processing frameworks. Experience with data pipeline orchestration, automation, monitoring, and CI/CD practices. Familiarity with DevOps and SRE principles in production environments. Strong analytical, problem-solving, and stakeholder management skills.