Data Engineer (Data lake/Data warehouse)
Summary
Senior Data Engineer designing and maintaining a large-scale Weather Data Lakehouse platform, building cloud-based data pipelines for ingestion, transformation, and processing to support analytics and AI/ML workloads.
Our client is seeking a Senior Data Engineer to support the development and enhancement of a large-scale Weather Data Lakehouse platform. The platform serves as a central repository for meteorological, climate, and operational data, enabling analytics, AI/ML initiatives, research, and operational forecasting.
Responsibilities
Design, develop, and maintain scalable data ingestion, transformation, and processing pipelines.
Build and enhance cloud-based data lakehouse architectures for analytics and AI/ML workloads.
Implement data engineering best practices including data governance, metadata management, monitoring, and automation.
Optimize data platform performance, scalability, reliability, and operational efficiency.
Support CI/CD, DevOps, and SRE practices to improve deployment and system resiliency.
Troubleshoot and resolve complex data pipeline and platform issues.
Ensure compliance with security requirements and enterprise architecture standards.
Collaborate with cross-functional teams to deliver high-quality data solutions.
Requirements
Degree in Computer Science, Data Engineering, Information Systems, or related disciplines.
Strong experience in designing and implementing enterprise-scale data platforms, data lakes, or lakehouse solutions.
Hands-on experience in data engineering, ETL/ELT development, cloud technologies, and distributed data processing frameworks.
Experience with data pipeline orchestration, automation, monitoring, and CI/CD practices.
Familiarity with DevOps and SRE principles in production environments.
Strong analytical, problem-solving, and stakeholder management skills.