Cloud Data Engineer
Summary
Cloud Data Engineer responsible for designing scalable batch and real-time data pipelines, managing cloud data platforms on AWS, Azure, or GCP, and developing ETL/ELT processes with IaC and CI/CD automation.
- Design, develop, and maintain scalable data pipelines for batch and real-time data processing.
- Build and manage cloud-based data platforms using AWS, Azure, or Google Cloud.
- Develop ETLELT processes to extract, transform, and load data from multiple sources.
- Create and optimize data lakes, data warehouses, and data marts.
- Ensure data quality, integrity, security, and compliance across data platforms.
- Implement data integration solutions using cloud-native services and big data technologies.
- Monitor and troubleshoot data pipeline performance and resolve production issues.
- Collaborate with business analysts, data scientists, architects, and application teams to understand data requirements.
- Optimize data storage and processing for performance and cost efficiency.
- Implement automation, CICD, and Infrastructure as Code (IaC) practices for data platforms.
- Manage metadata, data lineage, and data governance processes.
- Support reporting, analytics, AIML, and business intelligence initiatives by providing reliable data solutions.