Data Engineer
Posted Updated
Summary
Data Engineer in Gurugram designing, building, and maintaining large-scale ETL data pipelines on AWS, plus developing data warehousing solutions and troubleshooting data quality, performance, and security. Core tech: Python, Spark, Glue, Hadoop, Kafka, and AWS services like S3, Lambda, and Redshift.
Roles and Responsibilities :
- Design, develop, test, deploy and maintain large-scale data pipelines using ETL tools such as Spark and Glue.
- Collaborate with cross-functional teams to gather requirements for data warehousing solutions and implement them efficiently.
- Develop scalable architectures on AWS cloud platform to handle high-volume datasets from various sources.
- Troubleshoot issues related to data quality, performance, and security in the data pipeline.
Job Requirements :
- 4-9 years of experience in Data Engineering with expertise in ETL (Extract Transform Load) processes.
- Strong understanding of Python programming language and its libraries like Pandas, NumPy, Matplotlib etc. .
- Experience working with big-data technologies like Spark, Hadoop, Kafka etc. .
- Knowledge of AWS services including S3 buckets, Lambda functions, Redshift tables etc. .