Data Engineer (AWS+Pyspark)
Posted Updated 2
views
Summary
Build and maintain AWS-based data pipelines using PySpark and Python, handling ingestion, transformation, and analytics across cloud services like EMR, Glue, and RDS.
DRequired Skills
- 4–7 years of hands-on experience as a Data Engineer, designing, building, and supporting data pipelines on AWS.
- Strong programming skills in Python and PySpark, with hands-on experience in distributed data processing and big data frameworks.
- Strong SQL expertise, including complex query writing, performance optimization, and working with large datasets across relational and analytical databases.
- Proven experience working with AWS core services, including Lambda, RDS, CloudWatch, CloudTrail, SNS, and SQS.
- Hands-on experience with AWS data and analytics services such as:
IAM, EMR, AWS Glue, Lambda, Lake Formation, RDS, DynamoDB, and related services. - Strong expertise in IAM management, including role design, policy creation, permission boundaries, and enforcing least-privilege access.
- Experience in building and maintaining end-to-end data ingestion, transformation, and processing pipelines.
- Solid understanding of monitoring, logging, and alerting, leveraging CloudWatch and CloudTrail.
- Excellent debugging, troubleshooting, and root cause analysis skills across data pipelines and AWS services.