Sr. Data Engineer(AWS)
Posted 5
views
Summary
Senior Data Engineer at Trantor in Chandigarh, India, designing and optimizing AWS-based data platforms: building ETL/ELT pipelines with Glue and Step Functions, managing S3 data lakes, running Athena queries, exposing APIs via API Gateway/Lambda/ECS/EKS, and event-driven ingestion with SQS/SNS/EventBridge/Kafka/Kinesis.
- 7+ years of Data Engineering experience with strong AWS cloud expertise and architecture understanding.
- Hands-on experience with AWS S3, Glue, Lambda, Athena, and Step Functions.
- Experience building APIs using API Gateway with Lambda/ECS/EKS.
- Strong experience in S3 bucket design, partitioning, lifecycle policies, IAM, and encryption.
- Hands-on Amazon Athena experience for data validation, ad-hoc queries, and performance optimization on large S3 datasets.
- Strong ETL/ELT pipeline development and optimization experience with AWS Glue and Step Functions.
- Experience with file-based ingestion, including SFTP, file drops, inbox/outbox, and event-driven S3 ingestion.
- Experience migrating data from RDBMS/data warehouses to AWS S3 or Redshift.
- Strong Python and SQL skills for data processing and automation.
- Experience designing data lake architectures using Raw, Curated, and Consumption layers.
- Strong understanding of partitioning, distributed processing, data quality, and pipeline optimization.
- Experience with event-driven pipelines using one or more of SQS, SNS, EventBridge, Kafka, or Kinesis.
- Strong debugging, monitoring, troubleshooting, and performance optimization skills.