Senior Data Engineer (AWS)
Summary
Senior Data Engineer building and supporting a config-driven, multi-tenant AWS Data Lakehouse platform: developing shared frameworks, resolving production pipeline issues, and consulting for onboarding tenant teams. Core stack is AWS (Glue, EMR, Redshift), Python, PySpark, SQL, and Terraform in an infrastructure-as-code environment.
Job Description
Job Title: Senior Data Engineer (AWS)
Years of Experience: 7+Years
Notice Period: Immediate to 30 Days
Team: Enterprise AI & Data — Platform & Architecture
About the Role
We are looking for a Senior Data Engineer to support and enhance our config-driven, multi-tenant AWS Data Lakehouse platform. You will work across multiple tenants, contribute to framework development, resolve production issues, and provide technical consulting to onboarding teams. This is a hands-on engineering role requiring strong AWS, Python, PySpark, SQL, and Terraform skills within a fast-paced, infrastructure-as-code heavy environment.
Required Skills & Experience
- 7+ years of hands-on data engineering experience in cloud-native environments (AWS required)
- 5+ years of strong proficiency in Python and PySpark for data processing, transformation, and pipeline development
- 7+ years of experience in advanced SQL skills , complex queries, window functions, performance tuning, query optimization across Redshift and Spark SQL
- 5+ years of hands on experience with AWS Glue, EMR, Databricks, or similar Spark-based compute services
- 5+ years of experience with Terraform for infrastructure provisioning and management
- 5+ years of experience in different ingestion & ETL tools like Glue, DMS, Golden Gate, HVR, Informatica, Talend etc.
- Experience supporting multi-tenant platforms, understanding tenant isolation, shared frameworks, and config-driven patterns
- Strong troubleshooting and debugging skills — ability to diagnose and resolve production pipeline failures quickly
- Excellent communication skills, ability to engage with multiple tenant teams, provide technical consulting, and explain complex solutions clearly
Nice to Have
- Experience with Airflow or Autosys for workflow orchestration
- Experience with AWS Lake Formation for governance and fine-grained access control
- Familiarity with SAP source systems or enterprise ERP data
- Experience with CloudWatch for monitoring and observability
- AWS GovCloud experience
- Data modeling experience (star schema, data vault)
- Experience with Parquet, Avro, and Iceberg file/table formats