Data Engineer
Summary
Data Engineer building and modernizing cloud data pipelines on AWS using Databricks (PySpark, Delta Lake) and Informatica, with a focus on ETL/ELT, data migration from legacy systems, and medallion architecture.
About the Role
We are looking for a skilled Data Engineer to help modernize, scale, and optimize our cloud data platform. In this role, you will build robust data pipelines, migrate legacy workflows, and manage enterprise datasets using AWS, Databricks, and Informatica .
Key Responsibilities
- Pipeline Development: Design and maintain scalable batch/real-time ETL/ELT pipelines using Informatica (IDMC) and Databricks.
- Architecture: Implement Delta Lake medallion architecture (Bronze/Silver/Gold) and data governance via Unity Catalog on AWS.
- Data Migration: Convert legacy Informatica/on-prem ETL workflows into modern PySpark and SQL cloud solutions.
- Optimization: Fine-tune Spark performance, manage AWS cloud infrastructure costs, and automate CI/CD deployments.
Key Requirements
- Technical Stack: Databricks (PySpark, Delta Lake), Informatica (IDMC), AWS (S3, Glue, Lambda, Redshift).
- Core Languages: Strong proficiency in Python and SQL .
- Experience: 3–6+ years of hands-on data engineering, data modeling, and cloud pipeline experience.