Senior Data Engineer
Summary
Senior Data Engineer designing and optimizing scalable ETL pipelines with Databricks on AWS, PySpark, and Python, including refactoring legacy Informatica DEI workflows, to deliver high-quality governed data for analytics and data science teams in a client-facing environment.
Role Overview :
We are seeking a highly skilled Senior Data Engineer with a strong background in building scalable data pipelines, performing complex data transformations, and enabling analytics workloads. The ideal candidate should have hands-on experience with Databricks on AWS, PySpark, and Python, with a working understanding of ETL orchestration and automation. Familiarity with Informatica DEI is a plus, as the role may involve refactoring existing Informatica workflows into PySpark.
Key Responsibilities:
- Design and implement robust, scalable data pipelines using Databricks on AWS and PySpark
- Perform data ingestion, transformation, cleansing, and validation from various sources
- Collaborate with analytics and data science teams to deliver high-quality data for reporting and modeling
- Understand and translate Informatica DEI workflows into Databricks-based implementations (if applicable)
- Optimize and automate ETL workflows for performance and cost-efficiency
- Ensure data quality, lineage, and governance are upheld in all solutions
Required Skills & Experience:
- Minimum 8 years of experience in Data Engineering
- Strong proficiency in Databricks on AWS, PySpark, and Python
- Solid experience in ETL development, data transformations, cleaning, data quality frameworks/standards, and pipeline automation
- Understanding of data quality principles and performance tuning techniques - Familiarity with Informatica DEI is a nice to have
- Excellent communication skills in client-facing environments
2. SKILLS
- Data Engineering
- Data Science
- ETL
- Informatica
- Java