Sr Data Engineer Databricks and Informatica
Summary
Design and implement scalable data platforms using Databricks Lakehouse, lead ETL modernization from Informatica to Databricks, and architect batch and real-time pipelines on cloud platforms using Spark, PySpark, SQL, and Delta Lake.
Job Title: Sr Data Engineer with strong Databricks and Informatica
Brampton, ON
Contract: 6 - 12 Months
Hybrid 4 days in Office
Job Description:
- Databricks Architect (with Informatica)-Databricks, Informatica PowerCenter and/or IICS/IDMC, Spark, PySpark, SQL, Delta Lake, Unity Catalog.
- Brampton, ON - Hybrid (4 Days WFO)
- Contract: 6-12 months
Role Descriptions:
- Design and implement scalable data platforms using Databricks Lakehouse| Delta Lake| Spark| and PySpark.Architect batch and real-time data pipelines on Azure/AWS/GCP.
- Lead ETL modernization and migration from Informatica PowerCenter/IICS/IDMC to Databricks.
- Define data governance| security| performance optimization| and best practices.
- Collaborate with business and technical teams to deliver enterprise data solutions.
- Required Skills:
- Strong expertise in Databricks| Spark| PySpark| SQL| Delta Lake| Unity Catalog.
- Hands-on experience with Informatica PowerCenter and/or IICS/IDMC.
- Experience in data warehousing| data modeling| and cloud platforms (Azure preferred).
- Knowledge of CI/CD| Git| and Agile methodologies.
- Preferred Skills:
- Databricks and Cloud certifications
- Experience with Snowflake| Azure Synapse| Kafka| or Data Governance tools.