Senior DataOps / Cloud Data Engineer

• Design, develop, implement, and maintain scalable end-to-end data pipelines from source systems to downstream consumers.
  
• Build and optimize ETL/ELT processes for batch and streaming data integration, ingestion, transformation, and provisioning.
  
• Implement cloud-native data engineering solutions using Databricks, Delta Lake, Databricks SQL, and Azure data services.
  
• Apply Medallion Architecture principles to organize, transform, validate, and manage enterprise data.
  
• Develop and maintain integration workflows using Azure Data Factory, Informatica, or equivalent enterprise integration platforms.
  
• Operationalize common data models and metadata repositories, ensuring consistent data definitions, standards, and integration across enterprise systems.
  
• Design and implement automated metadata collection, synchronization, lineage tracking, and data quality processes.
  
• Implement DataOps practices, including CI/CD, automated testing, deployment automation, monitoring, and operational support.
  
• Support cloud data platform implementation, configuration, performance tuning, scalability, and reliability.
  
• Develop and manage data lake, data lakehouse, data warehouse, and cloud storage solutions.
  
• Support large-scale data migration and modernization initiatives involving OLTP and OLAP environments and cloud SaaS, PaaS, and IaaS platforms.
  
• Develop data integration solutions using APIs, event-driven architectures, and batch or streaming technologies.
  
• Implement data governance, security, metadata management, and operational best practices.
  
• Support interoperability and data exchange between cloud platforms, enterprise applications, and on-premises systems.
  
• Collaborate with solution architects, data modellers, governance specialists, and business stakeholders to operationalize enterprise data standards.
  
• Troubleshoot complex data engineering and platform issues and deliver practical, scalable solutions.
  
• Document technical designs, data mappings, workflows, deployment processes, and operational procedures.
  
• Meet project milestones and deadlines while adapting to changing business and technical priorities.
  

Requirements

• Experience designing, developing, and supporting data engineering solutions using Databricks.
  
• Practical experience with Delta Lake for reliable data storage, transformation, and management.
  
• Experience using Databricks SQL for querying, processing, and working with enterprise data.
  
• Ability to develop and optimize scalable data pipelines within modern cloud data platforms.
  

• Knowledge and practical application of Medallion Architecture for organizing data processing and storage.
  
• Understanding of layered data architecture, including Bronze, Silver, and Gold data layers.
  
• Ability to implement data ingestion, cleansing, transformation, validation, and curated data delivery processes.
  

• Hands-on experience with Microsoft Azure Data Services and Azure Data Factory (ADF).
  
• Experience developing, orchestrating, monitoring, and maintaining data integration pipelines.
  
• Knowledge of cloud data storage, data movement, transformation, scheduling, and workflow automation.
  
• Experience integrating cloud data services with enterprise applications and data platforms.
  

• Experience with Informatica or an equivalent enterprise data integration platform.
  
• Ability to develop and maintain integration workflows for data ingestion, transformation, synchronization, and provisioning.
  
• Experience integrating heterogeneous data sources and supporting enterprise data exchange requirements.
  

• Experience operationalizing common data models and metadata repositories within enterprise data environments.
  
• Ability to implement and maintain standardized data definitions, metadata structures, and data mappings.
  
• Experience automating metadata collection, synchronization, and integration with data platforms and governance processes.
  
• Ability to collaborate with data architects, modellers, and governance teams to implement and maintain enterprise data standards.
  

• Demonstrated experience designing, implementing, and supporting cloud-based data platforms and services.
  
• Experience developing scalable ETL/ELT processes, data pipelines, and integration solutions.
  
• Experience implementing DataOps practices, including CI/CD, deployment automation, automated testing, monitoring, and operational support.
  
• Knowledge of cloud-native data engineering architectures, data security, governance, and operational best practices.
  
• Experience with data platform performance optimization, troubleshooting, and scalability.
  

• Experience supporting cloud modernization, data migration, and interoperability initiatives.
  
• Knowledge of structured, semi-structured, and unstructured data ingestion, processing, and provisioning.
  
• Experience implementing data quality, metadata, lineage, and data governance capabilities.
  
• Knowledge of API-based integration, event-driven architectures, and batch or streaming data processing.
  

• Strong analytical, problem-solving, and decision-making abilities.
  
• Excellent written and verbal communication skills.
  
• Strong organizational and interpersonal skills.
  
• Ability to work collaboratively with multidisciplinary technical and business teams.
  
• Demonstrated ability to manage competing priorities, meet tight deadlines, and deliver high-quality solutions.
  

• Previous experience working with the Ontario Public Service (OPS), other provincial governments, or public-sector organizations.
  
• Experience with Microsoft Purview for metadata management, data cataloguing, lineage, and governance.
  
• Experience with Microsoft Fabric and related Azure data services.
  
• Experience with Databricks Lakehouse solutions and advanced Delta Lake capabilities.
  
• Experience implementing enterprise data lakes, data lakehouses, and data warehouses.
  
• Experience with Informatica or other enterprise integration and data management platforms.
  
• Experience with graph technologies and semantic repositories.
  
• Experience operationalizing enterprise metadata repositories and common data models.
  
• Knowledge of the Accessibility for Ontarians with Disabilities Act (AODA) and related accessibility standards.
  
• Ability to adapt quickly to changing project requirements and delivery priorities.
  

1. Data Pipelines and Integration Services: Scalable, reliable data ingestion, transformation, orchestration, and integration solutions.
   
2. Automated Metadata Management: Automated metadata collection, synchronization, and repository integration processes.
   
3. Cloud Data Platform Implementation: Configuration and implementation of cloud data services and modern data platform components.
   
4. Data Quality and Lineage: Capabilities supporting data validation, quality management, traceability, and lineage.
   
5. CI/CD and DataOps Automation: Automated deployment, testing, monitoring, and operational processes.
   
6. Data Migration and Interoperability: Services supporting cloud modernization, enterprise data migration, and cross-platform data exchange.
   
7. Cloud-Hosted Data Products: Reusable data services, curated datasets, and data products for enterprise consumption.
   
8. Metadata and Governance Integrations: Integration of metadata repositories, common data models, and governance platforms into operational data environments.