Data Engineering Lead
Summary
Leads data engineering efforts building fault-tolerant, highly available batch/distributed and real-time data pipelines, onsite in Calgary (possible 3-month contract-to-hire). Day to day involves tuning distributed query engines over billions of rows, leading complex implementations, and driving root cause analysis.
Data Engineering Lead
Possible 3 Month CTH
Role Name: Data Eng Lead_SOW176
Work Site: Calgary, CA Onsite
Start Date: Immediate Availability
Role Summary
We are seeking a Data Engineering Lead with deep expertise in data engineering and distributed data platforms. The role will focus on building and supporting highly available data capabilities, including both batch/distributed and real-time data pipelines.
The successful candidate will be responsible for developing fault-tolerant and adaptive data pipelines, leading complex technical implementations, optimizing distributed queries, and driving permanent resolutions to software and business process issues.
Key Responsibilities
Demonstrate deep knowledge of the data engineering domain to build and support:
Batch and distributed data processing
Real-time data pipelines
Highly available data platforms
Data and technology capabilities
Build fault-tolerant, self-healing, adaptive, and highly accurate data computational pipelines.
Provide technical consultation and lead the implementation of complex data engineering programs.
Develop and maintain comprehensive documentation for assigned systems and projects.
Tune queries processing billions of rows of data within distributed query engines.
Perform detailed root cause analysis (RCA) to identify permanent resolutions to software and business process issues.
Support ongoing optimization, reliability, and performance improvements across data engineering capabilities.
Required Skills & Experience
Deep expertise in data engineering and distributed data processing.
Experience designing and supporting batch, distributed, and real-time data pipelines.
Strong understanding of highly available and fault-tolerant data architectures.
Experience building self-healing and adaptive data pipelines.
Strong experience with distributed query engines and large-scale data processing.
Proven ability to optimize queries across datasets containing billions of rows.
Strong technical leadership and consultation skills.
Experience leading complex technical implementations.
Strong troubleshooting and root cause analysis capabilities.
Excellent documentation and communication skills.