Senior Data Engineer - AI-Driven Data Platform
Posted Updated
Description
Cover Genius in Vancouver, BC is seeking a Senior Data Engineer to own the data pipelines and infrastructure powering analytics and ML workloads. You will design and operate scalable Airflow-based workflows, write production-grade Python services, and manage CI/CD across cloud environments.
You will work with BigQuery, Dataflow and Pub/Sub on GCP, implement governance and security controls, and collaborate with ML/Analytics teams to deliver reliable batch and real-time data products.
Design, build and own Airflow orchestration platform with scalable DAGs and reusable components. Engineer scalable data pipelines and platform services in Python with best practices like modular design, code review and testing. Own CI/CD for the data platform across environments. Operate and optimize GCP data stack (BigQuery, Dataflow, Pub/Sub) for performance and cost efficiency. Build observability and alerting to meet data availability and freshness KPIs; lead RCA and preventive fixes. Partner with ML/Analytics teams to deliver batch and real-time pipelines and self-service tooling. Implement data governance and security measures in line with privacy standards. 6+ years of experience as a Data Engineer, Platform Engineer or Software Engineer working on data-intensive systems. Hands-on experience building and operating Airflow in production with DAGs, frameworks and reusable components. Strong Python and software engineering skills with production-grade code, testing and modular design. Deep hands-on experience with Kubernetes (GKE) and infrastructure as code using Terraform. Experience designing and maintaining CI/CD pipelines and working with cloud data warehouses (BigQuery, Snowflake, Redshift) with solid SQL knowledge. Degree in CS/SE/Data Science or related STEM field, or equivalent practical experience. Hybrid work environment Competitive compensationSkills
Sign in to read and join the discussion.