Databricks Spark Data Engineer Opening
Posted 7
views
Summary
A senior data engineer role at Pacer Group in Toronto focused on migrating legacy Cloudera systems to a Databricks Lakehouse architecture. Day to day involves building and optimizing high-volume PySpark and Spark SQL ELT pipelines, designing scalable Delta Lake/data warehouse architectures, and working with Finance and Risk data models for reporting.
Take on a vital Senior Data Engineer role at Pacer Group, specializing in Databricks and Apache Spark. Spearhead the migration of legacy systems to a robust Data Lakehouse architecture.
This position involves hands-on work to modernize Cloudera technologies, including transforming existing platforms to Databricks Lakehouse. You'll focus on designing scalable architectures and crafting high-volume PySpark and Spark SQL ELT pipelines while optimizing for performance and cost efficiency.
Key Responsibilities: • Implement Databricks Lakehouse from legacy Cloudera systems • Create scalable architectures for Data Warehouses and Delta Lakes • Develop and optimize PySpark/Spark SQL ELT data pipelines • Enhance performance and cost-effectiveness of workloads • Work with complex Finance and Risk data models for reporting
Requirements: • Proven hands-on experience with Databricks and Apache Spark • Expertise in Cloudera ecosystems (CDH/CDP, Hive, HBase) • Knowledge of Finance and Risk data frameworks • Ability to design efficient data architectures • Strong skills in PySpark and Spark SQL
Drive innovation at Pacer Group by utilizing your Databricks and Apache Spark expertise in a pivotal engineering role.
This position involves hands-on work to modernize Cloudera technologies, including transforming existing platforms to Databricks Lakehouse. You'll focus on designing scalable architectures and crafting high-volume PySpark and Spark SQL ELT pipelines while optimizing for performance and cost efficiency.
Key Responsibilities: • Implement Databricks Lakehouse from legacy Cloudera systems • Create scalable architectures for Data Warehouses and Delta Lakes • Develop and optimize PySpark/Spark SQL ELT data pipelines • Enhance performance and cost-effectiveness of workloads • Work with complex Finance and Risk data models for reporting
Requirements: • Proven hands-on experience with Databricks and Apache Spark • Expertise in Cloudera ecosystems (CDH/CDP, Hive, HBase) • Knowledge of Finance and Risk data frameworks • Ability to design efficient data architectures • Strong skills in PySpark and Spark SQL
Drive innovation at Pacer Group by utilizing your Databricks and Apache Spark expertise in a pivotal engineering role.