Senior Data Engineer

Overview

Our team is looking for a high-performing Data Engineer who will be responsible for the design, development, implementation, and on-going support of a data driven software platform. You enjoy and thrive in a fast-paced environment on multiple projects simultaneously, including both enhancements as well as new project development. The candidate must be a self-starter with a sense of urgency and a commitment to quality and professionalism.

Responsibilities

• Analyze ambiguous business requirements and partner with product team to provide a strategic solution


• Collaborate with frontend engineers to design or modify schema for optimal software performance


• Advanced database administration and development including stored procedures, user defined functions, triggers and ETL packages as well as security and roles


• Uphold quality standards and create monitoring tools to ensure high fidelity data


• Optimization and tuning of existing stored procedures to improve performance


• Troubleshoot database issues, identify root causes, and implement optimal solutions


• Extract, transform, and load data from multiple data sources using languages and platforms such as Python and Snowflake/Databricks


• Design, Build, Test, Debug, Monitor, and Troubleshoot ETL and software processes in Python


• Recommend solutions to existing or anticipated issues


• Serve as an escalation point for any issues that may arise


• Design testing requirements and prepare test plans and test modules


• Ability to follow implementation standards, develop documentation and transfer of knowledge


• Design, develop, and implement scalable data engineering solutions using Databricks, Apache Spark and distributed systems (e.g., AWS, Azure, GCP).

Qualifications

Required Experience (5+ years):

• 5+ years’ experience with ETL and databases


• 3+ years’ experience with PySpark


• Experience in Databricks and experience with Spark for big data processing.


• Familiarity with cloud technologies such as Amazon Web Services (Redshift, S3, Glue, Lambda)


• Advanced SQL programming skills (stored procedures, functions, etc.)


• Experience with marketing data and digital media


• Experience with complex processing logic against large data sets


• Excellent data modelling skills and experience working with multiple datasets


• Possession of strong communication and presentation skills as well as an analytical mindset


• Good problem solving and testing skills


• Familiarity with CI/CD tools and Git-based development workflows.


• Strong organizational skills & attention to detail


• Possess a desire to work for a fast-paced, results-based company


• Experience managing multiple projects simultaneously


• Self-driven/entrepreneurial spirit with the ability to think outside of the box


• Bachelor’s Degree in Computer Sciences, Information Technology, or equivalent


• English B2+

Preferred Experience

• Experience with Snowflake/Databricks


• Experience with Terraform


• Experience with Business Intelligence software (Tableau)


• Experience and thorough understanding of a variety of APIs