Python PySpark Data Engineer
Summary
Five-month contract Data Engineer working remotely for LeverageTek's Ottawa-based customer. Day to day: converting Python packages to PySpark, integrating and validating them in nightly ETL pipelines, and optimizing performance and accuracy. Core stack: Python, Apache Spark/PySpark, GitLab or GitHub.
Python and PySpark expertise define this Ottawa-based contract for a Data Engineer supporting an existing customer remotely. Convert code and optimize nightly ETL pipelines.
LeverageTek seeks a Data Engineer for a five-month contract with an Ottawa-based customer. The work requires intermediate to senior hands-on experience with Apache Spark, PySpark, and Python. You will also use GitLab or GitHub while improving existing ETL architecture, pipeline accuracy, and efficiency.
Key Responsibilities:
- Convert Python packages to PySpark
- Test, validate, and integrate packages into nightly ETL
- Identify and implement performance optimizations
- Write and validate code improvements
- Document ETL processes when time permits
Requirements:
- Intermediate to senior Apache Spark experience
- Intermediate to senior Python experience
- Experience converting and optimizing code
- GitLab or GitHub experience
- Ability to work independently remotely
Apply Python, PySpark, testing, and optimization expertise to strengthen nightly ETL pipeline performance for LeverageTek's customer.