Python PySpark Data Engineer
Summary
Lancesoft is hiring a Python PySpark Data Engineer in Montréal to build reusable Python modules and large-scale ETL/ELT pipelines, processing data with PySpark, Spark DataFrames, and Spark SQL while optimizing queries over relational databases and using CI/CD on AWS, Azure, or GCP.
Build scalable data solutions with Python 3.x, PySpark, and SQL while working across large-scale transformation workflows. Apply distributed computing and cloud platform experience to modern data engineering.
This Python PySpark Data Engineer position focuses on reusable Python modules, Spark DataFrames, and Spark SQL. You will develop ETL and ELT pipelines, optimize queries, and work with relational databases. The role also involves data formats, CI/CD tools, and cloud platforms including AWS, Azure, or GCP.
Key Responsibilities:
- Develop reusable Python modules using object-oriented programming
- Build ETL and ELT pipelines for large-scale transformations
- Process data with PySpark, Apache Spark, and Spark SQL
- Optimize SQL queries across relational databases
- Work with distributed computing and parallel processing
Requirements:
- Strong Python 3.x programming experience
- Hands-on PySpark, Spark DataFrames, and Apache Spark experience
- Strong SQL and relational database expertise
- Experience with AWS, Azure, or GCP
- Knowledge of Git, Jenkins, or CI/CD tools
Apply Python, PySpark, SQL, and cloud expertise to deliver scalable data transformation and pipeline solutions.
Skills
Sign in to read and join the discussion.