Python Pyspark Developer
Posted
Description
Pay Range: CAD 60-70/hr
Required Technical Skills
- Strong programming experience in Python 3.x, including OOP, data structures, exception handling, and reusable modules.
- Hands-on experience with PySpark, Apache Spark, Spark SQL, and Spark DataFrames.
- Strong expertise in SQL, query optimization, and relational databases.
- Experience developing ETL/ELT pipelines and large-scale data transformation workflows.
- Knowledge of distributed computing, parallel processing, and big data architecture.
- Experience with data formats such as Parquet, Avro, JSON, CSV, and ORC.
- Experience with databases such as PostgreSQL, SQL Server, Oracle, MySQL, or similar.
- Hands-on experience with cloud platforms (AWS/Azure/GCP).
- Knowledge of Git, Jenkins, Azure DevOps, GitHub Actions, or other CI/CD tools.
Preferred / Nice-to-Have Skills
- Experience with Databricks, Delta Lake, and Lakehouse architecture.
- Knowledge of Snowflake, Amazon Redshift, Azure Synapse Analytics, or BigQuery.
- Experience with Apache Kafka and real-time streaming.
- Knowledge of Apache Airflow, AWS Glue, Azure Data Factory, or similar orchestration tools.
- Experience with Docker and Kubernetes.
- Understanding of data modeling, dimensional modeling, and data warehousing concepts.
- Experience with Terraform and cloud infrastructure automation.
Skills
- Airflow
- Analytics
- Automation
- Avro
- AWS
- Aws Glue
- Azure
- Azure Data Factory
- Azure DevOps
- Azure Synapse
- Big Data
- BigQuery
- CAD
- CI/CD
- Cloud
- Data Modeling
- Data Warehousing
- Databricks
- Delta Lake
- DevOps
- Dimensional Modeling
- Distributed Computing
- Docker
- ELT
- ETL
- GCP
- Git
- GitHub
- GitHub Actions
- Jenkins
- JSON
- Kafka
- Kubernetes
- Lakehouse
- MySQL
- OOP
- Oracle
- Parquet
- PostgreSQL
- PySpark
- Python
- RDBMS
- Redshift
- Snowflake
- Spark
- SQL
- SQL Server
- Terraform
Sign in to read and join the discussion.