Data Engineer (PySpark)

We are seeking an experienced Data Engineer (PySpark) to design, build, optimize, and maintain scalable data pipelines for production environments. The role requires strong hands-on experience in big data processing, pipeline optimization, and deployment using modern data engineering tools and frameworks.

Key Responsibilities

• Design, develop, and maintain robust, scalable data pipelines using Python and PySpark
  

• Perform data ingestion, transformation, cleansing, and validation across structured and unstructured datasets
  

• Conduct Exploratory Data Analysis (EDA) to identify data patterns, anomalies, and quality issues
  

• Apply data imputation techniques, data linking, and cleansing to ensure high data quality
  

• Implement feature engineering pipelines to support analytics and downstream use cases
  

• Optimize Spark jobs for performance, scalability, and cost efficiency
  

• Deploy and tune production-grade data pipelines, ensuring reliability and performance
  

• Automate workflows using Apache Airflow and/or Jenkins
  

• Collaborate with cross-functional teams to integrate data solutions into production systems
  

• Write and maintain unit tests to ensure code quality and reliability
  

• Manage source code, CI/CD, and deployments using Git, GitHub, and GitHub Actions
  

Requirements

Required Technical Skills

• Strong proficiency in Python
  

• Extensive hands-on experience with Apache Spark (PySpark)
  

• Experience working with Jupyter Notebooks
  

• Strong knowledge of SQL and NoSQL databases
  

• Proven experience with Git for version control and CI/CD
  

• Hands-on experience with Apache Airflow and/or Jenkins for scheduling and automation
  

• Solid understanding of data engineering best practices in production environments
  

• Demonstrated experience in Spark performance tuning and optimization
  

• Ability to write clean, testable, and maintainable Python code
  

Mandatory Requirement

• Previous production experience is a MUST, specifically in deploying, tuning, and maintaining data pipelines in production environments
  

Preferred Qualifications

• Experience working in high-volume or big data environments
  

• Strong problem-solving and analytical skills
  

• Ability to work independently in a fast-paced environment
  

Why Join?

• Competitive salary package
  

• Opportunity to work on production-scale data platforms
  

• Exposure to modern data engineering tools and practices
  

• Dubai-based role with a dynamic and collaborative work environment