Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Black Pearl Consult

Data Engineer (PySpark)

Posted Updated

Description

Black Pearl Consult in Abu Dhabi invites an experienced Data Engineer (PySpark) to design, build, optimize, and maintain scalable data pipelines for production environments. You will develop ingestion, transformation, cleansing, and validation processes using Python and PySpark.

The role requires hands-on Spark performance tuning, Airflow/Jenkins automation, CI/CD with Git, and close collaboration with cross-functional teams to deploy robust data solutions.

Design, develop, and maintain robust data pipelines using Python and PySpark. Ingest, transform, cleanse, and validate data across datasets. Perform Exploratory Data Analysis to identify quality issues. Implement feature engineering pipelines for analytics. Optimize Spark jobs for performance, scalability, and cost. Automate workflows with Airflow and Jenkins. Hands-on PySpark data pipelines in production environments. Strong Python coding with clean, testable, maintainable code. Experience with Spark performance tuning and data quality checks.

Skills

Apply

See also

Data Engineering jobs by country — openings, pay and top skills →