Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Go Ahead Consulting

NewBe an early applicant

Senior Data Engineer (Hadoop, PySpark & Apache Airflow) | Go Ahead Consulting | Dubai, UAE

Posted 34 views

Summary

Senior Data Engineer in Dubai taking end-to-end ownership of big data pipelines: designing and scaling Hadoop cluster architectures, writing and optimizing PySpark transformations, and orchestrating automated Airflow DAG workflows with GitHub-based CI/CD. Permanent, full-time, on-site role requiring existing UAE work authorization.

Position Summary

Driving advanced big data engineering, distributed data pipelines, Hadoop architectures, PySpark analytics, and Apache Airflow orchestration in Dubai, United Arab Emirates, Go Ahead Consulting is actively seeking an experienced Senior Data Engineer for a permanent full-time engagement. In this vital data platform and engineering role, you will take full operational, tactical, and technical ownership of designing, building, scaling, and maintaining high-performance big data processing pipelines, Hadoop clusters, PySpark transformations, and automated Airflow DAG workflows integrated with GitHub version control. Bringing deep technical proficiency in distributed computing, data warehousing, and modern data engineering frameworks, you will empower an elite international technology consulting and analytics corporation to achieve uncompromised data reliability, lightning-fast query execution, and superior business intelligence delivery. This prestigious career opportunity offers competitive remuneration, enterprise client exposure in the GCC region, and professional growth within a leading global technology recruitment and consulting corporation.

Detailed Job Description

As a Senior Data Engineer supporting Go Ahead Consulting on a professional basis for client operations in Dubai, UAE, you will take full operational, tactical, and technical ownership of big data architecture, pipeline development, workflow orchestration, and cross-functional engineering collaboration. Your day-to-day responsibilities include designing, developing, optimizing, and maintaining scalable data pipelines utilizing Hadoop, PySpark, and Apache Airflow, managing distributed data storage and processing across enterprise clusters, writing clean, efficient, and production-grade code tracked via GitHub, collaborating proactively with data scientists, software developers, and technical recruitment leadership at Go Ahead Consulting, adhering to strict corporate data governance and security standards, and managing application requirements by submitting your updated CV via direct email to along with your notice period, current salary, expected salary, and current UAE visa status in the email body or application notes.

Key Responsibilities
  • Architect, design, build, and scale robust big data processing pipelines and distributed data architectures utilizing Hadoop and PySpark.
  • Develop, schedule, monitor, and optimize complex workflow orchestration DAGs using Apache Airflow.
  • Manage version control, code reviews, branching strategies, and CI/CD integration workflows via GitHub.
  • Process, transform, and clean large-scale structured and unstructured datasets to support downstream analytics and machine learning.
  • Optimize PySpark jobs, Spark configurations, and Hadoop cluster resource utilization for maximum query performance.
  • Collaborate proactively with cross-functional data science, software engineering, and product stakeholder squads.
  • Ensure strict adherence to enterprise data security baselines, privacy regulations, and governance frameworks.
  • Maintain meticulous technical documentation, data lineage maps, pipeline architecture blueprints, and operational runbooks.
  • Participate actively in agile sprint planning, architectural design reviews, and continuous improvement engineering sessions.
  • Provide technical leadership, troubleshooting support, and escalation expertise for mission-critical data pipelines.
Required Qualifications & Skills
  • Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or a related technical discipline.
  • Minimum of 5+ years of progressive professional hands-on experience in big data engineering and distributed systems.
  • Strong, demonstrable operational proficiency in Hadoop ecosystem technologies (HDFS, Hive, YARN) and PySpark.
  • Comprehensive working knowledge and practical hands-on experience with Apache Airflow for workflow orchestration.
  • Proven expertise in collaborative software development and version control using GitHub.
  • Solid practical grasp of data warehousing concepts, SQL tuning, ETL/ELT pipeline design, and performance optimization.
  • Strong analytical problem-solving skills and effective stakeholder communication capabilities.
  • Mandatory legal work authorization and current physical presence or visa status for operations in Dubai, UAE.
Nice-to-Have Skills
  • Professional vendor or cloud certifications such as Databricks Certified Associate Developer for Apache Spark, AWS Certified Data Analytics, or Google Cloud Professional Data Engineer.
  • Prior professional working experience supporting Tier-1 global financial institutions, telecommunications providers, or enterprise consulting firms in the GCC.
  • Hands-on familiarity with cloud data warehouses (Snowflake, BigQuery, Redshift) and containerization tools (Docker, Kubernetes).
  • Experience utilizing enterprise agile tracking and IT service desk platforms (Jira, Confluence, ServiceNow).
  • Active participation in open-source big data communities, data engineering user groups, or PySpark contributor networks.

Salary/Rate: Market Competitive / Enterprise Big Data Package

Contract Duration: Permanent / Full-Time Employment

Notice Period: Required in application (Notice period, current salary, expected salary, and UAE visa status must be included)

Work Schedule: Professional Operations (Dubai, UAE)

Deadline: Open until filled (Urgent Dubai Senior Data Engineer Opening)

Skills

What Senior Data Engineering jobs ask for — and how much of it you have →
Apply

See also

Data Engineering jobs by country — openings, pay and top skills →