Big Data Engineer - Government Sector

Description

Apt Resources is seeking an experienced Big Data Engineer for a government client in Abu Dhabi. You will design and implement large-scale data solutions to support AI/ML initiatives and public sector digital transformation.

Key Responsibilities:

Data Pipeline Development:

• Build robust data pipelines using Python SQL/NoSQL and Airflow
• Develop ETL/ELT processes for structured/unstructured data
• Manage data lakes and optimize storage solutions

Data Infrastructure:

• Design efficient data models for analytics
• Implement data governance and quality frameworks
• Work with cloud-based data platforms (Azure preferred)

AI/ML Support:

• Prepare and process datasets for machine learning applications
• Collaborate with ML teams on feature engineering

Requirements

• 10-12 years of hands-on big data experience
• Expertise in:
  • Python and SQL/NoSQL databases
  • Airflow for workflow orchestration
  • ETL/ELT pipeline development
  • Cloud data platforms (Azure AWS or GCP)

Benefits

To be discussed