Architect Lead - Data engineering

Job Description

We are looking for an experienced Architect Lead – Data Engineering to lead the design and implementation of enterprise-scale data platforms, data pipelines and cloud-based data solutions.

Key Responsibilities:

• Lead the architecture and design of scalable data engineering and data platform solutions.
• Design and develop high-volume data pipelines using Apache Spark and PySpark.
• Implement data warehousing and analytics solutions using Snowflake.
• Design workflow orchestration using Apache Airflow.
• Develop modern data lake/lakehouse solutions using Apache Iceberg.
• Design real-time and event-driven data pipelines using Apache Kafka.
• Develop ETL/ELT solutions using DBT and enterprise ETL frameworks.
• Work with GBI ETL Framework and ensure integration across enterprise data platforms.
• Leverage cloud computing technologies to build secure, scalable and highly available data solutions.
• Provide technical leadership, architecture guidance and mentoring to data engineering teams.
• Drive performance tuning, data quality, scalability and reliability of data platforms.
• Collaborate with business and technology stakeholders to translate requirements into technical solutions.
• Explore and implement appropriate Artificial Intelligence capabilities within data platforms.

Requirements:

• Minimum 10 years of experience in data engineering, big data or data architecture.
• Strong hands-on experience with Spark, PySpark, Snowflake, Airflow, Kafka and Iceberg.
• Strong experience in ETL/ELT frameworks and DBT.
• Experience with cloud-based data platforms and modern data architecture.
• Experience designing large-scale data pipelines and lakehouse architectures.
• Exposure to Artificial Intelligence / AI-based data solutions.
• Strong analytical, problem-solving, communication and technical leadership skills.
• Degree in Computer Science, Information Technology, Engineering or a related discipline.