Senior Data Engineer
Summary
The Senior Data Engineer will lead a team in developing scalable data pipelines, managing data warehouses, and building data APIs using cloud platforms like GCP, AWS, or Azure. The role involves technical leadership, stakeholder management, and implementing data governance and transformation logic for AI-driven projects.
We are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team. This individual will also take on the role as a Delivery Lead in data engineering projects, leading a small team of engineers, and ensuring seamless project execution. The ideal candidate will have a strong background in data engineering, cloud technologies, project delivery, and stakeholder management.
Key Responsibilities
- Data Pipeline Development: Design and implement scalable data integration pipelines, ensuring high performance and reliability.
- Data Modeling & Transformation: Develop robust data models and transformation logic to support data analysis and business needs.
- Data Warehouse & Governance: Set up and manage data warehouses, implement data governance controls, and ensure compliance with organisational standards.
- Dashboard Development: Create dashboards that provide actionable insights, helping stakeholders make informed decisions.
- API Development: Build scalable data APIs to facilitate data access and integration across systems.
- Technical Leadership: Lead a team of data engineers, providing technical guidance and ensuring best practices in data engineering and project execution.
- Stakeholder Management: Collaborate and communicate effectively with internal and external stakeholders to gather requirements, manage expectations, and ensure alignment with project goals.
Requirements
- Experience:
- At least 5 years of relevant experience in data and AI projects.
- Minimum 3 years of experience working with cloud platforms (preferably GCP; AWS or Azure is acceptable).
- Technical Skills:
- Cloud Data Platforms: Proficient in working with cloud-based data tools and platforms, including GCP BigQuery, Snowflake, Azure Synapse, AWS Redshift, Databricks, Fivetran, Confluent, and MongoDB Atlas.
- Big Data & Open Source Expertise: Leverage open-source big data technologies such as Spark, Hadoop, Apache Beam, Kafka, dbt, and Airflow to orchestrate, process, and analyse large datasets efficiently.
- Programming Skills: Strong programming abilities in Python, SQL, and NoSQL databases, with a focus on data transformation, processing, and querying.
- API & Microservices Development: Experience in designing and building scalable microservices and data APIs (considered a bonus).
- Generative AI Technologies: Familiarity with Generative AI tools and frameworks, with the ability to integrate them into data solutions (considered a bonus).
- Data Management & Governance: Good knowledge of data management practices and data governance frameworks, including implementing policies, ensuring data quality, and maintaining compliance (considered a bonus).
- Communication & Management Skills:
- Proven experience leading a team of data engineers and demonstrated ability to manage projects and ensure successful delivery.
- Excellent verbal and written communication skills, with the ability to convey complex technical concepts to both technical and non-technical audiences.
- Strong stakeholder management skills to navigate expectations and drive alignment across teams.
Preferred Qualifications
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- Certification in cloud platforms such as Google Cloud Professional Data Engineer, AWS Certified Big Data, or Azure Data Engineer Associate.
What We Offer
- Competitive salary and benefits package.
- Opportunities to work on cutting-edge data and AI projects on various cloud platforms.
- A collaborative and innovative work environment.
- Career growth and learning opportunities.