Senior Data Engineer
Summary
Designs, builds and maintains cloud data platforms and ETL/ELT pipelines for government, defence and regulated-sector clients, combining data engineering, Python/Java development and DevOps. Core stack spans AWS (Glue, Lambda, Athena, Redshift, S3) and Azure (Data Factory, Synapse), plus PySpark and SQL.
In this role you will design, build and maintain modern cloud-based data platforms to unlock data value for government, defence and regulated sectors. You’ll work with cross-functional teams to deliver reliable data pipelines and analytics capabilities that support critical services. The role combines data engineering, software development and cloud technologies to solve real-world problems, with opportunities to grow technical and leadership skills. You will contribute to scalable, quality-driven data solutions that enable informed decisions and measurable outcomes. This is a mission-driven opportunity to advance data capabilities across diverse client environments.
Pay / Benefits- flexible working arrangements
- development and certifications
- mentoring and knowledge sharing
- exposure to diverse projects across industries
- pension
- holiday allowance
- Design, build and maintain ETL/ELT data pipelines using AWS or Azure cloud-native services
- Develop and support data integration components and Java-based microservices where required
- Prepare, transform and validate structured and semi-structured data (parquet, avro, JSON)
- Write SQL to query and optimise data on cloud analytical platforms (Athena, Redshift, Synapse)
- Build scalable data processing solutions using PySpark, Python and cloud orchestration
- Implement data quality checks, monitoring and logging for trusted data delivery
- Support performance tuning and optimisation of pipelines, SQL workloads and integration services
- Collaborate with data architects, analysts, software engineers and client stakeholders to deliver solutions
- Support DevOps practices with source control, automated testing and CI/CD
- Communicate progress, risks and technical decisions to technical and non-technical stakeholders
- AWS Glue, Lambda, Step Functions or Azure Data Factory, Synapse Pipelines, Azure Functions and Logic Apps
- PySpark
- SQL (Athena, Redshift, Azure Synapse SQL)
- AWS S3 or Azure Data Lake Storage Gen2
- Python and/or Java
- ETL/ELT data pipeline development
- Data quality, validation and monitoring
- Git, CI/CD and Agile delivery practices
- Experience being most Senior of Lead within a project
- Eligibility for SC Clearance
- Collaboración y trabajo en equipo
- Comunicación eficaz de progresos y riesgos
- Colaboración con múltiples partes interesadas
- PySpark
- SQL (Athena, Redshift, Synapse)
- AWS Glue, Lambda, Step Functions