Data Engineer
Summary
Data Engineer building scalable AWS data pipelines and ETL/ELT processes for an AI-driven consumer intelligence platform, using Python, SQL, Spark, and Claude for AI-assisted development.
Location: London - Three days in the office and two from home
Job reference: J13171
Build the data platform powering a new generation of AI-driven consumer intelligence. You’ll work at the intersection of AWS, large-scale behavioural data and agentic AI, creating scalable pipelines and data products that turn ethically sourced, consented consumer data into meaningful insights.
Joining a collaborative, scaling technology company, you’ll have the freedom to experiment with cutting-edge tools, broaden your experience and make a visible impact on the platform from day one.
The team uses Claude extensively to support development, debugging, testing and documentation. You’ll combine AI-assisted development with strong Python, SQL and Spark fundamentals to produce reliable, well-tested solutions.
What you’ll be doing
- Designing, building and optimising scalable AWS data pipelines and ETL/ELT processes
- Developing data solutions using Python, SQL and Spark
- Processing large and complex behavioural datasets
- Improving data quality, reliability, performance and monitoring
- Using Claude and other AI-assisted development tools to support delivery
- Collaborating with engineers, analysts and data scientists
- Identifying opportunities to automate and improve data workflows
- Contributing ideas as the platform and its data products continue to evolve
What we’re looking for
- Hands-on data engineering experience using Python and SQL
- Practical Spark or PySpark experience
- A good understanding of scalable ETL/ELT design and data quality
- Experience using Claude or another AI-assisted development tool, with the ability to review and validate generated code
- Strong communication skills and an interest in working within a small, collaborative team
This is an opportunity to work with cutting-edge data and AI technology, take ownership beyond one narrow area and help shape a platform as the business grows.
Applicants must already have the right to work in the UK, as sponsorship is not available for this position.