Data Engineer
Posted Updated
Description
The Access Group is seeking a passionate Data Engineer to design, build, and operate scalable data pipelines on Databricks, ingesting data from internal and external APIs. You will develop dimensional data models to support analytics and downstream applications, while applying modern CI/CD practices and AI-assisted development tools.
You will collaborate with product, engineering, analytics, and business teams to deliver robust end-to-end data solutions, leveraging technologies like Python, SQL,
Design, build, and operate reliable data pipelines on Databricks from ingestion to curated layers. Ingest data from internal and third-party APIs with authentication, pagination, and schema evolution. Develop and maintain scalable data models using dimensional/medallion architectures. Apply modern data engineering practices: modular development, automated testing, orchestration, CI/CD, version control. Monitor pipeline performance, quality, reliability, and cost efficiency; resolve issues proactively. Collaborate with product, engineering, analytics and business teams for end-to-end solutions. Leverage AI-assisted development tools to accelerate coding, testing, and documentation while ensuring quality. Identify opportunities for automation and continuous improvement across the data platform. Contribute to engineering standards,Documentation, peer reviews and knowledge sharing. 3-5 years of professional experience in Data Engineering. Strong hands-on experience with Databricks, including Spark/PySpark, SQL, Delta Lake, and Databricks Workflows. Experience building and supporting API-driven data ingestion pipelines. Solid understanding of dimensional modelling and layered/medallion data architectures. Experience with data engineering frameworks and tools, including orchestration, testing, CI/CD, and Git-based development practices. Strong programming skills in Python and SQL. Experience working with structured and unstructured data sources. Understanding of AI-assisted development and practical use of AI in data engineering workflows. Strong communication and stakeholder management skills. Desirable Experience with AI-assisted development tools such as Claude Code, OpenAI Codex, GitHub Copilot CLI, or Devin. Experience with NoSQL technologies and non-relational data stores. Exposure to Azure services, including Entra ID and Azure DevOps. Experience handling observability, monitoring, security, vulnerability, or infrastructure-related datasets. Familiarity with Business Intelligence platforms and reporting tools. Experience developing lightweight services or APIs using frameworks such as FastAPI. Wellbeing days - 25 days leave Health contribution Charity daysSkills
- AI
- Analytics
- API
- Authentication
- Automation
- Azure
- Azure DevOps
- Business Intelligence
- CI/CD
- Claude Code
- CLI
- Data Engineering
- Data Ingestion
- Data Pipelines
- Databricks
- Delta Lake
- DevOps
- Dimensional Modeling
- Entra ID
- FastAPI
- Git
- GitHub
- Github Copilot
- NoSQL
- Observability
- OpenAI
- PySpark
- Python
- Spark
- SQL
- Stakeholder Management
- Test Automation
- Version Control
Sign in to read and join the discussion.