Data Engineer

Abacus | Data Engineer

Who We Are

At Abacus, we specialize in technology, outsourcing, and people solutions, bringing nearly 40 years of experience shaped alongside the organizations we support. As a global professional services leader, our focus remains on providing bespoke solutions that enable organizations to create the future of business and embrace change for sustainable growth, powered by a global team of over 5,000 people.

What We Do

Across digital transformation, emerging technologies, SAP enterprise solutions, outsourcing, and human capital, we support organizations in building environments that operate effectively and scale with the business. Today, this extends to more than 1,500 enterprise clients, where the focus remains on creating systems that are not only functional, but ready to adapt and evolve over time.

Position Overview

Role Summary

Designs, builds, and maintains scalable data pipelines and platforms that enable reliable reporting, analytics, and AI use cases. Responsible for integrating data from multiple sources, developing ETL/ELT workflows, optimising data storage and processing, and ensuring data quality, security, lineage, and availability. Works closely with data architects, analysts, data scientists, application teams, Product Owners, DevOps, and business stakeholders to deliver trusted, governed, and reusable data products aligned with business and technology objectives.

Responsibilities

• Design, build, test, deploy, and maintain scalable batch and real-time data pipelines
• Develop and optimise ETL/ELT workflows that integrate structured, semi-structured, and unstructured data from multiple sources
• Build and maintain data warehouses, data lakes, lakehouses, data marts, and reusable data products
• Collaborate with data architects, analysts, data scientists, application teams, Product Owners, and business stakeholders to translate requirements into fit-for-purpose data solutions
• Implement data quality controls, validation rules, metadata management, lineage, and observability across data pipelines
• Create and maintain efficient data models, schemas, transformation logic, and technical documentation
• Optimise data processing, storage, query performance, scalability, and cloud consumption costs
• Apply security, privacy, access-control, retention, and data-governance requirements throughout the data lifecycle
• Use version control, automated testing, CI/CD, and infrastructure-as-code practices to support reliable data delivery
• Monitor production data workflows, troubleshoot failures, perform root cause analysis, and improve operational resilience
• Support sprint planning, estimation, technical design reviews, dependency management, and delivery reporting
• Develop reusable frameworks, connectors, standards, and accelerators that improve data engineering efficiency
• Provide trusted, timely, and accessible data for business intelligence, regulatory reporting, analytics, and machine-learning initiatives
• Evaluate and recommend appropriate data technologies, tools, and engineering practices

Qualification & Experience

• Bachelor’s degree (BA/BS) in Computer Science, Information Systems, Data Engineering, Software Engineering, or a related field preferred
• At least 5–7 years of experience in data engineering, data integration, database development, or related environments
• Strong practical experience with SQL and a programming language such as Python, Java, or Scala
• Experience designing and implementing ETL/ELT pipelines, workflow orchestration, and data integration solutionsExperience with relational and non-relational databases, data warehouses, data lakes, or lakehouse architectures
• Hands‑on experience with a major cloud platform and its data services, preferably Microsoft Azure, AWS, or Google Cloud
• Experience with distributed processing, big‑data, or streaming technologies such as Spark, Databricks, Kafka, or equivalent tools
• Understanding of dimensional modelling, data governance, data security, data quality, metadata, and lineage
• Experience with Git, automated testing, CI/CD, DevOps practices, and Agile delivery models
• Relevant cloud or data‑platform certifications advantageous

Preferred Skills

Abacus | Enabling Organisations to Create the Future of Business

• Advanced SQL and data‑query optimisation
• Python, Java, Scala, or equivalent data‑programming capability
• ETL/ELT development and workflow orchestration
• Data modelling, warehousing, lakehouse, and database design
• Cloud data engineering and platform technologies
• Batch processing, distributed computing, and event‑streaming concepts
• Data quality, governance, security, metadata, lineage, and observability
• CI/CD, automated testing, version control, and infrastructure‑as‑code practices
• Performance tuning, reliability engineering, and cloud‑cost optimisation
• Analytical thinking, troubleshooting, and root cause analysis
• Understanding of analytics, business intelligence, and machine‑learning data requirements
• Strong stakeholder engagement and cross‑functional collaboration
• Ability to work independently and across multiple agile squads
• Good written, verbal, and interpersonal communication skills
• Good organisational, multi‑tasking, and time‑management skills

WHAT WE OFFER

• Competitive compensation package commensurate with experience.
• Exposure to large‑scale, enterprise‑grade platform and API projects.
• A collaborative, innovation‑driven, and growth‑oriented work environment.
• Continuous learning opportunities and access to professional development resources.
• Opportunity to work with a globally recognised professional services leader.