Data Engineer (Analytics Engineering)
Description
Key Responsibilities
Engage with business stakeholders to understand requirements and translate them into data models, datasets and pipelines.
Design and develop data models and transformation pipelines to integrate, clean, harmonize and curate data from multiple sources.
Build and maintain scalable and reliable data pipelines using Databricks, Apache Spark and SQL.
Design curated datasets optimized for reporting, analytics and data science use cases.
Implement data quality checks, validation and reconciliation to ensure data accuracy and consistency.
Monitor and troubleshoot data pipeline performance and operational issues.
Document data models, datasets, transformations and pipeline dependencies.
Collaborate with data engineers, analysts and data scientists to deliver solutions aligned with business requirements.
Must-Have Requirements
Strong proficiency in SQL, relational databases and data warehousing.
Strong data modelling skills, including dimensional modelling and analytical dataset design.
Proficiency in Python for data processing and pipeline development.
Hands-on experience with Databricks, Apache Spark and Delta Lake.
Experience building and maintaining data ingestion and transformation pipelines.
Ability to engage business stakeholders and translate requirements into technical data solutions.
Good to Have
Experience developing ETL/ELT pipelines and data transformation workflows.
Experience with AWS data services and cloud-based data platforms.
Familiarity with Git, CI/CD and data pipeline orchestration.
Experience with Databricks Workflows, Unity Catalog and Delta Lake.
Experience designing data products for BI and self-service analytics.
Knowledge of data governance, access control and data quality practices.
Experience in government or other highly regulated environments.
Degree in Computer Science, Information Technology, Data Engineering or a related discipline.
Skills
Sign in to read and join the discussion.