Data Engineer
Posted Updated 13
views
Summary
Data Engineer building and maintaining ETL/ELT pipelines on Microsoft Azure (Data Factory, Databricks, Azure SQL), optimizing SQL performance, and implementing CI/CD with Azure DevOps. Works with data modelers, Group Office tech teams, and local IT teams in an Agile environment.
Job Responsibilities:
- Design, develop, and maintain ETL/ELT pipelines on Microsoft Azure (Azure Data Factory, Databricks, Azure SQL / Azure SQL Managed Instance).
- Optimize SQL queries and stored procedures for performance, scalability, and reliability.
- Collaborate with data modelers to understand data structures, normalization, and data quality requirements.
- Implement CI/CD pipelines and deployment strategies using Azure DevOps (Repos, Pipelines, Release Management).
- Monitor, troubleshoot, and improve data pipelines; resolve data quality issues and incidents.
- Support the Data Tech Lead and internal teams with technical discussions and development work.
- Coordinate with Group Office tech team and local IT teams (architects, business analysts, etc.).
- Participate in Agile ceremonies and use tools like JIRA, Confluence, and Kanban boards.
Job Requirements:
- Minimum 3 years of hands-on experience in ETL/ELT, Databricks, SQL, and stored procedures.
- Strong skills in SQL tuning and performance optimization.
- Experience with Azure Data Factory and Databricks.
- Familiarity with Azure SQL or Azure SQL Managed Instance.
- Working knowledge of Azure DevOps (Repos, CI/CD pipelines).
- Understanding of data modeling and normalization principles.
- Experience working in Agile environments using tools like JIRA and Confluence.
- Experience in vendor management, including coordinating with external vendors, managing deliverables, and tracking SLAs.