Data Engineer/ Senior Data Engineer - Insurance
Summary
The Data Engineer will design, develop, and maintain ETL/ELT pipelines on Microsoft Azure using tools like Azure Data Factory and Databricks. The role involves optimizing SQL performance, implementing CI/CD pipelines, and coordinating with vendors to ensure data platform reliability.
Design, develop, and maintain ETL/ELT pipelines on Microsoft Azure (Azure Data Factory, Databricks, Azure SQL / Azure SQL Managed Instance).
Optimize SQL queries and stored procedures for performance, scalability, and reliability.
Collaborate with data modelers to understand data structures, normalization, and data quality requirements.
Implement CI/CD pipelines and deployment strategies using Azure DevOps (Repos, pipelines, release management).
Monitor, troubleshoot, and improve data pipelines; address data quality issues and incidents.
Coordinate with Office tech team and other local IT teams (architects, BAs, etc.) as needed.
Participate in Agile ceremonies and use collaboration tools (JIRA, Confluence, Kanban boards).
Qualifications:- Over 2 years of hands-on experience with ETL/ELT tooling, Databricks, SQL, and stored procedures.
- Strong SQL tuning/optimization skills.
- Experience with Azure Data Factory and Databricks; familiarity with Azure SQL or SQL Managed Instance.
- Working knowledge of Azure DevOps (Repos, CI/CD pipelines).
- Understanding of data modeling and normalization.
- Experience with Agile methodologies and collaboration tools (JIRA, Confluence, Kanban).
- Vendor management: experience coordinating with external vendors, managing deliverables, and tracking SLAs; ability to engage vendors for data platform needs.