Data Engineer 2 - Guidewire

Remote position with the ability to travel to our NJ and NY locations up to 25% of the time.

Position Summary: Data Engineer with previous experience in the insurance domain, design, build, and maintain scalable data pipelines and models that power analytics and Generative AI initiatives. The Data Engineer will collaborate closely with analysts and engineers, to understand data needs, develop efficient solutions, and ensure the integrity, performance, and security of our data systems.

Essential Duties and Responsibilities:

• Analyze integration and system requirements by understanding business needs and designing effective data solutions, particularly for Guidewire Policy, Billing Center and Commercial P&C data domains.
• Design, develop, and optimize ELT pipelines to ingest, transform, and load data into a Delta Lakehouse platform.
• Design, Develop and maintain data models and schemas ensuring data quality and integrity.
• Build and maintain dashboards and reports delivering actionable business insights.
• Monitor pipeline and storage performance; troubleshoot and resolve data issues promptly.
• Collaborate with cross-functional teams, including analysts and business users, to deliver end-to-end insurance data solutions.
• Implement data governance, security, and compliance standards across platforms.
• Conduct root cause analysis for system failures and performance events; drive continuous improvements in enterprise data integration pipelines.
• Create and manage testing procedures (unit, scenario, end-to-end) to ensure pipeline reliability.
• Stay current with emerging technologies and recommend improvements to workflows.
• Mentor team members on data engineering tools, Guidewire data model concepts and best practices.
• Build and maintain data pipelines to process structured and unstructured data (like documents and text) for Generative AI tasks, including creating embeddings and working with vector databases to support AI search features.
• Prepare and clean large datasets to ensure high-quality inputs for training and fine-tuning Generative AI models.
• Collaborate with AI and data science teams to understand data requirements and deliver scalable solutions that support model training and inference.
• Implement processes to enrich data with metadata and context to improve AI model accuracy and relevance.
• Optimize data storage and retrieval methods to support fast, low-latency responses for AI-powered applications.
• Monitor data workflows for Generative AI projects, troubleshooting issues, and ensure continuous pipeline performance.