Ground Truth Lead, GenAI

AIG is on a journey to revolutionize how insurance risk is accessed, and solutions are delivered to millions of customers globally using cutting-edge technologies like Generative AI.

Over the past 3 years we have successfully delivered highly impactful, enterprise-wide Generative AI solutions into production. Our success to date is fueling the rapid expansion of use cases across AIG, and by extension, the need for new highly skilled and experienced candidates in the GenAI space to join our team.

This is an exceptional opportunity to join a team that is shaping the future of insurance, engage with and learn from a talented team of data scientists, work on cutting edge technologies, and advance your career.

Role Summary

AIG is seeking a highly experienced Associate Director of Data Science to serve as the Right-Hand Lead for the Global Head of Model Risk Management and AI Delivery. In this high-impact role, you will bridge the gap between cutting-edge Generative AI innovation and the rigorous governance required of a global Tier-1 Insurance institution.

You will be responsible for developing, scaling, and implementing the day-to-day execution of the 1st Line of Defense (1LOD) framework, specifically driving the data strategy and automated evaluation frameworks that underpin model validation. You will lead the programmatic development of enterprise Ground Truth datasets across development and production, architect scalable "LLM-as-a-Judge" evaluation pipelines, and synthesize key oversight artifacts to ensure our agentic AI solutions are fully compliant, transparent, and ethically governed under SR 11-7 guidelines.

Additionally, as an experienced analytics leader, you will be responsible for driving complex, high-impact AI/GenAI initiatives and establishing elite Agile project management practices across the team to accelerate delivery velocity without sacrificing governance standards. When the Director is away, you can act as the primary technical, operational, and governance authority for the team.

Ground Truth Curation & "LLM-as-a-Judge" Governance (SR 11-7 Focused)

• Ground Truth Lifecycle Management: Lead the data strategy for model validation by engineering, versioning, and managing robust Ground Truth datasets across both development (dev) and production (prod) environments across multiple insurance domains.

• LLM-as-a-Judge Calibration: Architect and systematically calibrate automated "LLM-as-a-Judge" evaluation pipelines. Establish rigorous multi-criteria scoring rubrics, optimize evaluation prompts, and perform statistical alignment checks against human expert annotators.

• Statistical Integrity & Validation: Act as a primary technical validator within the 1LOD. Scrutinize the statistical foundations of both classical models and Transformer-based architectures to measure Model Drift, Perplexity, and Fairness, ensuring they meet the "Conceptual Soundness" requirements of SR 11-7.

• Advanced Observability: Lead the deployment of advanced observability tools (e.g., Fiddler AI, Arize, LangSmith) to seamlessly bridge production telemetry back to offline Ground Truth benchmarks, tracking hallucination rates and model drift in real time.

1LOD Governance & Oversight Artifact Execution

• Oversight Artifact Synthesis: Programmatically develop, maintain, and synthesize critical model validation reports, compliance scorecards, and comprehensive model cards that form the technical bedrock of AIG’s 1LOD audit trail.

• Model Risk Management (MRM) Workflow: Manage the MRM workflow, ensuring every GenAI "Agent" or pipeline has a corresponding model document detailing its statistical limits, training data bias, and performance thresholds.

• Effective Challenge Documentation: Translate Federal Reserve, OCC guidelines, and the EU AI Act into daily data science operations, ensuring that "Effective Challenges" are rigorously documented at every stage of the model lifecycle.

• Regulatory & Audit Interface: Serve as the lead technical point of contact during Internal Audit Group (IAG) and independent review challenges. Work directly with Legal, Compliance, and 2LOD teams to translate abstract regulatory requirements into actionable technical validation constraints.

Agile Practices & Delivery Excellence

• Agile Practice Leadership: Establish, champion, and mature Agile project management practices (Scrum/Kanban) specifically optimized for data science and GenAI evaluation lifecycles. Own backlog grooming, sprint planning, and daily execution to maximize delivery capacity.

• Operational Engine: Drive the production-oriented agile environment, managing cross-functional dependencies between model developers, MLOps engineers, and business stakeholders to streamline delivery and eliminate operational bottlenecks.

• Executive Backup: Serve as the technical "Right Hand" and primary backup for the Director in executive meetings, risk committees, and stakeholder briefings.

• Strategic Insights: Take on complex, ad-hoc advanced analytics projects to deliver high-impact insights and recommendations for enterprise-level AI/ML strategy.

Required Qualifications

• 12–15+ years of professional experience in Data Science, Machine Learning, or AI Delivery.

• Extensive experience within a highly regulated environment such as Insurance (preferred), Financial Services, or Healthcare.

• Proven Agile Expertise: Demonstrated experience successfully leading Agile project management methodologies and running high-performing sprints within data science, software engineering, or AI delivery teams.

• Deep Governance Familiarity: Strong working knowledge of SR 11-7 (Supervisory Guidance on Model Risk Management), OCC guidelines, or the EU AI Act, with a proven track record of taking a model from "Idea" through "Independent Validation" and defending methodologies to regulators.

• Advanced Evaluation & Stack Knowledge: Hands-on and strategic experience with LLM evaluation frameworks (e.g., DeepEval, Giskard, Ragas), vector databases, and enterprise ML Ops/Observability tools like Fiddler AI, LangChain, or weights & biases.

• Education: Master’s or PhD in a quantitative field (Computer Science, Statistics, Physics, or similar).

Preferred Qualifications

• Professional Agile or Project Management credentials (e.g., CSM, PMI-ACP, PMP) coupled with deep proficiency in Jira/Confluence.

• Hands-on experience with synthetic data generation techniques for adversarial model testing, edge-case simulation, and building robust human-in-the-loop annotation loops.

• Exceptional ability to "translate" complex AI behavioral risks, validation metrics, and data drift into clear business terms for non-technical executives.

• Demonstrated curiosity and ability to learn fast while doing.

Success Measures

• Governance & Agile Velocity: Maximizing team sprint predictability and velocity while reducing the time it takes for a model to move through the 1LOD review process with zero "high-risk" audit findings.

• Benchmark & Evaluation Integrity: Establishing 100% test coverage via production-grade Ground Truth datasets and automated evaluation suites for all active enterprise GenAI deployments.

• Operational Stability: Implementation of automated real-time monitoring and "LLM-as-a-Judge" guardrails that catch 100% of "out-of-bounds" behaviors or data drift before they impact the business.

• Team Leadership: Successful mentorship of junior staff, expanding the overall "risk-awareness" and Agile maturity of the broader Data Science organization.

At AIG, we value in-person collaboration as a vital part of our culture, which is why we ask our team members to be primarily in the office. This approach helps us work together effectively and create a supportive, connected environment for our team and clients alike.

Enjoy benefits that take care of what matters

At AIG, our people are our greatest asset. We know how important it is to protect and invest in what’s most important to you. That is why we created our Total Rewards Program, a comprehensive benefits package that extends beyond time spent at work to offer benefits focused on your health, wellbeing and financial security—as well as your professional development—to bring peace of mind to you and your family.

Reimagining insurance to make a bigger difference to the world

American International Group, Inc. (AIG) is a global leader in commercial and personal insurance solutions; we are one of the world’s most far-reaching property casualty networks. It is an exciting time to join us — across our operations, we are thinking in new and innovative ways to deliver ever-better solutions to our customers. At AIG, you can go further to support individuals, businesses, and communities, helping them to manage risk, respond to times of uncertainty and discover new potential. We invest in our largest asset, our people, through continuous learning and development, in a culture that celebrates everyone for who they are and what they want to become.

Welcome to a culture of inclusion

We’re committed to creating a culture that truly respects and celebrates each other’s talents, backgrounds, cultures, opinions and goals. We foster a culture of inclusion and belonging through learning, cultural awareness activities and Employee Resource Groups (ERGs). With global chapters, ERGs are a cornerstone for our culture of inclusion. The talent of our people is one of AIG’s greatest assets, and we are honored that our drive for positive change has been recognized by numerous recent awards and accreditations.

AIG provides equal opportunity to all qualified individuals regardless of race, color, religion, age, gender, gender expression, national origin, veteran status, disability or any other legally protected categories.

AIG is committed to working with and providing reasonable accommodations to job applicants and employees with disabilities. If you believe you need a reasonable accommodation, please send an email to candidatecare@aig.com.

#AIGPP

Functional Area: