Senior Data Scientist ( AI/GenAI, LLM Evaluation & Strategy)

Job Title: Senior Data Scientist AI/GenAI, LLM Evaluation & Strategy
Location: Onsite – Bengaluru
Employment: Full Time with VDart Digital

Key Responsibilities:

• Use case definition: Translate business requirements into clearly defined AI and GenAI use cases and task types, such as classification, extraction, summarization, retrieval augmented generation (RAG) and recommendation. Judge where an agent adds value and where traditional machine learning is the better fit.
• Baseline and success metrics: Define the baseline business and operational metrics the AI solution is expected to improve, and connect model metrics directly to those business outcomes before any build begins.
• Evaluation dataset: Establish a curated evaluation dataset before implementation, including edge cases and adversarial examples, with a clear labeling approach and ground truth.
• Exploratory data analysis: Develop and execute the EDA strategy and sampling approach, covering data quality, representativeness and early detection of data leakage. Analyze the findings and recommend how they should guide solution design.
• Design of experiments: Create the design of experiments (DOE), the formal plan for testing which approach works best, including hypotheses, factors and levels, experiment metrics, evaluation methodology and decision criteria agreed before the experiment runs.
• TC2 ownership: Lead and own all data science deliverables and artifacts required for TC2 readiness and approval.
• TC3 accountability: Serve as the accountable data science lead through TC3, validating proof of concept execution against the approved DOE, evaluation criteria and acceptance thresholds, and providing the technical recommendation on whether to progress to a minimum viable product (MVP).
• Production readiness: Ensure designs handed to the delivery team are fully executable and will not break in production, and act as the point of contact for data or model issues during implementation and after go-live.

Required Qualifications:

• Experience: 7 or more years of hands-on data science experience, including personal ownership of methodology design on complex AI projects from scoping through delivery.
• LLM and agent evaluation: Proven experience evaluating LLM and agentic systems beyond accuracy, including building ground truth test sets and measuring routing and tool call correctness, groundedness, faithfulness, completeness, hallucination rates and end to end task success.
• Experiment design and statistics: Strong grounding in experiment design and statistics, including comparing models or prompt versions on small labeled datasets, accounting for run to run variability, and setting acceptance thresholds in advance.
• Machine learning and GenAI depth: Deep hands-on experience with classic machine learning (classification, regression, time series) and with GenAI, including RAG, agent frameworks such as LangChain or LangGraph, and prompt engineering.
• Production awareness: Practical understanding of MLOps and production deployment, sufficient to design solutions that a separate engineering team can implement reliably.
• Stakeholder leadership: Demonstrated ability to shape scope with business stakeholders, push back with data, explain design trade-offs, and present recommendations at approval gates without needing close direction.

Preferred Qualifications:

• Industrial domain: Experience in energy, power generation, manufacturing or other industrial settings, particularly with sensor, time series or maintenance data.
• Advanced degree: A master’s degree or PhD in data science, statistics, applied mathematics or a related quantitative field.
• Mentoring: Experience guiding or mentoring other data scientists on methodology and evaluation practice.