Senior AI / ML Engineer

We are looking for a Senior AI/ML Engineer to own the end-to-end LLM, retrieval, evaluation, and agentic AI layer of our healthcare AI platform. You will build production-grade AI systems that automate workflows such as medical coding, claim edits, denials triage, appeal generation, and payer-rule reasoning. This is a hands-on production engineering role, not a research role. You will work on systems where accuracy, reliability, latency, cost, auditability, and explainability matter.

The candidate will have responsibilities across the following functions:

Self-Hosted LLM Infrastructure:

• Deploy, fine-tune, and operate open-source models such as Llama, Qwen, MedGemma, and their successors.
• Build and operate high-throughput inference systems using vLLM, SGLang, TensorRT-LLM, or equivalent technologies.
• Optimise inference for throughput, tail latency, batching, KV-cache utilisation, GPU utilisation, and cost.
• Own fine-tuning workflows including SFT, LoRA, QLoRA, and DPO.
• Work with clinical notes, claims, payer policies, and other domain-specific datasets.
• Make measurable build-vs-buy and self-hosted-vs-managed model decisions.

Knowledge Graphs and Retrieval:

• Design knowledge graphs representing ICD-10-CM, CPT, HCPCS, HCC, NCCI edits, LCD/NCD policies, and payer-specific rules.
• Build embedding-based retrieval systems across clinical notes, claims, denial reasons, and payer-policy documents.
• Work on chunking, embeddings, hybrid search, reranking, and retrieval quality.
• Combine knowledge-graph traversal with dense retrieval to produce grounded and auditable AI outputs.
• Own ingestion, versioning, and quality of knowledge sources.

LLM Evaluation and Monitoring:

• Build continuous evaluation pipelines for models, prompts, retrieval systems, and knowledge graphs.
• Develop offline evaluation suites using human-validated ground truth.
• Use LLM-as-a-judge where appropriate and calibrate it against human evaluation.
• Monitor hallucinations, model drift, regressions, accuracy, and production quality.
• Implement shadow-mode rollouts and cohort-level evaluation.
• Track business and engineering metrics including coding accuracy, clean-claim rate, denial impact, cost per chart, and end-to-end latency.

LLM Systems and RAG:

• Design prompts and context pipelines for medical coding, claim edits, denial classification, appeal generation, and payer-rule reasoning.
• Implement structured outputs using JSON, function calling, and constrained decoding.
• Build RAG systems grounded in medical coding standards and payer policies.
• Treat prompts as a versioned and evaluated layer rather than the primary source of system reliability.

Agentic AI and MCP:

• Build MCP servers and internal AI tools for code lookup, NCCI rule checks, payer logic, eligibility, and denial classification.
• Design multi-step agentic workflows with audit trails and human-in-the-loop checkpoints.
• Establish clear boundaries between deterministic systems and LLM-based reasoning.
• Build reliable AI workflows that can operate within real healthcare and RCM processes.

Requirements:

• 5+ years of experience in AI/ML engineering.
• At least 6+ months of hands-on production experience with LLM systems.
• Experience deploying and operating self-hosted LLMs using vLLM, SGLang, TensorRT-LLM, or equivalent.
• Strong experience building RAG, embedding-based retrieval, and/or knowledge-graph-based systems.
• Experience building or owning LLM evaluation infrastructure, including offline benchmarks, regression detection, and production monitoring.
• Strong Python, PyTorch, and Hugging Face skills.
• Experience owning production systems, including monitoring, troubleshooting, incidents, performance, and reliability.

Good to Have:

• Hands-on experience with SFT, LoRA, QLoRA, or DPO.
• Experience with Neo4j, ArangoDB, or other graph databases.
• Experience with vector databases, BM25 hybrid search, and reranking.
• Familiarity with Langfuse, LangSmith, Arize, Braintrust, or similar observability platforms.
• Experience in healthcare, RCM, medical coding, claims, insurance, or other regulated domains.
• Experience with MCP or similar tool-orchestration frameworks.
• Strong understanding of LLM evaluation and prompt engineering.
• Experience building agentic AI systems.