Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Insight Global

NewBe an early applicant

Lead AI/ML Engineer

Posted

Description

Lead AI/ML Engineer

IG Labs · AI Platform Engineering


About the Role:

You are a senior individual contributor on the AI/ML engineering team, responsible for how capable and reliable our agents actually are. That means owning the systems that make an agent smart — long-term memory, advanced retrieval, and the model adaptation (fine-tuning and reinforcement loops) behind them — while holding the evaluation bar that decides what ships. As a Staff engineer you choose the right approach for each problem, own the hardest ones end-to-end, and raise the bar for the engineers around you. The work is empirical and equal parts R&D and software engineering.


Key Responsibilities:

● Set technical direction for agent intelligence — for each agent-quality problem, decide whether the right lever is model adaptation, long-term memory, retrieval, or context engineering, and defend the call on evals and cost. In this landscape the naive move (just fine-tune) is usually wrong; the sophisticated one is knowing when.

● Own the long-term memory systems — episodic, semantic, and procedural memory; consolidation, forgetting, and retrieval of memory; and working-memory / context management so agents stay coherent across long, multi-session tasks.

● Advance the retrieval stack — agentic RAG, hybrid retrieval with reranking, late-interaction and contextual retrieval, query decomposition, and knowledge-graph-backed retrieval — every choice pushed on real evaluations.

● Own model adaptation where it pays — fine-tuning (SFT / LoRA / QLoRA), preference and reinforcement-learning loops (reward modeling, RLHF / RLAIF / RLVR), and distillation of smaller specialized models — and the judgment to know when not to train.

● Engineer context end-to-end — govern what enters the model's window at each step: retrieved knowledge, memory, tool results, and compaction. This is the discipline that most determines agent quality.

● Own the evaluation bar — trajectory-level and outcome evals across memory, retrieval, and models; regression suites that gate what ships; offline / online alignment; and the human-in-the loop labeling behind ground truth.

● Make it production-grade — instrument agents for observability (tracing, behavior and cost/latency drift), drive reliability, and own the quality / cost / latency tradeoff as agents make many model calls per task.

● Push the R&D frontier — evaluate and de-risk new methods (memory architectures, retrieval techniques, RL algorithms, scalable oversight) and turn the ones that pay off into production capabilities of the Agentic SDK.

● Own model- and agent-behavior safety — red-team for prompt injection, jailbreaks, memory poisoning, and untrusted-content handling; enforce per-tenant isolation across the memory, retrieval, and model layers; and hold the data-rights boundaries that govern training.

● Multiply the team — set the patterns others build on, review designs and eval methodology, mentor Senior and AI/ML engineers, and be the person others bring hard model-quality problems to.


Required Qualifications:

● 8+ years of software / ML engineering, including several years building LLM-powered agentic systems in production with end-to-end ownership from design through operation.

● Deep, hands-on strength in at least two of: model adaptation (fine-tuning with LoRA / QLoRA, preference / RL methods, reward modeling); long-term memory and context management for agents; advanced retrieval (hybrid + reranking, agentic / contextual RAG, query understanding); and evaluation methodology for models and agents — with working competence across the rest.

● Judgment about which lever fits — can take an agent-quality problem and decide, with evidence, whether to fine-tune, build memory or retrieval, or re-engineer context, and defend it on evals and cost.

● Rigorous evaluation practice — has designed eval methodology, including trajectory- or outcome level evals for agents, that gates what ships and caught regressions informal testing missed.

● Strong Python and systems engineering — builds the pipelines and services, not just the models; comfortable with async code, data pipelines, and integrating into a production SDK.

● Fluent in the modern stack — PyTorch and the HuggingFace ecosystem on the model side; vector / hybrid retrieval and reranking on the systems side; and the observability tooling to trace and debug agent behavior.

● Can drive ambiguous, cross-cutting model-quality problems to production without the scope being fully defined upfront.

● A track record as a force multiplier — technical leadership, design and code review, and mentoring other engineers.


Preferred Qualifications:

● Production long-term memory systems — episodic / semantic / procedural memory, consolidation and forgetting, and memory retrieval at scale.

● Reinforcement-learning depth — has run policy optimization (PPO, GRPO, DPO-family, RLHF / RLAIF, or RLVR) and reward modeling end-to-end, and understands reward hacking and training instability.

● Advanced retrieval depth — late-interaction (ColBERT), GraphRAG or knowledge-graph retrieval, contextual retrieval, or RAG systems tuned on real evals.

● Has built fine-tuning pipelines used by other engineers — reproducible training with data versioning, config management, checkpointing, and automated eval gates.

● Multi-domain adaptation — per-domain adapters, model merging (SLERP / TIES / DARE), routing, and small specialized models.

● Agent security — defending against prompt injection, memory poisoning, and untrusted-content handling in tool- and memory-enabled agents.

● Cost / latency optimization for multi-call agent workloads — caching, model routing, and high throughput serving (e.g., vLLM with adapter routing).

● Synthetic data and scalable oversight — self-instruct / Evol-Instruct, distillation, LLM-as-judge, constitutional AI / RLAIF.

● Open-source contributions to relevant projects — TRL, HuggingFace, vLLM, LangGraph, retrieval or memory frameworks, or comparable.

Skills

What Lead ML / AI jobs ask for — and how much of it you have →
Apply

See also

ML / AI jobs by country — openings, pay and top skills →