Senior Software Engineer (AI Platform & LLMOps)
Summary
Build and scale AI evaluation infrastructure, automated pipelines, and observability tools for LLM agents using Go, Python, OpenTelemetry, and Temporal.
Job Title:
Senior Software Engineer (AI Platform & LLMOps)
Job Duration:
6 months contract with a possibility to extend or to go perm!
Location:
One North, Singapore (100% On-site)
Schedule:
Monday – Friday, 10:00 AM – 7:00 PM (No Overtime)
The Role We are seeking a
Senior Software Engineer
to own our
ACS embedded-evaluation
workstream. You will embed with AI product teams to build the infrastructure, automated pipelines, and observability tools that ensure our LLM agents perform accurately, cost-effectively, and reliably at scale.
Key Responsibilities
AI Evaluation Infrastructure:
Own and scale
EvalsHub
(backend, SDK, CLI, frontend). Build golden datasets, regression pipelines, and CI/CD model-release integration.
Advanced Automation:
Convert human review procedures into LLM-as-judge, multi-turn, trajectory, and deterministic evaluators to eliminate manual testing.
Observability & Tracing:
Instrument distributed agent systems using
OpenTelemetry
across Go/Python services and Temporal workflows to ensure flawless debugging data.
Optimization:
Run experiments across model architectures, RAG, and embeddings to balance accuracy, latency, and cloud costs.
Enablement:
Author technical docs, lead tool migrations, and participate in the LLMOps production operational rotation.
Requirements
Experience:
Minimum 2+ years of relevant production experience.
AI & Workflow Ecosystem:
Hands-on with
LangSmith, LangGraph/LangChain, FastAPI, or Temporal .
AI Domain Expertise:
Proven track record in LLM-as-judge frameworks, agent-trajectory analysis, RAG, semantic retrieval, or Text-to-SQL.
Data & Core Tech:
Strong analytical skills handling
geospatial data . Proficiency in
React/TypeScript
(for developer tools),
Grafana , and
Redis .
Soft Skills:
Metrics-driven problem solver with excellent cross-functional communication skills.
Senior Software Engineer (AI Platform & LLMOps)
Job Duration:
6 months contract with a possibility to extend or to go perm!
Location:
One North, Singapore (100% On-site)
Schedule:
Monday – Friday, 10:00 AM – 7:00 PM (No Overtime)
The Role We are seeking a
Senior Software Engineer
to own our
ACS embedded-evaluation
workstream. You will embed with AI product teams to build the infrastructure, automated pipelines, and observability tools that ensure our LLM agents perform accurately, cost-effectively, and reliably at scale.
Key Responsibilities
AI Evaluation Infrastructure:
Own and scale
EvalsHub
(backend, SDK, CLI, frontend). Build golden datasets, regression pipelines, and CI/CD model-release integration.
Advanced Automation:
Convert human review procedures into LLM-as-judge, multi-turn, trajectory, and deterministic evaluators to eliminate manual testing.
Observability & Tracing:
Instrument distributed agent systems using
OpenTelemetry
across Go/Python services and Temporal workflows to ensure flawless debugging data.
Optimization:
Run experiments across model architectures, RAG, and embeddings to balance accuracy, latency, and cloud costs.
Enablement:
Author technical docs, lead tool migrations, and participate in the LLMOps production operational rotation.
Requirements
Experience:
Minimum 2+ years of relevant production experience.
AI & Workflow Ecosystem:
Hands-on with
LangSmith, LangGraph/LangChain, FastAPI, or Temporal .
AI Domain Expertise:
Proven track record in LLM-as-judge frameworks, agent-trajectory analysis, RAG, semantic retrieval, or Text-to-SQL.
Data & Core Tech:
Strong analytical skills handling
geospatial data . Proficiency in
React/TypeScript
(for developer tools),
Grafana , and
Redis .
Soft Skills:
Metrics-driven problem solver with excellent cross-functional communication skills.