Senior Voice AI Platform Engineer (Cost, Latency, Scale)

Senior Voice AI Platform Engineer (Cost, Latency, Scale)

Company: INTIME IT Services (INTIME Voice AI)

Location: Remote. At least 4 hours of overlap with IST.

Engagement: Full-time or part-time (minimum 20 hours per week)

Experience: 5+ years in software engineering, including 2+ years on production voice AI or real-time audio systems

Compensation: To be discussed, based on experience and engagement type (full-time or part-time)

About us

INTIME IT Services is an 11-year-old SAP and AI consulting company with operations in Kolkata, Dhaka, and Dubai. INTIME Voice AI is our real-time voice agent platform, live with paying enterprise and education clients. We are expanding across India, Bangladesh, Thailand, and the UAE. We need a senior engineer who has already scaled a voice AI system in production.

The role

You own the unit economics and real-time performance of every call. Your mandate: the lowest achievable cost per connected minute, voice-to-voice latency under 1 second, and 10 times the current concurrency, without breaking quality gates.

What you will do

Cost per minute

Model the fully loaded cost per connected minute (telephony + STT + LLM + TTS + compute + bandwidth), reported per tenant, per language, and per region

Build an interaction complexity router: deterministic responses, then cache, then a small or local model, then a fast cloud LLM, then RAG, then a premium reasoning model, picked per turn by complexity, latency budget, and cost

Run break-even analysis for self-hosting STT, TTS, and LLM inference on GPUs (vLLM, TensorRT-LLM, Triton) against per-minute API pricing, and migrate where volume justifies it

Reduce LLM token spend through prompt compression, context trimming, prefix and KV caching where the provider supports it, and response length control

Route web and app voice traffic over WebRTC to skip PSTN and SIP charges

Put a quality gate on every cost change: WER, task completion, and naturalness must hold

Latency (target: p95 voice-to-voice latency under 1,000 ms)

Define latency as the time from the callers end of speech to the first audible agent audio, measured server side and client side

Own the latency budget per stage: endpointing at or below 200 ms, final STT at or below 150 ms, LLM time to first token at or below 300 ms, TTS time to first byte at or below 150 ms, transport and jitter buffer at or below 150 ms

Stream at every stage: incremental STT partials, token streaming from the LLM, and sentence-chunked streaming TTS

Replace silence-timeout endpointing with semantic end-of-turn detection, using a VAD plus turn-detection model

Run speculative execution on STT partials (intent prediction, memory and RAG prefetch, tool preparation), all cancellable and none irreversible

Handle barge-in: detect caller speech, stop TTS playback, and cancel in-flight LLM and TTS work within 200 ms

Keep persistent, warm connections to providers. Remove cold starts.

Deploy media and inference in regions close to callers: India, Southeast Asia, and the Middle East

Scale and reliability

Load test with synthetic callers up to 10 times current peak concurrency. Find and remove bottlenecks.

Autoscale media workers and inference for outbound campaign spikes

Isolate tenants so one clients spike cannot degrade another clients calls

Build provider failover with health checks and circuit breakers across 3 or more STT, TTS, and LLM providers, with no dropped calls

Track SLOs for availability, latency, and call failure rate

Cache and memory

Semantic response cache with embedding similarity thresholds, keyed by tenant, persona, language, permissions, and knowledge version

TTS audio cache for repeated phrases, keyed by text, voice, language, and prosody settings

Session memory within a call and user memory across calls, with TTL, deletion, and consent controls

Tool and API result cache with per-source TTL

Hard rule: no cache hit crosses tenants, users with different permissions, or stale data

Observability

Per-call distributed tracing with OpenTelemetry, including per-stage latency spans

Live dashboards for cost per minute, p50/p95/p99 latency, cache hit rate, provider error rate, and concurrent calls

Alerts on latency, cost, and provider anomalies

You must have done this before

Owned a production voice AI or real-time audio system with real customer traffic

Reduced cost per minute or inference cost on a live system, with before and after numbers

Brought voice-to-voice latency under or near 1 second in production, with measured results

Built or deeply modified a streaming STT, LLM, and TTS pipeline, not only configured a no-code platform

Run load tests and scaled for high concurrency

Integrated 3 or more STT and 3 or more TTS providers

Technical requirements

Expert in asynchronous Python (asyncio) or Node.js/TypeScript. Go or Rust is a plus.

WebRTC and WebSocket audio streaming, codecs (Opus, PCM, mu-law), and jitter buffers

Redis, a vector database (pgvector, Qdrant, or Milvus), message queues (Kafka, NATS, or RabbitMQ)

Docker, Kubernetes, and AWS, GCP, or Azure in multiple regions

OpenTelemetry, Prometheus, and Grafana

Strong plus

LiveKit Agents, Pipecat, or a custom voice orchestration framework

GPU inference serving (vLLM, TensorRT-LLM, Triton)

SIP trunking and carrier cost optimization

Experience at a voice AI, CPaaS, or contact center AI company

Outcomes expected in 90 days

Cost per connected minute cut to the lowest level that still passes quality gates. Target: 40 percent or more below the current baseline.

p95 voice-to-voice latency under 1,000 ms in production

Concurrency capacity up 10 times, proven by load test

Semantic cache and audio cache live, with zero cross-tenant leakage

Cost and latency dashboards live per tenant and per region

Please do not apply if

You have less than 2 years of hands-on production voice AI experience

Your voice AI experience is limited to no-code platforms or demos

You cannot share measured cost or latency results from past work

Include:

The largest voice system you worked on (calls per day, peak concurrency)

Your best cost or latency reduction, with numbers

Hours per week available, if applying part-time