#EG AI Engineer

NCS is a leading AI Tech Services company. With a 15,000‑strong team across the Asia Pacific, NCS scales its platforms and capabilities to provide clients with greater agility and AI expertise across a range of industries. Embracing a strong ecosystem of global partners, NCS transforms technology services delivery combining AI with digital resilience to drive real business impact. NCS is a subsidiary of the Singtel Group.

Job Description

• Design, build, and operate production‑grade agentic and GenAI systems—end to end. Ship services (not just notebooks): robust APIs, reusable components, and secure pipelines that connect LLMs, tools, knowledge, and enterprise systems.
• Pair strong software engineering with modern AI practices (RAG, agent orchestration, policy chains, evals) to deliver measurable business outcomes at scale.
• Agent & Application Engineering
• Design multi‑agent systems (MAS) with planning, tool‑use, and delegation (e.g., LangGraph/Semantic Kernel); expose them via REST/gRPC APIs (FastAPI/Express/Java/Go).
• Implement tool adapters (SQL, search, document stores, web calls, code exec) with strict type contracts and safe sandboxes.
• Build model gateway integrations (OpenAI/Azure OpenAI/Bedrock/Vertex; self‑hosted vLLM/TGI) with routing, rate‑limits, retries, and fallback chains.
• Retrieval, Data & Knowledge
• Stand up RAG services: chunking, enrichment, embeddings, indexing, hybrid/vector search (pgvector/Pinecone/Weaviate; OpenSearch/Azure AI Search).
• Implement ingestion pipelines (Airflow/Prefect/Celery/Ray) for docs, tickets, chat, and ERP/CRM data; handle PII redaction and metadata governance.
• Optimize retrieval quality (chunking strategies, re‑rankers, query rewriting) with offline/online evaluation and A/B tests.
• Quality, Testing & Evaluation
• Treat prompts and graphs as code: version, diff, and test them (unit tests for prompts/tools; golden sets; regression suites).
• Build evaluation harnesses (latency, cost, accuracy, toxicity, hallucination, guardrail hit‑rates); wire into CI.
• Add drift detection for conversational systems; implement safe shutdown and auto‑rollback.
• Platform & Operations
• Package services as containers; deploy to Kubernetes with Helm/Argo CD; configure autoscaling, HPA/VPA, and resource quotas.
• Implement policy chains and guardrails (OPA/Gatekeeper for policy, Presidio for PII, Trivy for image scanning).
• Instrument deep observability: tracing (OpenTelemetry), metrics (Prometheus), logs (ELK/OPENSEARCH), cost meters per request/model.
• Security & Compliance
• Manage secrets (HashiCorp Vault/KMS), signed images, SBOMs; enforce least‑privilege IAM.
• Build tenant isolation and data residency controls; implement red/blue team prompts and jailbreak defenses.
• Integration & Enterprise Workflows
• Ship connectors and events for SAP/CRM/ITSM and Kafka topics; design idempotent, retry‑safe processors.
• Automate business workflows with pro‑code services first; expose low‑code surfaces only where appropriate.
• Collaboration & Leadership
• Partner with Product, Data, and Platform teams to define SLAs/SLOs and success metrics.
• Mentor engineers on “AI as software” practices; run design reviews and postmortems.

Qualifications

• 6+ years in software engineering (prod services, not just prototypes), including 1+ year leading small projects.
• Strong in one systems language (Python/TypeScript/Go/Java) and comfortable in a second.
• Hands‑on with containers, Kubernetes, CI/CD (GitHub Actions/GitLab/Jenkins), IaC (Terraform), and cloud (Azure/AWS/GCP).
• Practical LLM experience: building RAG/agent apps, prompt design, tool‑use, and safety patterns.
• Data skills: designing schemas, batch/stream pipelines, and search indexes; proficiency with SQL and one vector DB.
• Testing mindset: unit/integration tests, load tests, golden datasets for LLM evals.
• Security basics: secrets, policies, scanning, and least‑privilege IAM.

Preferred Qualifications

• Agent orchestration (LangGraph, Semantic Kernel) and distributed compute (Ray) in production.
• Search/retrieval tuning (BM25 + vector hybrid, re‑ranking, query planning).
• Observability at scale with OpenTelemetry; cost/perf optimization across model/router layers.
• Experience in regulated or high‑throughput domains (e.g., telco, finance, healthcare); multi‑tenant and data‑residency patterns.
• Domain integrations (SAP/CRM/ITSM), event‑driven architectures (Kafka/Debezium), and policy enforcement (OPA/Gatekeeper).
• Familiarity with TM Forum APIs / BSS‑OSS patterns is a plus (if in telco context).

Tech Stack (Illustrative)

• Languages: Python, TypeScript/Node.js (plus Go/Java bonus)
• Frameworks: FastAPI/Express, LangGraph/Semantic Kernel, Ray/Celery, Airflow/Prefect
• Storage/Search: Postgres, Redis, S3/Blob; pgvector/Pinecone/Weaviate; OpenSearch/Azure AI Search
• LLM Runtime: OpenAI/Azure OpenAI/Bedrock/Vertex; vLLM/TGI; inference routers/gateways
• Platform: Docker, Kubernetes, Helm, Argo CD, Terraform, Vault, Istio
• Observability: OpenTelemetry, Prometheus/Grafana, ELK/OpenSearch
• Quality & Safety: pytest/Jest, prompt/unit test harnesses, guardrails, Presidio, Trivy