AI/Solution Architect

Role Summary

The AI/Solution Architect will be the design authority for AI services built by in-house teams and empanelled agencies, defining the reference architecture and ensuring that components built once are reused across government departments. The role requires depth across LLM, retrieval, speech, vision and agentic systems, sound judgement on hosting and model choices under sovereign-AI requirements, and the authority to approve or reject designs before build and at go-live.

Educational Qualification

• B.Tech./B.E., M.Tech./M.S. or Ph.D. in Computer Science, Information Technology, AI/ML, Electronics or a related discipline
• Advanced degree or published work on large-scale ML or distributed systems is highly desirable
• Certifications are not mandatory; architectures the candidate has designed, as listed under Minimum Evidence at Screening, shall carry greater weight
• Candidates from non-traditional backgrounds with strong verifiable work will be considered in lieu of formal qualification

Experience

• 10+ years in software engineering and architecture (12+ years preferred); minimum 5 years designing AI/ML systems deployed in production
• Demonstrable experience as lead architect of at least one LLM, retrieval or agentic system serving end users, with documented design decisions and production metrics
• Experience designing shared platforms or services reused by several products, business units or clients
• Experience with both self-hosted open-weight models on GPUs and managed cloud AI services, including cost and latency trade-offs between them
• Experience integrating AI services with identity, document and workflow systems at scale
• Experience in regulated sectors or with systems serving crores of users is a plus

Key Responsibilities

Reference Architecture & Design Authority

• Define and maintain the programme's reference architecture, covering document intake, OCR and extraction, retrieval, speech, identity verification, agents, the AI gateway, evaluation and guardrails
• Review and approve solution designs from every in-house and agency team before build, and sign off architecture conformance at go-live
• Maintain architecture decision records and reference implementations that teams can adopt directly

Model & Hosting Strategy

• Set open-weight and Indian sovereign models hosted on IndiaAI GPU compute as the default, and define when approved third-party model APIs may be used
• Ensure every service reaches models through the AI gateway, so that providers can be switched by configuration without code change
• Size GPU, storage and API budgets for each service against latency, throughput and cost targets

Integration with Government Systems

• Design integration with Digital India platforms, government identity and single sign-on services, e-office and payment systems, and departmental databases
• Define standard API contracts, identity and access patterns, event flows and data-exchange formats for AI services

Reuse & Portfolio Architecture

• Maintain the register of reusable AI services, and decide which team builds a shared component when several projects need it
• Prevent duplicate builds across teams and agency contracts, and plan the migration of existing services onto shared components

Security, Privacy & Responsible AI by Design

• Embed DPDPA 2023, CERT-In directions, MeitY security guidelines, audit logging and data residency in every design
• Work with the AI Safety and Security teams on threat models, guardrails, retrieval provenance and agent permissions

Technical Leadership & Procurement Support

• Run architecture reviews, mentor pod leads and engineers, and resolve cross-team technical disputes
• Contribute architecture requirements to NeGD RFQs and RDRs, and evaluate agency proposals and deliverables for architectural soundness

Technical Competencies

• AI Systems: LLM applications, retrieval-augmented generation with hybrid search and re-ranking, agents and the Model Context Protocol (MCP), speech and vision pipelines, document AI
• Model Serving & Inference: vLLM, SGLang, TensorRT-LLM or NVIDIA Triton Inference Server; NVIDIA Dynamo or llm-d for distributed inference; quantisation, continuous batching and prefix caching; GPU capacity planning
• AI Gateways & Providers: LiteLLM, Portkey, Kong AI Gateway or Envoy AI Gateway; OpenAI-compatible APIs; at least one of AWS Bedrock, Azure AI Foundry or Google Vertex AI; Indian providers (Sarvam, Bhashini)
• Architecture Patterns: microservices, event-driven and domain-driven design, multi-tenancy, API management, caching and asynchronous processing
• Platforms & Data: Kubernetes and Helm, Terraform, Kafka or equivalent; PostgreSQL with pgvector, object storage, vector databases (Qdrant, Milvus or Weaviate)
• Identity & Integration: OAuth 2.0, OpenID Connect, SAML, Keycloak or equivalent; REST with OpenAPI, GraphQL, WebSockets and server-sent events
• Evaluation & Observability: release gates built on golden datasets; Langfuse or Arize Phoenix, OpenTelemetry, Prometheus and Grafana
• Security & Governance: OWASP Top 10 for LLM Applications, MITRE ATLAS, zero-trust networking, secrets management, software supply-chain security (SBOMs, signed artefacts, pinned dependencies); DPDPA 2023, CERT-In directions, MeitY Responsible AI advisory, IndiaAI Safe & Trusted AI framework
• Communication: architecture documents and diagrams that engineers can build from and executives can decide on