Associate Engineering Manager - GenAI

The role is responsible for leading the design and development of enterprise-grade GenAI capabilities for banking applications, with a strong focus on LLM inference, RAG, intelligent agents, and scalable backend systems. The role will drive the architecture and implementation of highly available, secure, and low-latency distributed systems that enable GenAI use cases across banking functions while optimizing model serving for performance, scalability, reliability, and cost. The role will work closely with product, ML/AI, data, and platform teams to take GenAI solutions from concept to production, ensuring they meet the security, compliance, reliability, and operational requirements of a banking environment.

Responsibilities:

• Design and build scalable backend and inference systems for production-grade GenAI applications, including LLM serving, APIs, and distributed services.
• Build and optimize RAG pipelines, including document ingestion, chunking, embeddings, vector search, retrieval, reranking, and context orchestration.
• Develop and productionize GenAI capabilities such as LLM inference, prompt engineering, agents, tool calling, fine-tuning, and model evaluation.
• Optimize inference performance across latency, throughput, GPU utilization, scalability, availability, and cost through techniques such as batching, caching, quantization, and efficient model serving.
• Build robust data and AI infrastructure covering data pipelines, model serving, observability, evaluation, monitoring, and continuous improvement of GenAI systems.
• Collaborate with ML, Product, Platform, and Infrastructure teams to take GenAI solutions from prototype to reliable, scalable production systems.

Secondary Responsibilities:

• Evaluate emerging LLM, GenAI, inference, and agentic AI technologies and identify opportunities for adoption.
• Contribute to POCs, technical prototypes, and architecture initiatives to validate new GenAI capabilities and approaches.
• Establish best practices for GenAI evaluation, prompt/version management, model quality, safety, and responsible AI.
• Improve engineering productivity through automation, reusable frameworks, tooling, and developer platforms for GenAI teams.
• Support production operations, incident resolution, capacity planning, and performance troubleshooting for GenAI services.
• Contribute to technical documentation, architecture standards, knowledge sharing, and engineering best practices across the organization.

Managerial and Leadership Responsibilities:

• Lead and mentor a team of backend and inference engineers building scalable, reliable, and high-performance GenAI systems.
• Own technical strategy and architecture for GenAI inference, model serving, backend services, APIs, and distributed systems.
• Drive end-to-end execution of engineering initiatives, including planning, prioritization, delivery, and cross-functional collaboration.
• Drive optimization of inference systems for latency, throughput, scalability, GPU utilization, reliability, and cost efficiency.
• Build a strong engineering culture through technical mentorship, performance management, hiring, and development of high-performing teams.
• Partner with product, ML/AI, and infrastructure teams to productionize GenAI capabilities and ensure operational excellence.