AI Governance & Operations Specialist
Summary
Manages enterprise conversational AI chatbot platforms day to day: monitoring performance and response quality, enforcing AI guardrails, mitigating risks like hallucinations and prompt injection, optimizing LLM token costs, and reporting to stakeholders. Core tech: OpenAI/Azure/Gemini/Claude/Bedrock LLMs, RAG, vector DBs, Python, SQL, AWS, Salesforce.
Role & responsibilities
Role Summary
We are seeking an AI Chatbot Governance & Operations Specialist to manage enterprise conversational AI platforms and ensure their performance, compliance, reliability, and
continuous improvement. The role focuses on chatbot monitoring, AI governance, risk management, analytics, cost optimization, and stakeholder coordination.
Key Responsibilities
* AI Chatbot Operations & Performance Monitoring:
- Monitor chatbot performance, availability, response quality, and user experience across channels.
- Track key metrics including Resolution Rate, Escalation Rate, CSAT,
Response Accuracy, and Hallucination Rate.
* AI Governance & Risk Management
- Implement and monitor AI guardrails to ensure safe, compliant, and responsible AI behaviour.
- Identify and mitigate risks related to hallucinations, prompt injection, data privacy, security, and policy violations.
* Continuous Improvement & Feedback Management
- Analyse chatbot conversations and feedback to identify improvement opportunities.
- Collaborate with development teams to enhance chatbot effectiveness and user experience.
* Token Consumption & Cost Management
- Monitor LLM usage, token consumption, and AI operational costs, recommending optimization strategies.
* Reporting & Stakeholder Management
- Develop dashboards and reports on chatbot performance, governance, and utilization metrics.
General
- Coordinate with Product, IT, Security, Legal, and Compliance teams to
support governance reviews and release management.
Technical Skills
* Conversational AI Platforms & Generative AI (LLMs)
- LLMs (OpenAI, Azure OpenAI, Gemini, Claude, Bedrock)
* Prompt Engineering & Knowledge Management
* AI Governance, Responsible AI & Compliance
* AI Risk Management, Hallucination Monitoring & Security
* AI Monitoring, Analytics & Performance Reporting
* SQL, API Integrations, AWS and Architecture
* RAG, Knowledge Graphs & Vector Databases
* Python, APIs, Microservices
* Salesforce Ecosystem
* CI/CD, MLOps, Observability
Preferred Experience
* 4+ years in Conversational AI Operations, AI Governance, AI Product Support, or
related roles.
* Hands-on experience with OpenAI GPT, Microsoft Copilot, Google Gemini, AWS
Bedrock, or Salesforce Einstein Bots.
* Experience supporting enterprise customer service or employee support
chatbots.
* Hands-on experience with LLMs, AI orchestration frameworks, RAG,
vector databases and AI agents.
* Experience in cloud platforms (AWS/Azure), microservices and API architecture.
* Experience delivering customer service, sales, service operations or
conversational AI platforms.
Competencies
* Analytical Thinking
* AI Governance Mindset
* Problem Solving
* Stakeholder Management
* Communication & Presentation Skills
* Continuous Improvement Orientation
If you are interested in above profile please contact on (M) 9328933981 or e.mail - contact@authentic-staffing.com