Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

BIP

AgentOps Engineer – AI Agent Observability

Posted Updated

Description

About the role

🚀 Want to join the challenge?

At BIP - xTech we are looking for people with a desire to grow, learn, and take on real technology projects: cloud, data, AI, security, modern workplace, automation, DevSecOps, IoT, Industry 4.0... You won't get bored here!

We are a diverse and enthusiastic team, where we combine technology with strategy to deliver top solutions to our clients. If you are interested in being at the center of digital transformation, this is the place for you.

What you'll do

We are looking for people who move fluently between the world of cloud observability and that of AI agents to join our AgentOps managed service for a leading international consumer goods client. You will be part of the team that monitors, operates, and evolves a growing fleet of enterprise AI agents (Azure AI Foundry and Copilot Studio) in production, ensuring they function well, securely, within agreed service levels, and improve over time.

  • Agent observability: design, deploy, and evolve dashboards in Grafana and Azure Monitor to track agent health, latency, availability, errors, token consumption, and cost.
  • Proactive monitoring and alerting: define and monitor KPIs, SLAs, and alert thresholds, and early detect degradations, anomalies, and quality loss (drift).
  • Incident management: triage, log and trace analysis, root cause analysis (RCA), and resolution or escalation within SLAs, stepping into the configuration of the agent itself when necessary.
  • Agent improvement and evolution: adjust prompts, knowledge sources, connectors, and actions, validate changes with non-regression testing, and manage model updates.
  • Lifecycle and deployments: support the versioning, deployment, and rollback of agents, and ensure that new agents reach production with the necessary observability and runbooks.
  • Reporting and documentation: periodic service health reports, runbooks, and incident playbooks.

What we're looking for

✅ What we need from you:

Essential to have at least 2 years of experience in Cloud environments and practical experience in:

  • Azure Observability: Azure Monitor, Log Analytics, and Application Insights, with KQL queries.
  • Grafana: creation and maintenance of dashboards, panels, variables, and alerts on sources such as Azure Monitor, Log Analytics, or Prometheus/PromQL.
  • Agent Observability: monitoring AI agents through traces, metrics, and logs (OpenTelemetry or similar): latency, tokens, cost, errors, response quality, and tool usage.
  • AI Agents: understanding how they work (LLMs, prompting, RAG, tool calling, orchestration) and having worked with them in a real environment, not just at a theoretical level.
  • Production service operations: incident management, RCA, and working with SLAs and ticketing tools.
  • Fluent English, oral and written (the service is provided to an international client).

Nice to have

Additionally, although not essential, it is important that you have notions of:

  • Copilot Studio: how an agent is built and configured (topics, actions, connectors, and knowledge sources) and where to consult its analytics.
  • Azure AI Foundry / Azure OpenAI: how models and agents are deployed and which services are involved.
  • Python: enough to read code, understand an agent, or automate simple tasks.

🔍 It's a plus if you also have:

  • You have developed pro-code agents with frameworks such as Microsoft Agent Framework, Semantic Kernel, AutoGen, or LangChain.
  • You know LLM evaluation (quality metrics, groundedness, golden datasets, non-regression testing).
  • You have used Azure DevOps, Git, and CI/CD pipelines to deploy and version agents.
  • You have experience with AI security: Azure AI Content Safety, guardrails, prompt injection, or PII management.
  • You have worked with infrastructure as code (Terraform or Bicep) or services such as Azure Functions, APIM, or Key Vault.

Education & certifications

  • You have technical training (Computer Science, Telecommunications, Mathematics, etc.).
  • You have official certifications such as AI-102, AZ-104, PL-200, AI-900, or similar.

What you'll get

🌟 Why join BIP?

  • People first: we value your talent and your work-life balance.
  • Agile and flexible environment: we work remotely with schedules that adapt to you.
  • Continuous learning: you'll have access to top courses in technology and innovation.
  • Impactful projects: you will work with leading companies in key sectors and with cutting-edge AI technologies.
  • A team that adds value: great atmosphere, real collaboration, and zero pretense.

Hiring process

📩 What's next?

If you're interested, send us your CV. We'll review it carefully, and if you're a good fit, we'll get in touch.

If now isn't the right time, but you think you could be a good fit in the future, we'd also love to keep you on our radar.

What do we value?

  • Drive to contribute, work as a team, and grow
  • Consultative attitude and client focus
  • Analytical and problem-solving mindset regarding production incidents
  • Ability to move between operations and development, understanding both the platform and the agent
  • Responsibility for your work and commitment to the team
  • Curiosity about generative AI and the ability to stay up to date

Additional information

Bip Iberia, in commitment and coherence with the Comprehensive Law for Equal Treatment and Non-discrimination, has all positions for profile searches open to all persons who are duly qualified to cover the vacancy, regardless of their age, disability, gender, sexual orientation or politics, race or belief.
BUSINESS INTEGRATION PARTNERS CONSULTING IBERIA, S.L., with NIF B84701903 and contact Liliana.mendoza@bip-group.com will treat the data received when applying for your candidacy in the offer as the Data Controller, for the purpose of managing the selection process currently active based on the application of pre-contractual measures under article 6.1. b) of the GDPR, as well as for future selection processes if your profile were of interest, based on the consent of article 6.1 a).
BUSINESS INTEGRATION PARTNERS CONSULTING IBERIA, S.L. will not communicate your data to third parties except for legal obligation. Likewise, the retention period of your CV will be a maximum of 1 year, after which it will be destroyed.
Likewise, you may exercise your rights of access, rectification, erasure, opposition, limitation of processing, portability, and not to be subject
to automated individualized decisions or consult additional information about the processing at the email: dpo-bipspain@bip-group.com.

Skills

Apply

See also