Senior AI Platform Engineer – LLM Gateway
Description
About the role
Senior AI Platform Engineer -
LLM Gateway
BBVA is looking for a senior professional to join the implementation of a corporate LLM Gateway platform. The platform will provide common, secure, governed, and observable access to foundational models for the various use cases, applications, and AI agents of the
Group.
The objective of the role is to contribute to materializing and industrializing the solution, working on the defined corporate architecture and standards, and actively participating in its technical evolution. We are not looking for an exclusively functional profile or one focused on AI API consumption: practical experience in enterprise cloud platforms, security, integration, operation, and automation is required.
What you'll do
Job Summary
Implement and evolve the technical capabilities necessary so that corporate consumers can access generative AI models through a Gateway, avoiding the distributed management of provider credentials and ensuring homogeneous controls for security, governance, observability, resilience, and costs.
Main responsibilities
- Implement the LLM Gateway components and configurations according to BBVA's architecture, standards, and corporate controls.
- Technical design of integrations, consumption patterns, and evolution of platform capabilities, providing implementation and operability criteria.
- Integrate applications, agents, and consuming services with the corporate inference endpoint, facilitating reusable adoption patterns.
- Integrate the LLM Gateway with existing corporate identity, authentication, and authorization mechanisms, applying the standards defined by BBVA for workload identities, OAuth2/OIDC, JWT, audiences, scopes, and the principle of least privilege.
- Configure the secure and centralized management of secrets and provider credentials, including their segregation by environment and their rotation.
- Implement the Gateway configuration necessary for connectivity with model providers and coordinate its validation with the responsible Network and Security teams, including DNS, proxy, TLS, certificates, and controlled egress.
- Configure and evolve governance and protection capabilities: access policies, rate limiting, quotas, routing, consumption limits, filtering/guardrails, and traceability.
- Implement resilience and quality of service capabilities: timeouts, retries, circuit breaking, error handling, fallback where applicable, and load testing.
- Instrument the platform for its operation: metrics, structured logs, distributed traces, alerts, dashboards, and audit support.
- Enable the measurement of consumption and costs by application, agent, project, environment, and model, in coordination with the Control and FinOps teams.
- Automate deployments, configuration, and testing through Infrastructure as Code and
CI/CD pipelines.
- Prepare technical documentation, integration guides, operating procedures, and transfer material for BBVA teams.
- Collaborate closely with the Architecture, Cybersecurity, IAM,
Cloud/Platform Engineering, Networking, SRE, and development teams.
What we're looking for
Technical requirements
- At least 6 years of experience in platform engineering, cloud engineering, distributed systems integration, API management, or equivalent fields.
- Demonstrable previous experience of at least 1 year with AI Gateway integration.
- Demonstrable experience implementing and operating APIs, shared platforms, or transversal services in complex enterprise environments.
- Practical experience integrating inference APIs and generative AI models in production environments with high concurrency, including authentication, streaming, error management, consumption limits, latency, and token usage.
- Experience with API Gateways, API management, or equivalent integration platforms.
- Experience implementing platform capabilities applied to APIs: access policies, rate limiting, quotas, traceability, observability, and error management.
- Solid knowledge of API security and integration with existing corporate identity and access control mechanisms: OAuth2, OIDC, JWT, secrets management, scopes, audiences, and the principle of least privilege.
- Experience with AWS, especially in identity capabilities, secrets management, observability, networking, and Infrastructure as Code.
- Knowledge of cloud networking and security: VPC, controlled egress, proxy, DNS, TLS, certificates, and connectivity with external services.
- Experience implementing service observability and operation: metrics, structured logs, distributed traces, alerts, SLI/SLO, and incident diagnosis.
- Development and automation experience with Python, including implementation of APIs, integrations, middlewares, or platform components.
- Experience with Terraform, CloudFormation or CDK, CI/CD, and versioned configuration management.
- Ability to work coordinately with Architecture, IAM,
Cybersecurity, Networking, Cloud/Platform Engineering, and development teams in a regulated corporate environment.
- Autonomy in identifying problems during incidents or complex issues such as bottlenecks, integration problems, etc.
Nice to have
Valuable knowledge
- Specific experience with AWS AgentCore Gateway, Amazon Bedrock, or other AWS capabilities oriented toward generative AI.
- Previous experience in the implementation of an LLM Gateway or access platform governed for foundational models.
- Integration with OpenAI and other model providers, as well as with OpenAI-compatible
APIs.
- Experience with multi-model or multi-provider patterns, model routing, fallback, and resilience mechanisms.
- Implementation of guardrails, content filtering, protection against prompt injection,
DLP, and data classification.
- Knowledge of FinOps applied to generative AI, including measurement, imputation, and cost optimization per consumer or use case.
- Experience with Kubernetes/EKS, service mesh, and advanced SRE practices.
- Previous experience in banking or other sectors with high requirements for security, traceability, segregation, and regulatory compliance.
What you'll get
SBA: 45.000€
Mode: Remote work
Skills
- Agentic AI
- AI
- API
- Api Security
- Authentication
- Automation
- AWS
- AWS Bedrock
- CDK
- CI/CD
- Cloud
- CloudFormation
- Cybersecurity
- Distributed Systems
- DLP
- DNS
- EKS
- FinOps
- Generative AI
- IAM
- Infrastructure as Code
- JWT
- Kubernetes
- LLM
- Networking
- OAuth
- Observability
- OpenAI
- OpenID
- Python
- Regulatory Compliance
- Secrets Management
- SRE
- System Integration
- Terraform
- TLS
- VPC
Sign in to read and join the discussion.