DevOps Engineer
Summary
Onsite contract DevOps Engineer in Harrisburg, PA joining a Platform Engineering team to build and run self-hosted AI/LLM infrastructure on Kubernetes (LiteLLM, vLLM, Langfuse, MLflow, Kubeflow, LangGraph), manage on-prem GPU workloads, and automate via Helm, GitOps (Argo CD/Flux) and GitLab CI, with monitoring, security, and FERPA compliance duties and an on-call rotation.
e&e is seeking a DevOps Engineer for an onsite contract opportunity in Harrisburg, PA!
We are seeking an experienced DevOps Engineer to join our Platform Engineering team, with a primary focus on supporting and maintaining AI infrastructure and services. This role will be responsible for deploying, operating, securing, and optimizing the platforms that support AI-driven applications and agentic software development initiatives.
The ideal candidate will have strong experience with Kubernetes, DevOps automation, CI/CD pipelines, and AI/LLMOps technologies. This individual will work closely with engineering teams to ensure AI services are reliable, scalable, secure, and production-ready while contributing to broader platform engineering initiatives.
Key Responsibilities
AI Infrastructure & Platform Operations
- Deploy, manage, and maintain self-hosted AI infrastructure on Kubernetes, including LiteLLM, vLLM, Langfuse, MLflow, Kubeflow, and LangGraph.
- Manage on-premises GPU workloads, including scheduling, capacity planning, performance optimization, and model deployment and rollback.
- Operate and maintain the central Model Context Protocol (MCP) gateway, ensuring secure access to internal systems and proper user authorization.
- Collaborate with AI engineering teams to troubleshoot issues, eliminate technical blockers, and transition prototypes into reliable production services.
- Ensure AI platforms remain scalable, highly available, and optimized for performance.
DevOps Automation & CI/CD
- Develop and maintain Helm charts, GitOps deployment processes, and CI/CD pipelines.
- Automate infrastructure provisioning, configuration management, deployments, and operational processes.
- Implement production readiness standards, including monitoring, alerting, backup strategies, and disaster recovery procedures.
- Support GitOps tools such as Argo CD or Flux and CI/CD platforms such as GitLab CI.
- Participate in platform engineering projects, sprint planning, code reviews, and continuous improvement initiatives.
Monitoring, Security & Compliance
- Implement observability solutions to monitor AI service performance, latency, token usage, costs, error rates, and model quality.
- Maintain logging, monitoring, and alerting systems to ensure platform stability and operational visibility.
- Apply security best practices, including identity and access management, secrets management, audit logging, and data protection.
- Ensure AI infrastructure complies with applicable privacy and security requirements, including FERPA.
- Identify and address infrastructure vulnerabilities, performance bottlenecks, and operational risks.
Documentation & Team Collaboration
- Develop and maintain technical documentation, operational runbooks, and troubleshooting procedures.
- Provide knowledge transfer and hands-on cross-training to other DevOps engineers.
- Collaborate with development, infrastructure, security, and platform engineering teams.
- Participate in a scheduled on-call rotation to support production environments.
- Promote automation, reliability, and operational best practices across the engineering organization.
Required Qualifications
- 3+ years of experience in DevOps, Platform Engineering, or Site Reliability Engineering (SRE), including managing production Kubernetes environments.
- Hands-on experience deploying and operating at least two AI/LLMOps technologies, such as LiteLLM, vLLM, MLflow, Kubeflow, LangGraph, Langfuse, or comparable tools.
- Strong working knowledge of Kubernetes, Docker or other container technologies, and Linux environments.
- Experience with Helm, GitOps tools such as Argo CD or Flux, and CI/CD pipelines, preferably GitLab CI.
- Proficiency in at least one scripting language, preferably Python.
- Understanding of networking fundamentals, infrastructure automation, and production troubleshooting.
- Experience implementing monitoring, logging, alerting, and backup solutions.
- Strong automation-first mindset with the ability to develop repeatable, scalable infrastructure solutions.
- Ability to effectively leverage AI tools while validating outputs for accuracy and reliability.
- Strong problem-solving, troubleshooting, communication, and collaboration skills.
- Experience creating technical documentation and sharing knowledge with engineering teams.
- Willingness to participate in scheduled on-call support rotations.
Preferred Qualifications
- Experience deploying and optimizing open-weight AI models on on-premises GPU infrastructure.
- Knowledge of model quantization, batching, inference optimization, and model selection strategies.
- Familiarity with Model Context Protocol (MCP), MCP servers, agent frameworks, and LLM tool-calling capabilities.
- Experience with Cilium, RKE2, Rancher, and Harbor.
- Familiarity with the Grafana observability stack, including Mimir, Loki, and Tempo.
- Experience managing PostgreSQL, Redis, or Apache Airflow in production environments.
- Knowledge of identity management, Single Sign-On (SSO), secrets management, and secure AI data pipelines.
- Experience supporting environments with strict security, regulatory, or data privacy requirements.
- Previous experience in education, government, or public-sector technology environments.
Skills
- Agentic AI
- AI
- Airflow
- Argo CD
- Automation
- CI/CD
- Data Pipelines
- DevOps
- Docker
- Flux
- GitLab
- GitOps
- GPU
- Grafana
- Helm
- IAM
- Kubeflow
- Kubernetes
- Langfuse
- LangGraph
- Linux
- LiteLLM
- LLM
- LLMOps
- Loki
- MCP
- MLflow
- Model Deployment
- Networking
- Observability
- PostgreSQL
- Python
- Quantization
- Rancher
- Redis
- Secrets Management
- SRE
- SSO
- vLLM
Sign in to read and join the discussion.