AI Engineer, Enterprise AI Platform

AI Engineer, Enterprise AI Platform

Good Sam Analytics – AI Enablement

Location: Chicago, Illinois (Hybrid)

Employment Type: Full-time

The Opportunity

Good Sam Analytics is building the internal platform that enables Good Sam departments to talk with purpose-trained AI agents to answer real questions. We are seeking an AI Engineer who can design and build agent architecture, implement LLM pipelines, validate their outputs rigorously, and work directly with non-technical stakeholders to scope and ship new agents. This role blends Data Engineering, Data Architecture, Backend Software Engineering, and AI Orchestration. You will work inside a small, tightly-scoped squad whose mission is shipping production-quality agents on a defined staged build plan.

What You'll Actually Be Building

Agent Platform & Orchestration

• Build and extend our agent-serving architecture: a hierarchical orchestrator routing questions through specialized agents, with all LLM calls governed through a gateway for per-agent cost attribution, rate limiting, and circuit breaking.

• Bridge internal-only systems (Snowflake, internal Postgres, internal APIs) to agents securely via MCP or custom tool patterns, without ever exposing internal infrastructure to the public internet.

• Design and wire multi-agent workflows using frameworks like LangGraph, LangChain, or agentic harnesses where the task genuinely requires fan-out, specialized sub-agents, or a dedicated verifier step -- and just as importantly, know when a single well-built agent loop is the right call instead.

• Implement Retrieval-Augmented Generation (RAG) for department knowledge bases: hierarchical chunking, hybrid search, and selective retrieval so agents ground answers in real company content instead of hallucinating.

AI-Accelerated Delivery

• Use Claude Code and similar AI coding tools as the primary means of writing, refactoring, and testing code across the stack -- direct, prompt, review, and harden AI-generated output rather than hand-writing every line.

• Translate department requirements into precise structured specs and agent instructions (system prompts, tool schemas, knowledge files) that produce reliable, production-ready behavior.

• Build and maintain a golden question set for each agent domain -- question-to-expected-result pairs that run in CI and catch regressions before they reach stakeholders.

• Maintain sound judgment on when to trust AI-generated code versus when to intervene manually -- speed never substitutes for correctness on a shipped agent.

Governance & Reliability

• Build in security and cost controls including per-agent spend caps and rate limits, human-approval gates on any consequential action, read-only enforcement, etc.

• Implement audit logging for every tool call, session, and human approval -- tie every action back to the requesting user's identity.

• Apply a progressive-autonomy rollout: pilot with one team before wider release, gate write/action capability behind a proven read/advise phase. Autonomous write-backs are explicitly out of scope at launch -- agents answer questions, they do not initiate actions.

• Versioned, human-reviewed knowledge artifacts (retrieval documents, prompt playbooks) for teaching an agent department-specific knowledge.

Cross-Functional Collaboration

• Partner directly with department stakeholders (Sales, Marketing, etc.) to scope what their agent actually needs to answer, gather real stakeholder phrasing for the golden question set, and communicate technical tradeoffs in plain terms.

• Document architecture decisions and open questions as you go -- this team keeps a living internal knowledge base of what's been tried, what worked, and what's still unresolved. You are expected to contribute to it.

• Mentor other engineers on agent-building patterns, eval design, and validation discipline as the team and platform grow.

Requirements

• 2+ years of software engineering experience with strong backend fundamentals in Python -- sufficient to read, debug, and extend AI-generated code with confidence, not just run it.

• Hands-on experience building with LLMs: prompt engineering, tool/function calling, structured output parsing, and evaluating model output with engineering rigor -- you can explain why an agent failed and what specifically you would change.

• Experience with at least one agent orchestration pattern or framework (LangGraph, LangChain, a manual tool-use loop, or a comparable approach acceptable) -- you understand what a graph edge is and why it matters for retry and routing logic.

• Solid SQL and data warehouse proficiency -- Snowflake experience strongly preferred. You can write and debug CTEs, window functions, and joins, and you understand why a query returns zero rows without panicking.

• Direct experience directing AI coding assistants (Claude Code, Cursor, GitHub Copilot, or similar) to generate and modify production code, with demonstrated spec-writing skill.

• Solid understanding of API design, integration patterns, and data modeling -- you design integrations, not just consume them.

• Familiarity with LLM eval concepts: golden test sets, LLM-as-judge, precision/recall for RAG, eval-driven development. You've built or contributed to an eval harness, not just run one.

• Experience with Git and standard CI/CD practices.

• Comfort operating in a fast, iteration-heavy delivery model with sound judgment about where rigor cannot be compressed.

Preferred Qualifications

• Experience with Model Context Protocol (MCP) -- server/tool integration or permission policies.

• Familiarity with RAG architectures: chunking strategies, hybrid dense/sparse search, pgvector or comparable vector stores, cross-encoder reranking.

• Experience with MLflow for LLM observability: experiment tracking, trace inspection, LangGraph autologging.

• Experience with LLM API gateways or proxy layers (Bifrost, LiteLLM, PortKey, or similar) -- understanding of virtual keys, spend caps, and per-agent routing.

• Familiarity with Airflow or comparable batch schedulers for running eval DAGs, schema refresh jobs, and pre-computation pipelines.

• Understanding of AI governance concepts: sandboxing, network egress control, spend/budget caps, audit logging, or human-in-the-loop approval design.

• Experience translating a business unit's ambiguous ask into a scoped, shipped technical solution.

• Experience with cloud platforms (Azure, AWS, or GCP) and containerization (Docker/Podman).

Mindset

• Validation-first builder: you ship working prototypes in days, but you don't skip the SQL safety gate, the eval pass, or the human approval gate to get there. An agent that gives a wrong answer confidently is worse than no agent.

• Curious about the frontier: you actively try new AI tools and models and bring back what's worth adopting -- with a clear explanation of why, not just enthusiasm.

• Comfortable owning a technical component end-to-end -- from architecture choice through something real people at Camping World or Good Sam depend on in production.

• Honest about what AI can't do yet: you scope agents to what they can actually answer reliably, communicate uncertainty to stakeholders, and build the feedback loop that makes the system improve over time.