Principal AI Engineer - Remote
Posted Updated 10
views
The Opportunity
We’re partnering with a leading global management consulting firm that advises C-suite leaders on their most complex problems. Its internal platform and product group builds and runs the 30+ products its consultants use, and AI is now going into all of them. Existing products are adding AI features, and new products are being built from AI use cases that are still being defined.
This role owns how AI gets built across that portfolio, and you’ll build much of it yourself. The firm is hiring Principal Engineers first and building teams around them, so you’d come in early and set the patterns everyone else follows. Your users are senior partners who have little patience for AI that demos well and breaks in production. A separate platform team owns infrastructure, so your focus stays on the AI itself.
Expect 70-80% hands-on engineering, with the rest on architecture and mentoring. The role is internal facing with no travel, and it’s open to remote within the U.S. or hybrid in Midtown NYC. It’s a 12-month contract (W2, 1099, or C2C) with a strong likelihood of extension.
What You’ll Own
- Build and ship ML and LLM capabilities into existing products, and help turn early AI use cases into new products. This means training, fine-tuning, and evaluating models, not just prompting them.
- Set the AI architecture: reusable service scaffolding, model gateway and interface patterns, and multi-tenant LLM services with enforced data boundaries between engagements.
- Own AI security from the start. That covers prompt injection defenses, prompt data classification, model supply chain risk, and agent governance (permissions, code execution controls, access to internal systems, human-in-the-loop). You’ll partner with Security, Legal, and Compliance on SOC2/ISO-aligned controls and vendor DPA requirements.
- Build the evaluation and monitoring layer: model evaluation rubrics, regression testing for models and prompts, production-readiness criteria, and quality monitoring that teams adopt without hand-holding.
- Lead major AI decisions through written trade-off docs, advise senior stakeholders on AI risk, mentor 5-7 senior engineers, and work closely with product managers. You’ll also champion AI-assisted development practices across the engineering org.
What You Bring
- 12+ years in software engineering, ML, or data science, including production ML and LLM systems you built, trained, and shipped. Experience applying AI in consulting or professional services preferred.
- Deep Python and ML framework expertise (PyTorch, TensorFlow, Hugging Face), including training and fine-tuning, plus experience deploying models to public cloud. Azure preferred; AWS or GCP is fine.
- Production experience designing and running multi-tenant AI services and LLM integrations: RAG, retrieval, model routing, and managing latency and cost. Familiarity with AI gateway patterns and emerging integration protocols (MCP, etc.) preferred.
- Hands-on model evaluation: offline and online evals, golden datasets, LLM-as-judge with calibration, and production quality monitoring. MLOps tooling experience preferred.
- Substantive AI/LLM security experience: prompt injection, data boundary enforcement, model supply chain risk, AI threat modeling, and sandboxing agents. Experience defining AI controls for SOC2 or ISO 27001 preferred.
- Microservices architecture experience: decomposing monoliths, defining service contracts, and CI/CD for distributed systems. You should be comfortable with Docker, but you won’t own the infrastructure.
- Solid engineering habits around testing, code review, and releases. Go or TypeScript is a plus.
- Strong written and verbal communication, including with executives. You give the answer first and add detail when asked. Open-source contributions, publications, or a track record of internal docs and workshops preferred.
- Comfort working in Agile, cross-functional product teams.