Senior AI Automation Engineer
Summary
Leads quality engineering for an enterprise AI platform, building test frameworks for Agentic AI workflows and LLM-based systems using Python, Java, and modern CI/CD stacks.
We’re looking for a Senior AI Automation Engineer to lead quality for an enterprise-level AI platform. This isn't just standard web automation; you’ll be building the frameworks to validate Agentic AI workflows and LLM-based decision systems. If you have a passion for solving complex testing problems in the AI space and want to work with tools like Playwright and modern CI/CD stacks, I’d love to share more about this role
Mandatory Skills Description
- AWS Bedrock — hands-on: model access, Knowledge Bases, Lambda integration (primary AI platform
- AI agents & Agentic tooling — practical knowledge of designing and operating AI agents, including agentic workflows, reusable skills, rules/guardrails, commands, and multi-tool/multi-agent orchestration
- RAG pipeline — end-to-end implementation: chunking, embedding, vector indexing, retrieval, generation
- Prompt engineering — zero-shot, few-shot, chain-of-thought, structured output (JSON mode), multi-turn
- Vector databases — working knowledge of OpenSearch, Pinecone, or Faiss; understands vector vs. graph DB difference
- Fine-tuning vs. RAG — ability to reason through which approach fits a given problem
- LLM orchestration — LangChain, LangGraph, or LlamaIndex
- Embeddings — understands semantic similarity; experience with Amazon Titan Embed or equivalent
- Python — for Lambda functions, AI pipeline scripting, and data processing
- Java — 3+ years of hands-on test automation development
- ReportPortal or equivalent test reporting tool
- REST API — concepts and hands-on usage
- Jenkins / CI-CD — pipeline debugging and integration
- AWS — S3, Lambda, API Gateway, IAM, OpenSearch Serverless
- Docker — containerized test execution environment