AI Test Engineer: Accounting Eval & QA
Posted 17
views
Description
Sumary ApS in Copenhagen is seeking an AI Test Engineer to build and expand evals that verify accounting accuracy on live client books. You will design test cases, scoring, and automated pipelines used on every model update.
You will work with researchers and engineers building the AI CFO, based in our Copenhagen office. The role emphasizes rigorous testing, trustworthy results, and clear communication of findings to ensure high-quality automation across Europe.
Build and grow eval suites across accounting tasks (reconciliation to reporting) Convert real accounting cases into test cases with defined answers Design trustworthy scoring using model-graded checks where appropriate Run evals on every model, prompt and tool change to catch regressions Track agent quality over time and communicate results clearly to the team Help decide when an agent is ready for new tasks 5+ years of experience in software engineering, test engineering or ML Built a test harness for LLMs or agents before: datasets, runners, scoring Production-grade test systems with robust, auditable results Rigorous test design to avoid fools and premature conclusions Clear written communication of results and confidence level Office in Copenhagen Real ownership of projectsSkills
Sign in to read and join the discussion.