Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Gramian Consulting

NewBe an early applicant

Senior Software Security Engineer - AI Benchmark Development

Posted Updated

Description

Gramian Consulting is seeking an experienced Senior Software Security Engineer to develop AI training tasks that model real-world vulnerabilities. You will transform CVEs, advisories, and open-source bugs into Harbor-format audit-and-patch tasks across multiple languages, supporting remote collaboration.

You will research vulnerabilities, reproduce issues, build benchmarks, and author reliable tests and documentation while ensuring robust verifiers and realistic evaluation environments for safe

Vulnerability Research: Source and investigate real-world vulnerabilities from CVEs, advisories, bug reports, and open-source repos. Security Classification: Identify and classify vulnerabilities using CWE and map to OWASP Top 10 where appropriate. Vulnerability Reproduction: Reproduce issues and develop proof-of-vulnerability tests. Benchmark Development: Build Harbor-format audit-and-patch tasks across languages. Task Authoring: Write realistic task instructions without revealing vulnerable code or fixes. Environment Setup: Create reproducible Docker-based environments with pinned dependencies and clear setup steps. Verifier Development: Design automated verifiers to prevent gaming and overfitting. Patch Evaluation: Review AI-generated patches and reasoning against security rubrics. Quality Assurance: Identify tasks solvable by superficial pattern matching. Technical Documentation: Document context, reproduction steps, and test methodology. 5+ years of professional software engineering experience. Expert-level proficiency in at least one major programming language. Strong understanding of software security principles and common vulnerability classes. Ability to independently reproduce vulnerabilities and develop tests. Hands-on experience with security testing tools (sanitizers, fuzzers, static analysis like CodeQL/Semgrep). Proficiency with Git, Linux, Bash, and Docker. Excellent technical writing and remote-work capability.

Skills

What Senior Security jobs ask for — and how much of it you have →
Apply

See also

Security jobs by country — openings, pay and top skills →