Senior Software Engineer - AI Code Evaluation
About Gramian
Gramian Consultancy is a boutique consultancy specializing in IT professional services and engineering talent solutions. With a strong background in software engineering and leadership, we help companies build high-performing teams by matching them with professionals who truly fit their needs.
About The Role
We are looking for senior software engineers with 7+ years of industry experience to support the training and evaluation of large language models. You will review, analyze, and improve AI-generated code, applying real-world engineering judgment to assess correctness, security, scalability, maintainability, and production quality.
This is a software engineering and code-quality evaluation role, not a traditional software testing or manual QA position. You will analyze unfamiliar codebases, identify root causes, compare alternative implementations, and create robust solutions and evaluation criteria.
SENIORITY
Senior — 7+ years
Key Responsibilities
- Review and evaluate AI-generated code across programming languages and software engineering scenarios
- Assess code for correctness, reliability, security, scalability, readability, and maintainability
- Identify bugs, logical errors, incomplete implementations, edge-case failures, and architectural weaknesses
- Analyze unfamiliar codebases and understand the impact of proposed changes
- Review bug fixes, feature implementations, refactoring, API integrations, configuration changes, and database operations
- Compare alternative implementations and determine whether they satisfy technical requirements
- Rewrite or improve code to create high-quality reference solutions
- Provide clear technical explanations of identified issues and recommended improvements
- Develop evaluation criteria, technical annotations, and rubrics for code-quality assessment
- Collaborate with researchers and engineering teams to design coding benchmarks and improve LLM evaluation methodologies
Requirements
- 7+ years of professional software engineering experience
- Strong proficiency in at least one programming language, including Python, JavaScript/TypeScript, Java, C++, Go, C#, Ruby, PHP, or Rust
- Significant experience building, maintaining, debugging, and reviewing production-grade software
- Strong understanding of software design principles, clean code, modular architecture, abstraction, error handling, and maintainability
- Proven ability to identify functional, performance, security, and design issues in complex codebases
- Strong debugging, root-cause analysis, and problem-solving skills
- Experience with code reviews and collaborative software development workflows
- Familiarity with Git and modern software engineering practices
- Strong written English and ability to communicate technical feedback clearly
- Good understanding of data structures, algorithms, APIs, databases, and application architecture