AI Safety, Responsible AI & Cybersecurity Specialist (AI Evaluation)
Description
This role is for one of our clients
Compensation: $90 - $140 per hour
1. Role Overview
Join a leading AI lab’s cutting-edge Generative AI team and contribute to the development of next-generation foundational AI models. We are seeking experienced AI Safety, Responsible AI, and Cybersecurity Specialists with practical expertise in application security, offensive security, threat analysis, red teaming, or trust and safety.
In this role, you will bring an adversarial, security-focused, and safety-first perspective to AI model development and evaluation. You will help identify vulnerabilities, assess potentially harmful model behaviour, design realistic adversarial scenarios, and establish evaluation standards to improve the security, reliability, and responsible use of AI systems.
The ideal candidate combines strong cybersecurity or trust and safety expertise with analytical thinking and the ability to assess complex technical and ethical risks in AI-generated outputs.
This is a W-2 employment opportunity, with the opportunity to work as part of a leading AI lab’s extended workforce.
Requirements
2. Key Responsibilities
- Collaborate with research and engineering teams to identify knowledge gaps and improve AI model performance across cybersecurity, Responsible AI, application and infrastructure security, red teaming, and abuse prevention.
- Design challenging, domain-specific tasks based on real-world security and AI safety scenarios, developing accurate and well-structured solutions.
- Evaluate AI-generated security and Responsible AI outputs for technical accuracy, safety, policy compliance, and potential misuse.
- Create adversarial prompts, security challenges, and red teaming scenarios to test AI systems for vulnerabilities, unsafe responses, harmful behaviour, and policy violations.
- Assess AI systems against potential threats, including prompt injection, data leakage, insecure code generation, harmful instructions, and other relevant attack patterns.
- Develop evaluation frameworks, scoring rubrics, and quality standards to assess the safety, reliability, accuracy, and potential harm of AI-generated responses, particularly on security-sensitive topics.
- Identify weaknesses in AI model responses and provide clear, structured, and actionable technical feedback to support improvements.
- Apply relevant cybersecurity and trust and safety methodologies to evaluate model behaviour across different threat scenarios and sensitive use cases.
- Work closely with other subject matter experts to maintain consistency, accuracy, and quality across AI training datasets.
3. Core Qualifications & Experience
- 4+ years of professional experience in Cybersecurity, Application Security, Red Teaming, Penetration Testing, Threat Intelligence, Incident Response, or Trust and Safety, preferably within a recognised technology company, cybersecurity organisation, or established enterprise.
- Hands-on expertise in at least one of the following areas:
- Application Security: Secure code reviews, vulnerability assessments, application security testing, security architecture, and secure software development practices.
- Offensive Security & Penetration Testing: Vulnerability discovery, exploitation techniques, penetration testing methodologies, adversarial testing, and security assessments.
- Threat Intelligence & Incident Response: Threat analysis, attack investigation, security monitoring, incident handling, threat modelling, and risk assessment.
- Trust & Safety: Content moderation, platform abuse prevention, policy enforcement, harmful content identification, safety evaluations, and risk mitigation.
Skills
Sign in to read and join the discussion.