COMPUTER SCIENTIST (AI TEST AND EVALUATION)

This position may be filled using Direct Hire Authority: Z5CAV/Direct-Hire Authority (Certain DoD Personnel) PL 118-31, Sec 125OB (i)(2), 12/22/2023.This position is being filled under the memorandum from the Under Secretary of Defense for Personnel and Readiness (USD(P&R)) "Expansion of Direct Hire Authority for Certain Personnel of the Department of Defense," dated August 12,2024. This position is part of the Defense Threat Reduction Agency,

As a COMPUTER SCIENTIST (AI TEST AND EVALUATION) at the GS-1550-13 some of your typical work assignments may include: Plans and executes comprehensive test and evaluation (T&E) of AI/ML systems by developing test plans, defining quantitative and qualitative metrics (e.g., sensitivity, specificity, confidence levels, generalizability), conducting hypothesis testing, and performing verification and validation activities to ensure AI solutions meet performance specifications and requirements. Tests AI/ML systems for security, robustness, trustworthiness, and reliability by conducting adversarial testing in operationally realistic environments, performing risk assessments, evaluating functionality and compatibility across diverse scenarios, and building assurance cases that demonstrate the level of confidence in AI system capabilities. Develops and executes code-level testing and validation procedures for machine learning models, including automating testing processes, evaluating user experience and human-computer interaction factors, and recording and managing all test data with accuracy and integrity throughout the testing life-cycle. Prepares detailed T&E reports and communicates findings, risks, and recommendations to technical and non-technical audiences, including senior leadership, by translating complex test data and evaluative conclusions into clear, actionable courses of action that inform decision-making across the organization. Assesses AI systems for bias, security vulnerabilities, and high-impact risks by reviewing AI architectures, analyzing training data sets and system outputs for unintended bias, identifying failure modes and low-probability risks, and developing risk management plans and corrective actions to ensure ethical and trustworthy AI deployment. Collaborates with cross-functional teams - including developers, data scientists, cybersecurity professionals, and operational users- to integrate AI T&E frameworks into project test strategies and ensure seamless coordination across DataOps, MLOps, and DevSecOps pipelines, including AI deployment within cloud and IT infrastructure environments. Represents the organization in working groups, interagency forums, and professional eventsrelated to AI T&E and responsible AI, while monitoring evolving test and safety standards (e.g., MIL-STD 882E, DO-178C, ISO 26262) to ensure AI evaluation activities remain current, rigorous, and aligned with mission requirements and AI ethical principles.