Data Scientist

Upcoming Tasking

• AI Model Benchmarks — Design, implement, and maintain benchmark suites that measure model performance against defined evaluation criteria, including dataset curation, scoring, and results reporting.
• Agentic AI Benchmarks — Develop benchmarks and test scenarios that evaluate agentic workflows and multi-step task performance, including tool-use, task completion, and reliability metrics.
• Evaluation Dataset Development — Curate, validate, and version evaluation datasets and test scenarios, including any adversarial or edge-case sets needed to stress agent behavior.
• Metrics, Analysis, and Reporting — Produce scoring frameworks, visualizations, and written analyses that translate raw evaluation results into actionable findings for stakeholders.

Seven years experience as a SWE, in programs and contracts of similar scope, type, and complexity is required. Bachelor’s degree in Computer Science or related discipline from an accredited college or university is required. Four years of additional SWE experience on projects with similar software processes may be substituted for a bachelor’s degree.

• InspectAI
• Python
• MongoDB
• Jupyter notebooks
• Data visualization tools
• Statistical analysis
• Data cleaning and validation
• Cloud or containerized environments
• Git-based version control
• Collaboration tools such as Jira or Confluence
• Experience operating in airgapped or constrained environments