Technical Operations Analyst, Automation & Evaluations
Posted Updated
Description
You will partner with teams across User Safety & Risk Operations to support their data, analysis, measurement and automation needs. You will build tools, datasets, dashboards and workflows, evaluate them against expert-reviewed examples, and hand them off with clear ownership. The role may involve exposure to sensitive content.
Responsibilities
- Define the questions teams need answered and identify work that could benefit from automation
- Use SQL and Python to investigate operational problems and make recommendations
- Build and maintain datasets and dashboards
- Define metrics to assess quality and impact of work
- Build scripts and API integrations to automate recurring tasks using LLMs where useful
- Evaluate automated workflows against expert-reviewed examples
- Hand off workflows with clear ownership, maintenance instructions and feedback loops
Requirements
- Strong SQL skills
- Python for data analysis and tool building
- Experience with messy operational data
- Ability to turn loosely defined requests into clear problems
- Experience building datasets, dashboards, scripts or API integrations
- Ability to define metrics
- Understanding of workflow automation trade-offs
- Experience designing evaluations or quality checks
- Nice to have: experience transferring prototypes into scaled operations
Skills
Sign in to read and join the discussion.