Senior Machine Learning Researcher, Large Behavior Models & Diffusion Policy

At Toyota Research Institute (TRI), we’re on a mission to improve the quality of human life. We’re developing new tools and capabilities to amplify the human experience. To lead this transformative shift in mobility, we’ve built a world-class team advancing the state of the art in AI, robotics, driving, and material sciences.

Responsibilities

• Conduct ambitious research to advance the state-of-the-art in using new capabilities in generative AI (e.g., recent results in diffusion policy [1],[2]) for end-to-end perception, planning, and prediction in automated driving with a focus on computer vision as the primary sensing modality.
• Research and implement scalable end-to-end architectures that process raw sensor data to generate vehicle trajectories, addressing the challenges of long-tail driving scenarios with low data coverage.
• Prototype, validate, and iterate model architectures using imitation learning and large-scale data, ensuring robust performance across diverse scenarios.
• Perform closed-loop evaluations in sensor simulations and real-world testing environments to rigorously assess model performance, stability, and scalability.
• Explore multi-modal and language-conditioned models to broaden the applicability of end-to-end policies, using external data sources and transfer learning to enhance generalization.
• Collaborate with researchers and engineers across TRI, Woven by Toyota, and Toyota’s global ecosystem to accelerate model deployment and evaluation in both controlled environments (closed-course) and public road driving.
• Take the lead on writing and publishing research results in peer-reviewed venues.

Qualifications

• A PhD or equivalent experience in a robotics-relevant or embodied-AI field such as Computer Science, Mathematics, Physics, or Engineering.
• A consistent track record of publishing at high-impact conferences/journals (CVPR, ICLR, NeurIPS, ICML, CoRL, RSS, ICRA, ICCV, ECCV, PAMI, IJCV, etc.)
• A consistent track record of independent research.
• Demonstrated ability to independently formulate and complete a research agenda while collaborating across subject areas.
• Experience training large-scale models, including foundation models (e.g., vision-language models, text-to-video models).
• Proficiency in Python and C++ for implementing and evaluating research ideas.

Bonus Qualifications

• Experience with robot motion planning techniques like trajectory optimization, sampling-based planning, and model predictive control, or experience with automated driving domains (e.g., perception, prediction, mapping, localization, planning, simulation).
• Experience in developing production-level code for real-time operating systems.
• Experience optimizing runtime-critical systems for Linux, UNIX-like real-time operating systems on automotive-grade compute platforms, and building safety-critical software architectures.