LiftmyCVLiftmyCV

Model Evaluation Program Manager

Dyna RoboticsRedwood City, US
AI job summary

Dyna Robotics seeks a Model Evaluation Program Manager to lead a team focused on evaluating robot performance and improving model capabilities. The role involves managing workflows, designing evaluations, standardizing results, and translating research goals into practical evaluation plans. Candidates should have a background in robotics or machine learning, experience managing technical teams, and proficiency in Python. The position emphasizes collaboration with researchers and iterative improvements to evaluation processes. Dyna Robotics is committed to a diverse workforce and encourages applications from all backgrounds.

Job description

Dyna Robotics builds general-purpose robots powered by a proprietary embodied AI foundation model with top-in-industry generalization and real-world performance. Already deployed with customers across multiple industries, our robots do commercial-grade work in the physical world. Our team comes from Google DeepMind, Meta, and Cruise, and we're backed by CRV, First Round, and other leading investors.

The Role

Dyna Robotics is building general-purpose robots that perform useful work in real-world environments. We’re looking for a Model Evaluation Lead to lead the team responsible for understanding how our robot policies perform and whether new model versions deliver meaningful improvements. You’ll manage and develop the evaluation team, own evaluation planning and execution, and work closely with researchers to define what we test, how we measure it, and what we learn. This is a hands-on leadership role combining people management, experimental rigor, and technical judgment. Your work will help researchers understand model behavior, prioritize improvements, and assess readiness for deployment.

What You'll Do

  • Manage work flows for the evaluation team. Manage the team’s priorities. Establish clear ownership, provide regular coaching and feedback, and build onboarding and training practices that help team members execute evaluations and investigate issues independently.
  • Work closely with researchers. Translate research questions into evaluation plans with clear hypotheses, success criteria, and deliverables. Partner with researchers to compare model checkpoints, assess new capabilities, and balance recurring evaluations with urgent experiments and available robot capacity.
  • Design meaningful evaluations. Develop task definitions, scoring rubrics, and test scenarios that reflect real operating conditions. Measure task success, output quality, cycle time, human interventions, and recovery behavior. Choose appropriate trial counts and account for variability when interpreting results.
  • Make results repeatable and trustworthy. Standardize setup, resets, scoring, and intervention rules. Ensure results are traceable to model versions, robot configurations, test conditions, logs, and videos. Coordinate with hardware and operations partners to resolve issues that affect evaluation quality.
  • Turn failures into actionable findings. Review robot behavior, logs, and video with the team and researchers. Distinguish model failures from hardware, software, setup, or execution issues. Identify recurring failure patterns and recommend focused investigations, additional data collection, or annotation improvements.
  • Own regression coverage and reporting. Maintain consistent benchmarks and held-out test scenarios across tasks and conditions. Identify regressions, track progress across model versions, and communicate what improved, what remains unreliable, and where evidence is still limited. Provide clear recommendations that inform deployment decisions.
  • Improve evaluation tools and workflows. Use data analysis and lightweight scripting to improve reporting and reduce manual work. Partner with engineering on evaluation tooling, automation, and dashboards. Shorten the time from a new checkpoint to useful findings while preserving the quality of the evidence.

What You'll Bring

  • Experience in robotics, machine learning, applied research, or model evaluation, with direct ownership of evaluation design and analysis.
  • Demonstrated experience managing and developing a technical team, including setting priorities, giving feedback, and building effective working practices.
  • Strong experimental judgment: you can define success, control relevant variables, identify gaps in test coverage, and assess whether an apparent improvement is meaningful.
  • Technical fluency to work directly with researchers on model checkpoints, generalization, data quality, and experimental tradeoffs.
  • Comfort using Python or similar tools to analyze results, inspect logs, and improve evaluation workflows.
  • Clear communication and strong ownership. You can turn an ambiguous request into an executable plan, surface problems early, and explain findings to both research and operations teams.

Bonus Points For

  • Evaluating robot manipulation policies, embodied AI, autonomous systems, or other machine learning systems operating in the physical world.
  • Building regression suites, benchmark datasets, failure taxonomies, or evaluation tools, and connecting findings to model development or deployment readiness.

At Dyna Robotics, we build technology for the real world, which requires a team as diverse as the environments our robots inhabit. We are an equal opportunity employer committed to technical rigor and mutual respect. Don’t let a checklist stop you. Data shows that underrepresented groups often only apply if they meet 100% of the criteria. We value problem-solving and grit over keyword matching. If you’re passionate about robotics, no matter your discipline, we want to hear from you, even if you don't check every box.

Responsibilities

Manage work flows for the evaluation team. Manage the team's priorities. Establish clear ownership, provide regular coaching and feedback, and build onboarding and training practices that help team members execute evaluations and investigate issues independently. Work closely with researchers. Translate research questions into evaluation plans with clear hypotheses, success criteria, and deliverables. Partner with researchers to compare model checkpoints, assess new capabilities, and balance recurring evaluations with urgent experiments and available robot capacity. Design meaningful evaluations. Develop task definitions, scoring rubrics, and test scenarios that reflect real operating conditions. Measure task success, output quality, cycle time, human interventions, and recovery behavior. Choose appropriate trial counts and account for variability when interpreting results. Make results repeatable and trustworthy. Standardize setup, resets, scoring, and intervention rules. Ensure results are traceable to model versions, robot configurations, test conditions, logs, and videos. Coordinate with hardware and operations partners to resolve issues that affect evaluation quality. Turn failures into actionable findings. Review robot behavior, logs, and video with the team and researchers. Distinguish model failures from hardware, software, setup, or execution issues. Identify recurring failure patterns and recommend focused investigations, additional data collection, or annotation improvements. Own regression coverage and reporting. Maintain consistent benchmarks and held-out test scenarios across tasks and conditions. Identify regressions, track progress across model versions, and communicate what improved, what remains unreliable, and where evidence is still limited. Provide clear recommendations that inform deployment decisions. Improve evaluation tools and workflows. Use data analysis and lightweight scripting to improve reporting and reduce manual work. Partner with engineering on evaluation tooling, automation, and dashboards. Shorten the time from a new checkpoint to useful findings while preserving the quality of the evidence.

Job details

Published
November 3, 2026
Last checked
November 17, 2026
Job source
ashby
First seen
November 3, 2026

Skills mentioned in this job

roboticsmachine learningevaluationPython

LiftmyCV exclusive data

Dyna Robotics hiring intelligence

Dyna Robotics currently has 8 active jobs tracked by LiftmyCV. 4 new jobs in the past 7 days.

Verified November 17, 2026

Active openings

8

Currently tracked by LiftmyCV

New in 7 days

4

7 new jobs in the past 30 days

Published salaries

—

Not published by this employer yet

Average opening age

12 days

Across 8 active openings

Workplace breakdown

Remote0%
Hybrid0%
On-site0%

Hiring countries

United States of America

Top hiring cities

Redwood City

Most in-demand roles

  • • Model Evaluation Program Manager
  • • People Operations Coordinator
  • • Recruiting Coordinator
  • • Research Technical Program Manager, Robotics
  • • Robot Safety Operator - Anaheim
  • • Senior Network and Systems Engineer

Frequently requested skills

Python
robotics
ATS
autonomous systems
Bash
communication
control
DHCP

As of November 17, 2026

Hiring insights calculated from active jobs tracked by LiftmyCV.

AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Dyna Robotics

Similar jobs from this company

Across the market

Similar jobs from other companies

Related job searches

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.