LiftmyCVLiftmyCV

Agent Evaluation and Self-Improvement Engineer

NVIDIA USAOn-siteSanta Clara, US$224,000–$356,500 per year
AI job summary

NVIDIA is seeking a principal engineer to lead evaluation strategies for autonomous agents, ensuring optimized performance through robust assessment systems. This role involves designing a self-improvement framework that allows agents to evolve based on real-time evaluation data. Responsibilities include building a consistent evaluation methodology across tools, mining production data for insights, and creating realistic testing environments. Applicants should have over 12 years of experience in large-scale systems, a strong foundation in experiment design, and proficiency in languages like Python or Go. Strong candidates will have demonstrated experience in developing self-improving agent systems.

Job description

Autonomous agents now do real work across NVIDIA, and every piece underneath them keeps changing: models, harnesses, tools, skills, and plugins. Evaluation is how a team knows whether a change made an agent better or quietly made it worse. We're looking for a principal engineer to set evaluation strategy at every layer, from individual skills and MCP servers to whole agents and products, and to design how agents use those results to improve themselves. Agents can already rewrite the skills and plugins they run on, then test whether the change helped. Making that loop trustworthy is the hard part. What does it take for an agent to get better on its own, and prove the improvement is real? We're building a system that answers that question for every agent at NVIDIA. Come build it with us!

What you'll be doing:

  • Build evaluation that works the same across harnesses, with automated checks on every change that catch regressions before users do.
  • Mine production traces for failures, new usage patterns, missing skills and tools, cost and latency hot spots, and successful runs. Turn them into eval cases and improvement signals.
  • Evaluate the eval datasets themselves: coverage of real usage, duplicate cases, and how closely they match production.
  • Build the self-improvement loop for evolving harnesses: agents propose changes to their own skills, plugins, and harness configuration, and sandboxed evals test each one. Only measurable gains ship, with human sign-off and limits agents can't change.
  • Create realistic enterprise environments: sandboxed replicas of issue trackers, code repositories, chat, and internal APIs, filled with synthetic or anonymized data, where agents safely run long, end-to-end tasks.
  • Measure multi-agent swarms as a team: how agents split work, whether handoffs succeed, and whether permissions hold at every hop. Record how long each task takes and how many tokens it uses.
  • Keep evals honest: calibrate model and agent judges against human labels. Catch agents that cheat, such as working backward from expected answers or reaching data and tools they shouldn't access. We work closely with product, platform, and hardware teams throughout NVIDIA, and we help shape open evaluation standards and benchmarks across the industry.

What we need to see:

  • BS or MS in Computer Science, Engineering, or related field (or equivalent experience)
  • 12+ years building large-scale systems, including evaluation systems for ML or agents and the data pipelines that feed them
  • Hands-on experience building and evaluating agents on an open or closed agent harness
  • Solid grounding in experiment design: statistical significance, variance across repeated runs, and A/B testing
  • Experience setting technical direction across teams as a senior individual contributor
  • Proficiency in Python, Go, Rust, or similar

Ways to stand out from the crowd:

  • Built self-improving or self-evolving agent systems that get better from their own eval results
  • Shipped an agent benchmark, eval framework, or simulated test environment adopted by other teams
  • Detected reward hacking or eval gaming, or built oversight for systems that modify themselves
  • Built domain evaluations, such as for chip design or hardware verification

Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 224,000 USD - 356,500 USD for Level 5, and 272,000 USD - 431,250 USD for Level 6. You will also be eligible for equity and benefits. Applications for this job will be accepted at least until October 20, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes. NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.

Job details

Published
October 16, 2026
Experience
Principal
Job source
myworkdayjobs
Workplace
On-site · Santa Clara, US
First seen
October 17, 2026

Skills mentioned in this job

PythonGoRust

LiftmyCV exclusive data

NVIDIA USA hiring intelligence

NVIDIA USA currently has 34 active jobs tracked by LiftmyCV. 17 new jobs in the past 7 days.

Active openings

34

Currently tracked by LiftmyCV

New in 7 days

17

31 new jobs in the past 30 days

Published salaries

68%

Median disclosed salary: $245,000

Average opening age

10 days

Across 34 active openings

Workplace breakdown

Remote15%
Hybrid0%
On-site85%

Hiring countries

United States of America

Top hiring cities

Santa Clara
Redmond
Seattle
Westford

Most in-demand roles

  • • Applied Research Scientist Intern, Agentic Biology - Summer 2027
  • • Business Controls Specialist
  • • Developer Relations Manager, Federal Cybersecurity
  • • DriveOS Documentation Technical Program Manager
  • • Enterprise Sales Intern - Summer 2027
  • • Firmware System Integration Engineer

Frequently requested skills

Python
AI
Agile
C
C++
CUDA
Kubernetes
Linux

Median disclosed salary: $245,000 · 68% of jobs publish a salary

As of November 18, 2026

Hiring insights calculated from active jobs tracked by LiftmyCV.

AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at NVIDIA USA

Similar jobs from this company

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.