Agent Evaluation and Self-Improvement Engineer
NVIDIA is seeking a principal engineer to lead evaluation strategies for autonomous agents, ensuring optimized performance through robust assessment systems. This role involves designing a self-improvement framework that allows agents to evolve based on real-time evaluation data. Responsibilities include building a consistent evaluation methodology across tools, mining production data for insights, and creating realistic testing environments. Applicants should have over 12 years of experience in large-scale systems, a strong foundation in experiment design, and proficiency in languages like Python or Go. Strong candidates will have demonstrated experience in developing self-improving agent systems.
Job description
Autonomous agents now do real work across NVIDIA, and every piece underneath them keeps changing: models, harnesses, tools, skills, and plugins. Evaluation is how a team knows whether a change made an agent better or quietly made it worse. We're looking for a principal engineer to set evaluation strategy at every layer, from individual skills and MCP servers to whole agents and products, and to design how agents use those results to improve themselves. Agents can already rewrite the skills and plugins they run on, then test whether the change helped. Making that loop trustworthy is the hard part. What does it take for an agent to get better on its own, and prove the improvement is real? We're building a system that answers that question for every agent at NVIDIA. Come build it with us!
What you'll be doing:
- Build evaluation that works the same across harnesses, with automated checks on every change that catch regressions before users do.
- Mine production traces for failures, new usage patterns, missing skills and tools, cost and latency hot spots, and successful runs. Turn them into eval cases and improvement signals.
- Evaluate the eval datasets themselves: coverage of real usage, duplicate cases, and how closely they match production.
- Build the self-improvement loop for evolving harnesses: agents propose changes to their own skills, plugins, and harness configuration, and sandboxed evals test each one. Only measurable gains ship, with human sign-off and limits agents can't change.
- Create realistic enterprise environments: sandboxed replicas of issue trackers, code repositories, chat, and internal APIs, filled with synthetic or anonymized data, where agents safely run long, end-to-end tasks.
- Measure multi-agent swarms as a team: how agents split work, whether handoffs succeed, and whether permissions hold at every hop. Record how long each task takes and how many tokens it uses.
- Keep evals honest: calibrate model and agent judges against human labels. Catch agents that cheat, such as working backward from expected answers or reaching data and tools they shouldn't access. We work closely with product, platform, and hardware teams throughout NVIDIA, and we help shape open evaluation standards and benchmarks across the industry.
What we need to see:
- BS or MS in Computer Science, Engineering, or related field (or equivalent experience)
- 12+ years building large-scale systems, including evaluation systems for ML or agents and the data pipelines that feed them
- Hands-on experience building and evaluating agents on an open or closed agent harness
- Solid grounding in experiment design: statistical significance, variance across repeated runs, and A/B testing
- Experience setting technical direction across teams as a senior individual contributor
- Proficiency in Python, Go, Rust, or similar
Ways to stand out from the crowd:
- Built self-improving or self-evolving agent systems that get better from their own eval results
- Shipped an agent benchmark, eval framework, or simulated test environment adopted by other teams
- Detected reward hacking or eval gaming, or built oversight for systems that modify themselves
- Built domain evaluations, such as for chip design or hardware verification
Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 224,000 USD - 356,500 USD for Level 5, and 272,000 USD - 431,250 USD for Level 6. You will also be eligible for equity and benefits. Applications for this job will be accepted at least until October 20, 2026. This posting is for an existing vacancy. NVIDIA uses AI tools in its recruiting processes. NVIDIA is committed to fostering an inclusive work environment and proud to be an equal opportunity employer. As we highly value diversity in our current and future employees, we do not discriminate (including in our hiring and promotion practices) on the basis of race, religion, color, national origin, gender, gender expression, sexual orientation, age, marital status, veteran status, disability status or any other characteristic protected by law.
Job details
- Published
- October 16, 2026
- Experience
- Principal
- Job source
- myworkdayjobs
- Workplace
- On-site · Santa Clara, US
- First seen
- October 17, 2026
Skills mentioned in this job
LiftmyCV exclusive data
NVIDIA USA hiring intelligence
NVIDIA USA currently has 34 active jobs tracked by LiftmyCV. 17 new jobs in the past 7 days.
Active openings
34
Currently tracked by LiftmyCV
New in 7 days
17
31 new jobs in the past 30 days
Published salaries
68%
Median disclosed salary: $245,000
Average opening age
10 days
Across 34 active openings
Workplace breakdown
Hiring countries
Top hiring cities
Most in-demand roles
- • Applied Research Scientist Intern, Agentic Biology - Summer 2027
- • Business Controls Specialist
- • Developer Relations Manager, Federal Cybersecurity
- • DriveOS Documentation Technical Program Manager
- • Enterprise Sales Intern - Summer 2027
- • Firmware System Integration Engineer
Frequently requested skills
Median disclosed salary: $245,000 · 68% of jobs publish a salary
As of November 18, 2026
Hiring insights calculated from active jobs tracked by LiftmyCV.
See how well you match this job
Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.
Sign up to see your matchAlready have an account? Log inMore at NVIDIA USA
Similar jobs from this company
- Senior Software Engineer - GPUNVIDIA USASanta Clara, USOn-site$152,000–$241,500 per year
- Senior Manager, Software Engineering - Rack Scale Systems InfrastructureNVIDIA USASanta Clara, USOn-site$248,000–$391,000 per year
- Senior Software Engineer, Agentic EngineeringNVIDIA USASanta Clara, USOn-site
- Senior Cloud Software Engineer, Developer ToolsNVIDIA USASanta Clara, USOn-site
- Differentiable Simulation Research Intern - Summer 2027NVIDIA USASanta Clara, USOn-site$79,200–$195,360 per year
- Senior CPU Technical Product Marketing Manager - Accelerated ComputingNVIDIA USASanta Clara, USOn-site$148,000–$287,500 per year
Across the market
Similar jobs from other companies
- Building Self-Improving Agents EngineercareerMountain View, USOn-site
- Self-Directed Occupancy EvaluatorFAR InspectionsGallipolis, USOn-site
- Performance Improvement Evaluator IICareSource Management Services LLCRemoteRemote
- Researcher, Recursive Self-Improvement SafetyOpenAISan Francisco, USOn-site
- Employer Brand Specialist - Self-ImprovementSKELARWarsaw, PL
- Research Engineer / Scientist, Self-ImprovementLettaSan Francisco OfficeOn-site
Third-party content disclaimer
Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.
LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.
LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.
If you prefer, you may apply directly via the original listing for transparency.