Research Engineer - Evals
AGI, Inc. is seeking a Research Engineer specializing in evaluations. This role is crucial for enhancing human-AI collaboration through trustworthy and effective agents. The successful candidate will be responsible for designing an eval harness that monitors model capability, agent behavior, and end-user experience. Responsibilities include developing eval suites, optimizing researcher tools, and ensuring product readiness. Ideal candidates will thrive in a fast-paced environment and contribute meaningfully to AI research, impacting the company's trajectory in everyday AGI.
Job description
Think Different. Build the Future.
Our Mission
Build everyday AGI. Trustworthy, consumer-grade agents that redefine human–AI collaboration for millions. Software shouldn’t wait for commands; it should partner with you, amplifying what you can do every single day.
Why AGI, Inc.
We’re a stealth team of elite founders and AI researchers, with backgrounds spanning Stanford, OpenAI, and DeepMind. We’re industry leaders in mobile and computer-use agents, bringing these capabilities to consumer scale. Grounded in years of agent research, our AI is designed with trustworthiness and reliability as core pillars, not afterthoughts. We are supported by tier-1 investors who funded the first generation of AI giants; now they’re backing us to build the next: everyday AGI. If you see possibility where others see limits, read on. Models, agents, and product features all ship behind one question: did this actually get better? Without a strong evals function, the lab ships vibes. With one, every training run, every prompt change, every agent capability moves a number we trust — and the team makes decisions on real signal, not the loudest opinion in the room. You'll build the eval harness for AGI — across model capability, agentic behavior, on-device performance, and end-user experience. You'll set the bar for what counts as "shipped" and protect it from the gravity of product deadlines.
Tasks you will own
- The eval suites that gate every model and agent release — capability, behavior, regressions, and human-rated rubrics that catch what automated evals miss
- The dashboards and tooling that make researcher experiment loops fast and leadership decisions easy
- The bar — what counts as ready to ship, and how we know
Areas where you will assist
- Research, by making sure what we measure is what we want
- Product engineers, by instrumenting real-user behavior on real devices
- Partnerships, by translating "did it get better" into language an OEM partner can hold us to
Skills you'll be expected to teach
- How to measure non-deterministic systems — agent eval, tool use, long-horizon tasks, multilingual behavior
- How to push back on a metric that's being gamed without breaking the team
Skills you'll be expected to learn
- On-device perf trade-offs and how they show up in real-user evals
- What QA-ing AI at OEM scale actually looks like
- The realities of shipping consumer agents to production partners
Timeline of success
After 30 days — You've audited every eval we run today and produced a sharp doc on what's good, what's noise, and what's missing. You've fixed the most embarrassing gap. After 60 days — You've stood up a new eval surface — agentic, on-device, or behavioral — and the team is making real decisions on its output. Researchers come to you before launching a run, not after. After 90 days — Releases now ship against your eval bar, not a vibe-check. You've caught a regression that would have shipped, and cleared a launch the team was nervous about. You're shaping the research roadmap by surfacing where we're flat, where we're climbing, and where we're lying to ourselves.
Compensation
Competitive cash and meaningful equity. Top-tier relocation and immigration support. SF, in person.
Job details
- Published
- May 27, 2026
- Last checked
- November 17, 2026
- Job source
- ashby
- Workplace
- On-site · San Francisco, US
- First seen
- May 27, 2026
See how well you match this job
Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.
Sign up to see your matchAlready have an account? Log inMore at AGI, Inc.
Similar jobs from this company
None listed.
Across the market
Similar jobs from other companies
Third-party content disclaimer
Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.
LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.
LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.
If you prefer, you may apply directly via the original listing for transparency.