
Member of Technical Staff, Evals
Job description
Magic’s mission is to build safe AGI that accelerates humanity’s progress on the world’s most important problems. We believe the most promising path to safe AGI lies in automating research and code generation to improve models and solve alignment more reliably than humans can alone. Our approach combines frontier-scale pretraining, domain-specific RL, ultra-long context, and inference-time compute to achieve this goal.
About the role
The Evals team measures the performance and capabilities of internal and external models, helping teams across Magic make informed decisions. The team supports pretraining, RL, data, inference, and product, and sits on the critical path of many of the company's most important decisions. As a Member of Technical Staff on Evals, you will develop new benchmarks and evaluation methodologies, run data ablations, assess dataset quality, and investigate whether our measurements accurately reflect model performance. Sweating the details matters on this team. Many benchmarks, papers, and open-source evaluation frameworks contain subtle bugs or flawed assumptions that lead to misleading conclusions. We care deeply about correctness, reproducibility, and measurement quality.
What you'll work on
- Design and validate new evaluations and benchmarks across model training and deployment
- Investigate model behavior and identify gaps in existing evaluations
- Measure dataset quality and identify opportunities to improve training data
- Improve evaluation correctness, reproducibility, and reliability
- Audit and improve upon public benchmarks, research papers, and open-source implementations
- Define metrics that accurately reflect model quality
- Build tools that make evaluation results accessible and useful across Magic
What we're looking for
- Experience developing model evaluations, benchmarks, or ML experiments
- Strong programming skills and experience working with machine learning systems
- Track record of owning technical projects end-to-end
- Skepticism toward results that cannot be reproduced, validated, or explained
- Experience analyzing model performance and interpreting experimental results
- Comfortable navigating ambiguity and determining whether a measurement actually captures the behavior it claims to measure
- Commitment to raising the standard of rigor in AI evaluations
Our culture
- Integrity. Words and actions should be aligned
- Hands-on. At Magic, everyone is building
- Teamwork. We move as one team, not N individuals
- Focus. Safely deploy AGI. Everything else is noise
- Quality. Magic should feel like magic
Magic strives to be the place where high-potential individuals can do their best work. We value quick learning and grit just as much as skill and experience.
Compensation, benefits, and perks (US)
- Annual salary range between $200K - $550K depending on experience
- Equity is a significant part of total compensation, in addition to salary
- 401(k) plan with 6% salary matching
- Generous health, dental, and vision insurance for you and your dependents
- Unlimited paid time off
- Visa sponsorship and relocation support for candidates moving to San Francisco
- A small, fast-moving, highly collaborative team working on frontier AI systems
Job details
- Published
- November 11, 2026
- Last checked
- November 17, 2026
- Experience
- Staff
- Job source
- ashby
- Workplace
- Remote · San Francisco, US
- First seen
- November 11, 2026
LiftmyCV exclusive data
Magic, Inc hiring intelligence
Magic, Inc currently has 3 active jobs tracked by LiftmyCV. 3 new jobs in the past 7 days.
Verified November 17, 2026
Active openings
3
Currently tracked by LiftmyCV
New in 7 days
3
3 new jobs in the past 30 days
Published salaries
67%
Median disclosed salary: $11,440
Average opening age
2 days
Across 3 active openings
Workplace breakdown
Hiring countries
Top hiring cities
Most in-demand roles
- • Account Manager
- • Member of Technical Staff, Evals
- • Program Account Coordinator
Frequently requested skills
Median disclosed salary: $11,440 · 67% of jobs publish a salary
As of November 18, 2026
Hiring insights calculated from active jobs tracked by LiftmyCV.
See how well you match this job
Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.
Sign up to see your matchAlready have an account? Log inVisa sponsorship insights
Visa sponsorship available
This job posting states that visa sponsorship is available for qualified candidates.
More at Magic
Similar jobs from this company
- Founder Content Creator (Shoot + Edit)MAGICLondon, GBHybrid
- Social First VideographerMAGICLondon, GBHybrid
- On-Camera Fitness InstructorMAGICNew York City, USOn-site
- Head of Partnerships (B2B & Retail)MAGICLondon, GBHybrid
- Growth & Brand DesignerMAGICLondon, GBHybrid
- Account ManagerMagic, IncTaguig City, PHRemote$12,480 per year
Across the market
Similar jobs from other companies
Related job searches
Third-party content disclaimer
Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.
LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.
LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.
If you prefer, you may apply directly via the original listing for transparency.