
Expert Talent Pool, Evals and Benchmarks
Clera seeks senior evaluation experts to join a network supporting AI research projects. Responsibilities include leading benchmark design and validation, developing scoring methods, and conducting rigorous model comparison studies. Candidates should have at least 5 years of experience in evaluation or domain expertise related to assessment and be proficient in relevant tools such as Python. The role is fully remote, offering flexible project-based work. Competitive hourly compensation ranges from $75 to $175 based on the project and experience level.
Job description
About the Role
Join a network of senior evaluation experts supporting AI research projects. You will lead benchmark design, validation, and grading operations when projects align with your expertise, helping produce rigorous and reliable assessments of AI models.
What You'll Do
- Design benchmarks, including their scope, question types, difficulty levels, and scoring methods.
- Write and review rubrics to support consistent grading.
- Develop gold-standard answers and test sets that hold up to expert review.
- Lead grading rounds by calibrating graders, measuring agreement, and resolving disputes.
- Analyze model failures, identify root causes, and recommend improvements.
- Design and conduct rigorous model comparison studies.
- Validate benchmarks for contamination, ambiguity, and saturation.
- Design user studies on how people rate and interact with model outputs.
What We're Looking For
- At least 5 years of experience in evaluation, testing, or assessment, or at least 5 years of domain expertise combined with hands-on evaluation work.
- Experience designing a benchmark, test plan, exam, or assessment used by others.
- Experience writing or reviewing rubrics and gold-standard answers, and reviewing or leading graders or question writers.
- Proficiency in Python and evaluation tools such as Inspect, lm-eval-harness, HELM, or promptfoo.
- Experience analyzing model or test failures and recommending fixes.
- Background in ML evaluation, QA, test engineering, psychometrics, assessment design, UX research, data science, or deep subject matter expertise.
- Clear written English communication.
- Experience validating benchmarks, measuring grader agreement, conducting model comparisons, or running user studies is relevant.
Compensation & Benefits
Compensation is $75 to $175 per hour, depending on project, field, and experience level. Pay is based on hours worked.
Location
Fully remote. Projects may last from one week to several months.
Job details
- Published
- November 15, 2026
- Job source
- ashby
- Workplace
- Remote · remote
- First seen
- November 15, 2026
Skills mentioned in this job
LiftmyCV exclusive data
Clera hiring intelligence
Clera currently has 77 active jobs tracked by LiftmyCV. 77 new jobs in the past 7 days.
Active openings
77
Currently tracked by LiftmyCV
New in 7 days
77
77 new jobs in the past 30 days
Published salaries
61%
Median disclosed salary: $119,600
Average opening age
1 days
Across 77 active openings
Workplace breakdown
Hiring countries
Top hiring cities
Most in-demand roles
- • Account Executive
- • Deployment Strategist
- • Founding GTM Lead
- • Founding Harness Engineer
- • Founding Product Engineer
- • Founding Software Engineer
Frequently requested skills
Median disclosed salary: $119,600 · 61% of jobs publish a salary
As of November 17, 2026
CLERA GmbH · ACTIVE
Hiring insights calculated from active jobs tracked by LiftmyCV.
Company information from GLEIF.
See how well you match this job
Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.
Sign up to see your matchAlready have an account? Log inMore at Clera
Similar jobs from this company
- Account ExecutiveCleraChicago, USRemote$160,000–$200,000 per year
- Enterprise Account ExecutiveCleraMunich, DE
- Founding Electrical EngineerCleraSan Francisco, US$160,000–$200,000 per year
- Founding Customer Success ManagerCleraBerlin, DE$64,500–$75,500 per year
- Founding Account ExecutiveCleraBerlin, DE
- Founder's Associate Intern (GTM)CleraBerlin, DE
Across the market
Similar jobs from other companies
- AI Evals/Benchmarking EngineerActivate Interactive Pte LtdSingapore, SGOn-site
- Research Scientist, Benchmarks & EvalsKled AIUSRemote$150,000–$250,000 per year
- Global benchmarking expertNovartis Healthcare Private LimitedHyderabad (Office)Hybrid
- Talent PoolLaba GroupWarsaw, PLOn-site
- Talent PoolWINGS ICT SOLUTIONS S.A.Athens, GROn-site
- Talent PoolKasamaTELECOMMUTE, PhilippinesRemote
Third-party content disclaimer
Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.
LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.
LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.
If you prefer, you may apply directly via the original listing for transparency.