LiftmyCVLiftmyCV

Expert Talent Pool, Evals and Benchmarks

CleraRemoteremote$156,000–$364,000 per year
AI job summary

Clera seeks senior evaluation experts to join a network supporting AI research projects. Responsibilities include leading benchmark design and validation, developing scoring methods, and conducting rigorous model comparison studies. Candidates should have at least 5 years of experience in evaluation or domain expertise related to assessment and be proficient in relevant tools such as Python. The role is fully remote, offering flexible project-based work. Competitive hourly compensation ranges from $75 to $175 based on the project and experience level.

Job description

About the Role

Join a network of senior evaluation experts supporting AI research projects. You will lead benchmark design, validation, and grading operations when projects align with your expertise, helping produce rigorous and reliable assessments of AI models.

What You'll Do

  • Design benchmarks, including their scope, question types, difficulty levels, and scoring methods.
  • Write and review rubrics to support consistent grading.
  • Develop gold-standard answers and test sets that hold up to expert review.
  • Lead grading rounds by calibrating graders, measuring agreement, and resolving disputes.
  • Analyze model failures, identify root causes, and recommend improvements.
  • Design and conduct rigorous model comparison studies.
  • Validate benchmarks for contamination, ambiguity, and saturation.
  • Design user studies on how people rate and interact with model outputs.

What We're Looking For

  • At least 5 years of experience in evaluation, testing, or assessment, or at least 5 years of domain expertise combined with hands-on evaluation work.
  • Experience designing a benchmark, test plan, exam, or assessment used by others.
  • Experience writing or reviewing rubrics and gold-standard answers, and reviewing or leading graders or question writers.
  • Proficiency in Python and evaluation tools such as Inspect, lm-eval-harness, HELM, or promptfoo.
  • Experience analyzing model or test failures and recommending fixes.
  • Background in ML evaluation, QA, test engineering, psychometrics, assessment design, UX research, data science, or deep subject matter expertise.
  • Clear written English communication.
  • Experience validating benchmarks, measuring grader agreement, conducting model comparisons, or running user studies is relevant.

Compensation & Benefits

Compensation is $75 to $175 per hour, depending on project, field, and experience level. Pay is based on hours worked.

Location

Fully remote. Projects may last from one week to several months.

Job details

Published
November 15, 2026
Job source
ashby
Workplace
Remote · remote
First seen
November 15, 2026

Skills mentioned in this job

Python

LiftmyCV exclusive data

Clera hiring intelligence

Clera currently has 77 active jobs tracked by LiftmyCV. 77 new jobs in the past 7 days.

Active openings

77

Currently tracked by LiftmyCV

New in 7 days

77

77 new jobs in the past 30 days

Published salaries

61%

Median disclosed salary: $119,600

Average opening age

1 days

Across 77 active openings

Workplace breakdown

Remote100%
Hybrid0%
On-site0%

Hiring countries

Germany
United States of America
Austria
Switzerland

Top hiring cities

Berlin
San Francisco
Munich
New York
Vienna
Hamburg
Karlsruhe

Most in-demand roles

  • • Account Executive
  • • Deployment Strategist
  • • Founding GTM Lead
  • • Founding Harness Engineer
  • • Founding Product Engineer
  • • Founding Software Engineer

Frequently requested skills

Python
TypeScript
React
AWS
AI agents
Azure
CVAT
GCP

Median disclosed salary: $119,600 · 61% of jobs publish a salary

As of November 17, 2026

CLERA GmbH · ACTIVE

Hiring insights calculated from active jobs tracked by LiftmyCV.

Company information from GLEIF.

AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Clera

Similar jobs from this company

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.