LiftmyCVLiftmyCV

AI Agent Evaluator

Anyone AIRemoteAR
AI job summary

Join Anyone AI as an AI Agent Evaluator, where you'll assess how well AI assistants understand users and provide meaningful responses. This remote, part-time role involves having conversations with two AI versions and comparing their effectiveness based on your personal context and prior activity. You'll write prompts, engage in interactive dialogues, and provide feedback to enhance AI capabilities. Ideal candidates should be fluent in one of the supported languages, possess strong comprehension skills, and have recent experience with digital services. The position is project-based, lasting up to three months.

Job description

Help shape the future of AI by evaluating how well AI assistants understand users, use personal context, and deliver helpful conversations.

  • Location: Remote
  • Type: Independent contractor / Part-time
  • Project duration: up to 3 months

About the role

At Anyone AI, we create high-quality training and evaluation data for frontier AI models. We’re looking for experts to evaluate personalized AI conversations using information available through connected digital services and prior activity. In this role, you’ll have natural, back-and-forth conversations with two versions of an AI assistant and compare your experiences. Your feedback will help assess how effectively each assistant uses relevant information, understands your context, and responds to your needs.

What you’ll do

  • Write prompts about questions, interests, or needs relevant to you.
  • Have natural, multi-turn conversations with two AI assistants.
  • Include questions that draw on information or activity from your connected services, where relevant.
  • Compare how effectively each assistant uses available information and connects relevant context.
  • Assess whether responses feel helpful, natural, trustworthy, and safe.
  • Evaluate how each assistant responds to your feedback and corrections.
  • Provide clear explanations supporting your evaluations.

What we’re looking for

  • Fluency in one of the supported languages/locales listed below.
  • Strong reading comprehension and the ability to explain your feedback clearly.
  • Attention to detail and sound judgment when comparing conversation experiences.
  • Willingness to interact with AI assistants using relevant personal context and connected services.
  • Recent activity across a qualifying combination of digital services.
  • Completion of the project’s eligibility verification and corresponding expert requirements.

Supported languages/locales

  • Spanish — Latin America
  • Arabic
  • Portuguese — Brazil
  • Japanese — Japan
  • Turkish — Turkey
  • Indonesian — Indonesia
  • Hindi — India
  • Korean — Korea
  • French — France
  • German — Germany

Eligibility & verification

Before participating, you’ll complete a brief verification to confirm whether the digital services you currently use—and your recent activity across those services—meet the project’s requirements. Please answer all verification questions accurately based on your actual, current usage. Provide complete information about the services you actively use, as different combinations of qualifying activity may be considered. Meeting the required activity threshold is necessary for participation. Completing verification does not guarantee placement. Experts who do not qualify for this project may still be considered for other AI evaluation opportunities.

Nice to have

  • Experience using AI assistants for everyday questions, planning, or research.
  • Previous experience in AI evaluation, user research, or quality assurance.
  • Ability to identify subtle differences in relevance, personalization, and conversation quality.

Job details

Published
October 12, 2026
Last checked
November 17, 2026
Job source
ashby
Workplace
Remote · AR
First seen
October 12, 2026

Skills mentioned in this job

AI evaluationuser researchquality assurance

LiftmyCV exclusive data

Anyone AI hiring intelligence

Anyone AI currently has 44 active jobs tracked by LiftmyCV. 2 new jobs in the past 7 days.

Verified November 17, 2026

Active openings

44

Currently tracked by LiftmyCV

New in 7 days

2

27 new jobs in the past 30 days

Published salaries

57%

Median disclosed salary: $93,600

Average opening age

24 days

Across 44 active openings

Workplace breakdown

Remote100%
Hybrid0%
On-site0%

Hiring countries

Argentina
Brazil
Colombia
Ecuador
Mexico
Chile
Estonia
Peru

Most in-demand roles

  • • Backend Developer
  • • Quality Evaluator – Fluent English
  • • Academic Program Lead, AI Fellowship
  • • Accountant (ALE)
  • • Accounting Expert
  • • Admissions Advisor

Frequently requested skills

Python
TypeScript
C#
Go
Java
JavaScript
Jest
JUnit

Median disclosed salary: $93,600 · 57% of jobs publish a salary

As of November 17, 2026

Hiring insights calculated from active jobs tracked by LiftmyCV.

AI match explained

See how well you match this job

Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.

Sign up to see your matchAlready have an account? Log in

More at Anyone AI

Similar jobs from this company

Across the market

Similar jobs from other companies

Third-party content disclaimer

Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.

LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.

LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.

If you prefer, you may apply directly via the original listing for transparency.