
Multilingual Audio Evaluator
Join Anyone AI as a Multilingual Audio Evaluator, contributing to AI development by assessing conversational AI agents across various languages. This role involves live roleplay scenarios, capturing technical logs, and providing evaluations according to structured guidelines. Ideal candidates will have fluency in one or more specified languages and strong skills in JSON. The position is project-based with a duration of 2-3 months, starting on October 9, 2026. This is a remote opportunity, perfect for those with attention to detail and a passion for AI.
Job description
Help shape the future of AI by evaluating how conversational AI agents communicate and perform across different languages.
- Location: Remote / Global, subject to project location restrictions
- Type: Independent contractor / Project-based
- Commitment: To be confirmed
- Compensation: To be confirmed
- Project duration: 2–3 months
- Project start date: October 9, 2026
About the role
At Anyone AI, we create high-quality training and evaluation data for frontier AI models. We’re looking for multilingual audio evaluators with strong spoken-language proficiency and technical literacy to assess conversational AI agents. In this role, you’ll participate in live, real-time roleplay scenarios, capture session recordings and technical logs, label conversation transcripts, and evaluate interactions using structured rubrics. Your feedback will help improve the quality and reliability of AI conversations across languages.
What you’ll do
- Prepare and confirm the required tools and access before each live session.
- Conduct live, real-time roleplay scenarios with conversational AI agents.
- Capture recordings, session IDs, and tool-call or console logs.
- Label conversation transcripts according to project guidelines.
- Score interactions against structured evaluation rubrics.
- Provide clear written explanations supporting your scores.
- Read, validate, and troubleshoot JSON, identifying structural and formatting issues.
What we’re looking for
- C1/C2 spoken fluency in at least one of the target languages listed below.
- Ability to demonstrate language proficiency through a speaking and listening assessment.
- Strong JSON skills, including reading, validation, troubleshooting, and identifying structural or formatting errors.
- Comfort using digital tools and collecting recordings and technical logs.
- Strong attention to detail and the ability to follow evaluation guidelines consistently.
- Clear written communication and sound judgment when assessing AI interactions.
Languages we’re hiring for
- Welsh
- Dutch
- Greek
- Italian
- French
- German
- Portuguese
Nice to have
- Experience in AI evaluation, audio annotation, or transcript labeling.
- A background in technical support, quality assurance, or software development.
- Familiarity with conversational AI, APIs, and structured data.
Responsibilities
Prepare and confirm the required tools and access before each live session. Conduct live, real-time roleplay scenarios with conversational AI agents. Capture recordings, session IDs, and tool-call or console logs. Label conversation transcripts according to project guidelines. Score interactions against structured evaluation rubrics. Provide clear written explanations supporting your scores. Read, validate, and troubleshoot JSON, identifying structural and formatting issues.
Job details
- Published
- October 12, 2026
- Last checked
- November 17, 2026
- Job source
- ashby
- Workplace
- Remote · AR
- First seen
- October 12, 2026
Requirements
C1/C2 spoken fluency in at least one of the target languages listed below. Ability to demonstrate language proficiency through a speaking and listening assessment. Strong JSON skills, including reading, validation, troubleshooting, and identifying structural or formatting errors. Comfort using digital tools and collecting recordings and technical logs. Strong attention to detail and the ability to follow evaluation guidelines consistently. Clear written communication and sound judgment when assessing AI interactions.
Skills mentioned in this job
LiftmyCV exclusive data
Anyone AI hiring intelligence
Anyone AI currently has 44 active jobs tracked by LiftmyCV. 2 new jobs in the past 7 days.
Verified November 17, 2026
Active openings
44
Currently tracked by LiftmyCV
New in 7 days
2
27 new jobs in the past 30 days
Published salaries
57%
Median disclosed salary: $93,600
Average opening age
24 days
Across 44 active openings
Workplace breakdown
Hiring countries
Most in-demand roles
- • Backend Developer
- • Quality Evaluator – Fluent English
- • Academic Program Lead, AI Fellowship
- • Accountant (ALE)
- • Accounting Expert
- • Admissions Advisor
Frequently requested skills
Median disclosed salary: $93,600 · 57% of jobs publish a salary
As of November 17, 2026
Hiring insights calculated from active jobs tracked by LiftmyCV.
See how well you match this job
Log in or sign up and Eva will compare your profile with this role, explain your strengths and show what to improve.
Sign up to see your matchAlready have an account? Log inMore at Anyone AI
Similar jobs from this company
- Quality Evaluator – EnglishAnyone AIARRemote$14,560–$24,960 per year
- Quality EvaluatorAnyone AIPLRemote$14,560–$24,960 per year
- Quality Evaluator – Fluent English (Mexico)Anyone AIFully Remote, MXRemote
- Quality Evaluator – Fluent EnglishAnyone AIARRemote$14,560–$24,960 per year
- AI Coding EvaluatorAnyone AIARRemote
- Quality Evaluator – Fluent English (Brazil)Anyone AIBRRemote$14,560–$24,960 per year
Across the market
Similar jobs from other companies
- Generative Audio Evaluation SpecialistRWS TrainAIWarsaw, PLRemote$22,880 per year
- Multilingual Software Engineering & AI Evaluation SpecialistGramian Consulting GroupRURemote
- Generative Audio Evaluation - Portuguese (Portugal)RWS TrainAILisbon, PTRemote$16,640 per year
- Generative Audio Evaluation - Swedish (Sweden)RWS TrainAIStockholm, SERemote
- Generative Audio Evaluation - Dutch (The Netherlands)RWS TrainAIAmsterdam, NLRemote
- Generative Audio Evaluation - Spanish (Spain)RWS TrainAIMadrid, ESRemote$24,960 per year
Third-party content disclaimer
Full job descriptions are sourced directly from third-party applicant tracking systems (ATS), job boards, recruitment platforms, and employer career pages.
LiftmyCV does not claim ownership of this content or guarantee its accuracy or current availability. All original content is the intellectual property of the respective employers or other rights holders.
LiftmyCV hiring intelligence, availability checks, and related-job recommendations are generated from LiftmyCV data and public sources.
If you prefer, you may apply directly via the original listing for transparency.