Japanese Language Evaluation Expert (Freelance)

About the role

A language evaluation project is looking for Japanese-speaking experts to help evaluate AI-generated search answers. The role is practical and execution-focused, based on hands-on testing rather than theoretical research. It is a freelance contract, remote, starting as soon as possible, with a commitment of 20 hours per week.

Key responsibilities

• Identify realistic search-related failures where an AI model gives inadequate responses.
• Create search queries, test them and document what went wrong.
• Write instructions and evaluation criteria for correct answers.
• Focus on common issues such as local relevance, source quality, outdated information, language mismatches, incomplete answers and region-specific inapplicability.

Required profile

• Research skills: finding and verifying information across websites, databases and documentation.
• Critical thinking: spotting contradictions, outdated information and context mismatches.
• Attention to detail: catching nuances in language, local variations and edge cases.
• Problem-solving mindset: comfortable iterating and testing through trial and error.
• Writing clarity: able to explain the logic behind evaluation instructions and rubrics.
• Deep familiarity with one's own locale (regulations, business practices, cultural nuances) is valuable.
• Ideally a background in research, quality assurance, compliance or similar work requiring precision.

What we offer

• Freelance contract, remote, starting ASAP.
• Rate of $35-57.75 per hour.