About the role
Role Title: AI Rubric Writer
Role Type: Contractor
Location: Remote
We are seeking an experienced AI Rubric Writer to develop high-quality evaluation rubrics and scoring frameworks for AI and Large Language Model (LLM) assessments. The ideal candidate will create clear, objective evaluation criteria to assess AI-generated responses and help improve model performance across a variety of subject areas.
Responsibilities
Design and develop evaluation rubrics and scoring guidelines for AI-generated content.
Create objective, consistent, and well-structured assessment criteria.
Evaluate AI responses for accuracy, reasoning, completeness, and quality.
Provide detailed feedback to improve AI model performance.
Collaborate with cross-functional teams to refine evaluation standards.
Ensure rubrics are clear, unbiased, and easy to apply.
Required Qualifications
Bachelor's degree or higher in a relevant field.
Strong analytical, critical thinking, and written communication skills.
Experience creating evaluation criteria, scoring frameworks, assessment guidelines, or rubrics.
Subject matter expertise in Software Engineering, Finance, Law, Consulting, or other quantitative/STEM disciplines.
Preferred Qualifications
Experience as an AI Evaluator, Prompt Evaluator, or similar AI quality assessment role.
Experience evaluating Large Language Models (LLMs) or Generative AI systems.
Familiarity with rubric development, AI quality assurance, or data annotation.
Experience reviewing and assessing technical or domain-specific content.
Originally posted on Himalayas