name

AI Workflow Evaluator - Fully Remote | Upto $190/hr

Remote, United States remote Entry $50 - $190
remote Technology & IT Salary listed Curated
Sign in to apply Free account — we bring you straight back to this role.

About the role

About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: MCP & Plug In Connectors Expert
Type:Contract
Compensation:$50–$190/hour
Location:Remote
Commitment:20+ hours/week
Role Responsibilities

Evaluate AI systems on complex personal workflows, including personal health, travel, and activity planning.

Design realistic prompts for complex personal-life tasks using Google Drive, Expedia, Notion, and other plugins/connectors.

Execute tasks while recording your screen and using personal plugins/connectors.

Write clear explanations of AI successes and failures to improve AI model performance.

Judge AI outputs for practicality, personalization, and reasoning.

Identify where models miss context, overreach, or fail to use tools correctly.

Qualifications

Must-Have

US-based only.

Strong MCP experience and plugin/connector usage.

Experience using LLM plugins/connectors like Google Drive, Expedia, and Notion.

Heavy personal usage of LLM products.

Active LLM account with 6+ months of history.

Willingness to sign a data-share consent form via DocuSign.

Experience using AI for multi-step planning, research, and decision-making.

Strong written judgment and attention to detail.

Ability to evaluate AI outputs and write against rubrics.

Preferred
Extensive rubric experience with 100+ hours on prior projects.
Compensation & Legal

Hourly contractor.

Experts added to the project will begin in a trial period.

Application Process (Takes 20–30 mins to complete)

Upload resume

AI interview based on your resume

Submit form

Resources & Support

For details about the interview process and platform information, please check:

For any help or support, reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Originally posted on Himalayas

Interview prep

Walk in with sharper answers.

Use this as a quick practice sheet before you speak with the employer.

Role
Technology & IT Human Resources Remote Collaboration Workflow Evaluator Fully remote

Likely questions

  1. Tell us about work you have done that is close to the AI Workflow Evaluator - Fully Remote | Upto $190/hr role.
  2. How would you approach your first 30 days at name?
  3. Which of Human Resources, Remote Collaboration and Workflow have you used recently, and what did it help you achieve?
  4. Describe a time you solved a problem without waiting to be told exactly what to do.
  5. How do you stay organised and communicate clearly when working remotely?

Prepare before the call

  • A recent example that proves your experience with Human Resources, Remote Collaboration and Workflow.
  • One short story with a problem, your action, and the result.
  • Two examples that show the strengths listed on your CV.
  • A clear reason why this role and company interest you.
  • Your availability, preferred work style, and salary expectations.

Ask them

  • What would success look like in the first 90 days?
  • What are the main problems this hire should help solve?
  • How does the team give feedback and measure good work?
  • What does a normal working week look like for this role?
Practice line

I am interested in the AI Workflow Evaluator - Fully Remote | Upto $190/hr role because I can bring practical experience in Human Resources, Remote Collaboration and Workflow, learn the team quickly, and contribute to the outcomes name needs from this hire.

Related jobs.

More roles from this company or category.