name

Data Scientist - AI Evaluation

United States full-time Mid $100 - $150
full-time Mid level Technology & IT Salary listed Curated
Sign in to apply Free account — we bring you straight back to this role.

About the role

About the job
Mercor connects elite creative and technical talent with leading AI research labs. Headquartered in San Francisco, our investors include Benchmark, General Catalyst, Peter Thiel, Adam D'Angelo, Larry Summers, and Jack Dorsey.
Position: Data Scientist Talent Network
Type:Contract
Compensation:$100–$150/hour
Location:Remote
Role Responsibilities

Design precise, task-specific grading criteria for data science deliverables, including exploratory data analyses, statistical modeling work, machine learning pipelines, and A/B test write-ups.

Evaluate AI-generated or human-created work against established criteria to ensure quality and consistency.

Provide detailed written justifications for evaluations and scores to maintain transparency and accountability.

Apply consistent, evidence-based judgment to ensure assessments are reproducible and defensible.

Incorporate structured feedback from senior reviewers and iterate on submitted work for continuous improvement.

Work independently and asynchronously to meet deadlines and improve AI model performance.

Qualifications

Must-Have

1+ years of professional data science experience.

Experience at a leading technology, research, or quantitative firm.

Strong command of Python, SQL, statistical modeling, machine learning, and experimentation.

Exceptional written communication skills.

Detail-oriented and consistent approach to evaluating complex work.

Comfort receiving feedback and calibrating judgment against established standards.

Application Process (Takes 20–30 mins to complete)

Upload resume

AI interview based on your resume

Submit form

Resources & Support

For details about the interview process and platform information, please check:

For any help or support, reach out to:

PS: Our team reviews applications daily. Please complete your AI interview and application steps to be considered for this opportunity.
Originally posted on Himalayas

Interview prep

Walk in with sharper answers.

Use this as a quick practice sheet before you speak with the employer.

Mid
Technology & IT MySQL Python Remote Collaboration SQL Mid level full-time

Likely questions

  1. Tell us about work you have done that is close to the Data Scientist - AI Evaluation role.
  2. How would you approach your first 30 days at name?
  3. Which of MySQL, Python and Remote Collaboration have you used recently, and what did it help you achieve?
  4. Describe a time you solved a problem without waiting to be told exactly what to do.
  5. How do you handle busy days, changing priorities, or pressure at work?

Prepare before the call

  • A recent example that proves your experience with MySQL, Python and Remote Collaboration.
  • One short story with a problem, your action, and the result.
  • Two examples that show the strengths listed on your CV.
  • A clear reason why this role and company interest you.
  • Your availability, preferred work style, and salary expectations.

Ask them

  • What would success look like in the first 90 days?
  • What are the main problems this hire should help solve?
  • How does the team give feedback and measure good work?
  • What does a normal working week look like for this role?
Practice line

I am interested in the Data Scientist - AI Evaluation role because I can bring practical experience in MySQL, Python and Remote Collaboration, learn the team quickly, and contribute to the outcomes name needs from this hire.

Related jobs.

More roles from this company or category.