Find your next job. Start today.

Direct from employer

AI Benchmarking Specialist, Performance Benchmarking Evaluation

Date Posted: Sep 30, 2026
Apply on company website

Job Detail

  • location_on
    Location Hyderabad, Telangana, India
  • desktop_windows
    Job Type: Full Time/Permanent
  • schedule
    Shift:
  • analytics
    Career Level:
  • group
    Positions:
  • calendar_view_day
    Experience:
  • male
    Gender: No Preference
  • school
    Degree:
  • calendar_month
    Apply Before: Nov 15, 2026
Email to Friend Add to Favourite Report Abuse

Job Description

At Amazon, we're evolving how to support sellers, customers in solving their query resolved faster using AI. That is where our team steps in, we support multiple AI models in benchmarking their responses for it relevancy, correctness and completeness.
- You perform straightforward annotation related tasks across ML data process areas. You are able to determine the right task or procedure (or when a slight deviation is needed) to achieve desired outcome.
- You act as point of contact for one or more simple annotation tasks in a given ML data process area. You are responsible for the quality of execution for these tasks.
- You understand the dependencies of tasks across ML data workflows and are able to articulate customer impact. You use the existing annotation methods and modify the SOPs. You document the changes in SOP and once approved, you share the knowledge with the team, audit SOP adoption and execution.
- You test new SOP/tools and share feedback on their quality and provide inputs for improvement to support on boarding of new tools.

Key job responsibilities
- You structure data collection, analyse results and share inputs for SOP changes.
- You collate, track, and report progress on key metrics agreed to with respective stakeholders (e.g., Program managers, Applied Scientist) specific to your functional area.
- You identify operational issues related to process and tooling and recommend suggestions to improve key project metrics such as productivity and quality.

Basic qualifications

- Bachelor's degree
- Speak, write, and read fluently in English
- Experience with Microsoft Office products and applications
- Experience with Excel

Preferred qualifications

- 0 to 1 years of equivalent experience
- Performed annotation related tasks across ML data process areas.
- Technical proficiency in SQL querying and Python programming for data analysis

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Source: Amazon careers — Read the original posting and apply

This role is listed by the employer on its own careers site. Kaam Ki Khoj does not process applications for it.

Company Overview

Amazon
Amazon · India
1,727 open roles

Amazon openings shown on Kaam Ki Khoj are read directly from the company's own careers site and link back to it. Apply there; Kaam Ki Khoj does not process applications for these roles and is not affiliated with Amazon. Read More

Related Jobs

One upload, every employer

Let the right employer find you

Upload your CV once — we read it, build your profile and put you in front of every employer hiring on Kaam Ki Khoj. No forms, no fees.

Upload your CV Browse jobs