Find your next job. Start today.

From the employer's own careers site

Test Analyst - AI Assurance

Date Posted: Nov 11, 2026
Apply on company website
Sign in to keep track of your applications.
Get jobs like this sent to you
Your own AI Job Agent checks every new job against your CV and sends only the ones that fit, on WhatsApp. ₹199 a month, GST included. Full refund if it does nothing.
Start my Job Agent on WhatsApp

Job Detail

  • location_on
    Location India
  • desktop_windows
    Job Type: Full Time/Permanent
  • schedule
    Shift:
  • analytics
    Career Level:
  • group
    Positions:
  • calendar_view_day
    Experience:
  • male
    Gender: No Preference
  • school
    Degree:
  • calendar_month
    Apply Before: Dec 30, 2026

Job Description

Test Analyst - AI Assurance
Role Overview
Strong understanding of GenAI system architecture, including Retrieval-Augmented Generation (RAG) and Agent-based systems, with hands-on experience in implementing Python-based AI evaluation frameworks such as Ragas, DeepEval, and similar tools. Proven ability to perform Model and Agent quality evaluations using platforms like Arize, LangFuse, and Confident AI, while driving Responsible AI (RAI) validation and reporting key RAI metrics. Experienced in golden dataset curation, synthetic dataset generation, and establishing AI observability and tracing through OpenTelemetry (OTEL) instrumentation to enable effective monitoring, diagnostics, and continuous quality improvement of AI systems.
Key Responsibilities
•  Ability to develop AI Assurance test strategy to assess AI systems for quality, safety, reliability and responsible AI behavior
•  Design AI evaluation strategy for LLMs, RAG and agent evaluation across functional and non-functional dimensions
•  Experience in Golden dataset curation & Synthetic dataset generation
•  Experience in implementing AI evaluation frameworks like Ragas, DeepEval, etc. 
•  Experience in using AI Evaluation tools like LangSmith, Arize, Langfuse, Galileo, Evidently, etc.
•  Experience in instrumentation to establish AI Observability / tracing using OTEL libraries
•  Experience in Integrating AI evaluation into CI/CD pipeline
•  Experience in Responsible AI validation 

 Required Skills and Capabilities
•  Strong understanding of GenAI System Architecture of RAG & Agents
•  Proficiency in Python & AI evaluation SDKs
•  AI Evaluation Frameworks (Ragas, DeepEval) 
•  Model Evaluation 
•  Agent Evaluation 
•  Responsible AI (RAI) Validation 
•  RAI Metrics Reporting 
•  Golden Dataset Curation 
•  Synthetic Dataset Generation 
•  AI Observability 
•  AI Tracing 
•  OpenTelemetry (OTEL) 
•  LLM Evaluation 
•  AI Quality Engineering 
•  AI Governance 
•  AI Monitoring 
•  AI Testing & Assurance 
•  Root Cause Analysis (AI Systems)
•  Knowledge of regulatory considerations for AI systems
Role-Specific Requirements
•  Proficiency in Python & AI evaluation SDKs
•  Strong understanding of RAG & agentic system architecture
•  Ability to develop AI Assurance test strategy to assess AI systems for quality, safety, reliability and responsible AI behavior
•  Design AI evaluation strategy for LLMs, RAG and agent evaluation across functional and non-functional dimensions
•  Experience in Golden dataset curation & Synthetic dataset generation
•  Experience in implementing AI evaluation frameworks like Ragas, DeepEval, etc. 
•  Experience in using AI Evaluation tools like LangSmith, Arize, Langfuse, Galileo, Evidently, etc.
•  Experience in instrumentation to establish AI Observability / tracing using OTEL libraries
•  Experience in Integrating AI evaluation into CI/CD pipeline
•  Experience in Responsible AI validation
•  Knowledge of regulatory considerations for AI systems
Experience Requirements
More than 4 years of experience in AI Assurance Testing

 

 

Source: Hexaware Technologies careers — Read the original posting and apply

This role is listed by the employer on its own careers site. Kaam Ki Khoj does not process applications for it.

See how well you match this job

Make a free profile in a minute — no CV needed — and we score every job against it.

Make my profile

Already registered? Sign in

Company Overview

Hexaware Technologies
Hexaware Technologies · Mumbai, Maharashtra, India
101 open roles

Hexaware Technologies is an employer in the IT/Computers - Software, Software Services sector with operations in India. This is a directory listing maintained by Kaam Ki Khoj so that candidates can find the organisation; it is not an official company... Read More

Apply safely

From the employer's own careers site. Kaam Ki Khoj reads this job from fa-etqo-saasfaprod1.fa.ocs.oraclecloud.com. You apply there, on the employer's own website.

Applying is free. A genuine employer never asks you to pay: no registration fee, deposit, training fee or kit charge, and never your OTP or bank PIN. If anyone asks for money for this job, it is a scam.

How to spot a fake job · Check an offer free

Report this job

Related Jobs

One upload, every employer

Let the right employer find you

Upload your CV once — we read it, build your profile and put you in front of every employer hiring on Kaam Ki Khoj. No forms, no fees.

Upload your CV Browse jobs