Find your next job. Start today.

Direct from employer

Senior Data Scientist

Date Posted: Oct 20, 2026
Apply on company website

Job Detail

  • location_on
    Location Pune, Maharashtra, India
  • desktop_windows
    Job Type: Full Time/Permanent
  • schedule
    Shift:
  • analytics
    Career Level:
  • group
    Positions:
  • calendar_view_day
    Experience:
  • male
    Gender: No Preference
  • school
    Degree:
  • calendar_month
    Apply Before: Dec 07, 2026
Email to Friend Add to Favourite Report Abuse

Job Description

Job Summary

Requirement: AI Agent Runtime Observability & Performance

 

Core theme: Runtime behavior of AI agents — explicitly not development-time skills.

Key Responsibilities

  • Observe agents in production. Instrument deployed agents to trace what they do and how they do it (reasoning steps, tool calls, retrieval, handoffs), so behavior can be explained after the fact.
  • Benchmark across LLMs. Run the same agent against different models and compare quality, cost, latency, and failure modes.
  • Measure behavior. Define and track accuracy, reliability, and repeatability metrics. Build evaluation harnesses that run on live or production-like traffic, not just offline test sets.
  • Own runtime performance and scalability. Find bottlenecks under real load, such as model latency, tool-call chains, and token usage, and tune for throughput and concurrency.
  • Protect chat UX. Monitor and reduce response latency (his benchmark: 5 minutes is unacceptable). Track time-to-first-token, end-to-end response time, and session abandonment.
  • Validate that the agent solves real user needs. Check that it handles genuine customer problems and not just general-purpose questions, using real usage data and feedback.
  • Give pre-release confidence. Assess whether an agent will "behave nicely" in a customer's actual environment before DISW ships it.
  • Evaluate and recommend tooling. Since this is a new area, help select or build the observability stack.

Skill Requirements

  • Hands-on experience observing or evaluating an AI agent after deployment, in production or a production-like setup. Dev-only agent building doesn't qualify.
  • Agent observability and tracing tools (for example LangSmith, Langfuse, Arize Phoenix, or OpenTelemetry-based tracing; these are examples, not names he gave).
  • LLM and agent evaluation: metrics, test harnesses, regression testing, and handling non-determinism.
  • Experience working with multiple LLMs and comparing them.
  • Performance engineering for LLM applications: latency profiling, load testing, caching, and scaling.
  • Familiarity with chat-based or conversational interfaces and their latency and UX constraints.

Other Requirements

  • Python, plus general monitoring and logging stacks (Prometheus, Grafana, ELK or similar).
  • Agent frameworks and architectures (RAG, tool use, multi-agent orchestration), enough to understand what's being observed.
  • Production and DevOps or SRE exposure, such as deployments, CI/CD, and incident analysis.
  • Data analysis, to turn traces and logs into insight about behavior.

Source: HCLTech careers — Read the original posting and apply

This role is listed by the employer on its own careers site. Kaam Ki Khoj does not process applications for it.

Company Overview

HCLTech
HCLTech · Noida, Uttar Pradesh, India
4,246 open roles

HCLTech is an employer in the IT/Computers - Software, Software Services sector with operations in India. This is a directory listing maintained by Kaam Ki Khoj so that candidates can find the organisation; it is not an official company page and Kaam... Read More

Related Jobs

One upload, every employer

Let the right employer find you

Upload your CV once — we read it, build your profile and put you in front of every employer hiring on Kaam Ki Khoj. No forms, no fees.

Upload your CV Browse jobs