One million success stories. Start yours today.

Direct from employer

Hadoop / PySpark

Date Posted: Sep 19, 2026

Job Detail

  • location_on
    Location Bengaluru, Karnataka, India
  • desktop_windows
    Job Type: Full Time/Permanent
  • schedule
    Shift:
  • analytics
    Career Level:
  • group
    Positions:
  • calendar_view_day
    Experience:
  • male
    Gender: No Preference
  • school
    Degree:
  • calendar_month
    Apply Before: Nov 14, 2026

Job Description

Step into a high-impact consulting role where you’ll lead the design and delivery of scalable big data solutions that turn complex datasets into actionable insights. As a Lead Consultant, you’ll collaborate closely with data engineers, architects, analysts, and business stakeholders to shape modern data platforms built on Hadoop ecosystems and PySpark-based processing. You’ll guide teams through best practices, performance tuning, and reliable delivery—balancing hands-on problem solving with technical leadership. This role is ideal for someone who enjoys mentoring, driving technical decisions, and building data pipelines that are resilient, efficient, and production-ready. If you’re motivated by solving large-scale data challenges and enabling teams to deliver measurable outcomes, you’ll find a collaborative environment here that values ownership, clarity, and continuous improvement.

Responsibilities

  • Lead end-to-end delivery of big data solutions using Hadoop and PySpark, from requirements to production rollout.
  • Design and implement scalable batch/ETL pipelines for large datasets with strong focus on reliability and performance.
  • Drive technical architecture discussions, define standards, and ensure best practices for distributed data processing.
  • Optimize Spark jobs through partitioning strategies, caching, shuffle tuning, and efficient file formats where applicable.
  • Collaborate with stakeholders to translate business needs into technical specifications, delivery plans, and milestones.
  • Establish data quality checks, validation frameworks, and operational monitoring for production pipelines.
  • Perform root-cause analysis for pipeline failures and performance bottlenecks; implement preventive fixes.
  • Mentor engineers through code reviews, design reviews, and knowledge sharing to uplift team capability.
  • Ensure documentation, runbooks, and handover artifacts are created and maintained for support readiness.

Technical requirements

  • Primary skills:Technology->Big Data - Data Processing->PySpark,Technology->Big Data - Hadoop->Hadoop

Additional responsibilities

  • Bachelor’s or Master’s degree (or equivalent) in Engineering/Technology/Computer Applications/Science (BE/BTech/MTech/MCA/MSc or equivalent).
  • 9–11 years of overall experience in data engineering and large-scale data processing environments.
  • Strong hands-on experience with Hadoop ecosystem components and distributed data processing concepts.
  • Strong hands-on experience building data pipelines using PySpark.
  • Proven ability to lead technical delivery, guide teams, and manage stakeholder expectations in consulting engagements.

Preferred Qualifications:

  • Experience designing and implementing Spark-based data processing patterns (batch and incremental loads).
  • Strong understanding of data modeling and storage patterns for big data platforms (partitioning, compaction, schema evolution).
  • Experience with workflow orchestration and scheduling for data pipelines and dependency management.
  • Demonstrated expertise in production hardening: monitoring, alerting, SLAs, and incident management for data jobs.
  • Strong consulting mindset with ability to present solutions, document designs clearly, and influence technical decisions across teams.

Preferred skills

  • Technology › Big Data - Data Processing › PySpark
  • Technology › Big Data - Hadoop › Hadoop

About the role

  • Role: Lead Consultant
  • Experience: 9 - 11 years
  • Education: MCA, MSc, MTech, Bachelor of Engineering, BTech

Source: Infosys careers — Read the original posting and apply

This role is listed by the employer on its own careers site. Kaam Ki Khoj does not process applications for it.

Company Overview

Infosys
Infosys · Bengaluru, Karnataka, India
1,811 open roles

Infosys is an employer in the IT/Computers - Software, Software Services sector with operations in India. This is a directory listing maintained by Kaam Ki Khoj so that candidates can find the organisation; it is not an official company page and Kaam... Read More

Related Jobs

One upload, every employer

Let the right employer find you

Upload your CV once — we read it, build your profile and put you in front of every employer hiring on Kaam Ki Khoj. No forms, no fees.

Upload your CV Browse jobs