One million success stories. Start yours today.

Direct from employer

Databricks, Pyspark

Date Posted: Sep 19, 2026

Job Detail

  • location_on
    Location Chennai, Tamil Nadu, India
  • desktop_windows
    Job Type: Full Time/Permanent
  • schedule
    Shift:
  • analytics
    Career Level:
  • group
    Positions:
  • calendar_view_day
    Experience:
  • male
    Gender: No Preference
  • school
    Degree:
  • calendar_month
    Apply Before: Nov 14, 2026

Job Description

Join a high-impact data engineering team where you’ll shape modern analytics platforms and help turn raw data into trusted, actionable insights. In this role, you’ll lead the design and delivery of scalable data pipelines on Databricks using PySpark, partnering closely with analysts, data scientists, and platform teams to enable faster decision-making across the business. You’ll bring strong engineering discipline—clean code, performance tuning, and reliable operations—while guiding best practices and mentoring teammates. If you enjoy solving complex data challenges, optimizing distributed workloads, and building systems that are resilient, secure, and easy to evolve, this is a great opportunity to drive meaningful outcomes in a collaborative, growth-focused environment.

Responsibilities

Key Responsibilities:

  • Lead the development of end-to-end data pipelines on Databricks using PySpark for batch and incremental processing.
  • Design scalable data models and curated datasets to support analytics and downstream consumption.
  • Write and optimize advanced SQL for transformations, validations, and performance-critical queries.
  • Implement robust data quality checks, reconciliation logic, and monitoring to ensure trusted datasets.
  • Tune Spark jobs for performance and cost efficiency (partitioning, caching, file formats, cluster sizing).
  • Establish coding standards, reusable frameworks, and review practices to improve maintainability.
  • Collaborate with stakeholders to translate requirements into technical designs and delivery plans.
  • Troubleshoot production issues, perform root-cause analysis, and drive preventive improvements.
  • Mentor team members and provide technical guidance across design, implementation, and optimization.

Minimum Qualifications:

  • BTECH, MTECH, MCA, or MSC in Computer Science, Information Technology, or a related field.
  • 6–8 years of overall experience in data engineering or large-scale data processing roles.
  • Strong hands-on experience with PySpark for distributed data processing and transformation logic.
  • Strong hands-on experience with Databricks for building, running, and managing data workloads.
  • Proficiency in Advanced SQL including complex joins, window functions, and query optimization.
  • Experience building reliable pipelines with strong focus on data quality, performance, and stability

Technical requirements

Technology->Analytics - Solutions->SQL Server - Analytics

Technology->Big Data - Data Processing->PySpark

Technology->Data Engineering->Databricks

Additional responsibilities

Preferred Qualifications:

  • Experience designing lakehouse-style architectures and organizing curated layers for analytics readiness.
  • Strong experience with Spark optimization techniques and handling large-scale datasets efficiently.
  • Ability to build reusable PySpark utilities/frameworks for ingestion, transformation, and validation patterns.
  • Experience with orchestration and scheduling approaches for dependable pipeline execution and recovery.
  • Proven track record of technical leadership: mentoring, conducting reviews, and driving engineering best practices.

Preferred skills

  • Technology › Data Engineering › Databricks
  • Technology › Big Data - Data Processing › PySpark
  • Technology › Analytics - Solutions › SQL Server - Analytics

About the role

  • Role: Senior Consultant
  • Experience: 6 - 8 years
  • Education: MCA, MSc, MTech, Bachelor of Engineering, BTech

Source: Infosys careers — Read the original posting and apply

This role is listed by the employer on its own careers site. Kaam Ki Khoj does not process applications for it.

Company Overview

Infosys
Infosys · Bengaluru, Karnataka, India
1,811 open roles

Infosys is an employer in the IT/Computers - Software, Software Services sector with operations in India. This is a directory listing maintained by Kaam Ki Khoj so that candidates can find the organisation; it is not an official company page and Kaam... Read More

Related Jobs

One upload, every employer

Let the right employer find you

Upload your CV once — we read it, build your profile and put you in front of every employer hiring on Kaam Ki Khoj. No forms, no fees.

Upload your CV Browse jobs