Python & Databricks Developer Role Overview We are looking for a skilled Python and Databricks Developer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks platform. The role involves working closely with data engineers, data scientists, and client stakeholders to build reliable ETL pipelines and support advanced analytics use cases. Key Responsibilities Data Engineering & Development • Design, develop, and maintain ETL/ELT pipelines using Python (PySpark) and Databricks • Build and optimize Spark jobs for large scale data processing • Develop reusable and modular Python code following best practices • Implement data transformations, aggregations, and validations Databricks Platform • Create and manage Databricks notebooks, jobs, and workflows • Optimize Spark performance (partitioning, caching, joins, memory tuning) • Work with Delta Lake (Delta tables, ACID transactions, schema evolution) • Manage Databricks clusters and job scheduling Data Integration • Ingest data from multiple sources: o RDBMS (Oracle, SQL Server, MySQL, PostgreSQL) o Cloud storage (ADLS, S3, Blob Storage) o APIs and streaming sources (Kafka/Event Hubs – preferred) • Ensure data quality, reliability, and consistency Cloud & DevOps • Work with Azure / AWS / GCP (based on client environment) • Integrate pipelines with CI/CD tools (Azure DevOps, GitHub, GitLab) • Use Git for version control and collaborative development Collaboration & Client Interaction • Collaborate with business analysts and data scientists to understand requirements • Participate in sprint ceremonies (planning, review, retrospectives) • Support client demos, sprint reviews, and technical discussions • Troubleshoot production issues and perform root cause analysis Required Skills & Qualifications Technical Skills • Strong hands on experience with Python • Solid experience with Databricks and Apache Spark (PySpark) • Good understanding of data warehousing concepts • Experience with Delta Lake • Strong SQL skills • Experience working with large datasets and distributed systems Cloud & Tools • Experience with at least one cloud platform: o Azure (ADF, ADLS, Azure Databricks) – preferred o or AWS (S3, EMR, Glue) • Experience with CI/CD, Git, and Agile tooling (JIRA, Azure Boards) Soft Skills • Strong problem solving and analytical skills • Good communication and documentation skills • Experience working in Agile/Scrum delivery models • Ability to work in client facing environments ________________________________________ Good to Have • Experience with streaming frameworks (Kafka, Spark Structured Streaming) • Familiarity with data governance, security, and access controls • Exposure to machine learning workflows on Databricks • Databricks or cloud certifications Experience • 8+ years of overall experience in data engineering • 5+ years of hands on Databricks and Python/PySpark experience
Python & Databricks Developer Role Overview We are looking for a skilled Python and Databricks Developer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks platform. The role involves working closely with data engineers, data scientists, and client stakeholders to build reliable ETL pipelines and support advanced analytics use cases. Key Responsibilities Data Engineering & Development • Design, develop, and maintain ETL/ELT pipelines using Python (PySpark) and Databricks • Build and optimize Spark jobs for large scale data processing • Develop reusable and modular Python code following best practices • Implement data transformations, aggregations, and validations Databricks Platform • Create and manage Databricks notebooks, jobs, and workflows • Optimize Spark performance (partitioning, caching, joins, memory tuning) • Work with Delta Lake (Delta tables, ACID transactions, schema evolution) • Manage Databricks clusters and job scheduling Data Integration • Ingest data from multiple sources: o RDBMS (Oracle, SQL Server, MySQL, PostgreSQL) o Cloud storage (ADLS, S3, Blob Storage) o APIs and streaming sources (Kafka/Event Hubs – preferred) • Ensure data quality, reliability, and consistency Cloud & DevOps • Work with Azure / AWS / GCP (based on client environment) • Integrate pipelines with CI/CD tools (Azure DevOps, GitHub, GitLab) • Use Git for version control and collaborative development Collaboration & Client Interaction • Collaborate with business analysts and data scientists to understand requirements • Participate in sprint ceremonies (planning, review, retrospectives) • Support client demos, sprint reviews, and technical discussions • Troubleshoot production issues and perform root cause analysis Required Skills & Qualifications Technical Skills • Strong hands on experience with Python • Solid experience with Databricks and Apache Spark (PySpark) • Good understanding of data warehousing concepts • Experience with Delta Lake • Strong SQL skills • Experience working with large datasets and distributed systems Cloud & Tools • Experience with at least one cloud platform: o Azure (ADF, ADLS, Azure Databricks) – preferred o or AWS (S3, EMR, Glue) • Experience with CI/CD, Git, and Agile tooling (JIRA, Azure Boards) Soft Skills • Strong problem solving and analytical skills • Good communication and documentation skills • Experience working in Agile/Scrum delivery models • Ability to work in client facing environments Good to Have • Experience with streaming frameworks (Kafka, Spark Structured Streaming) • Familiarity with data governance, security, and access controls • Exposure to machine learning workflows on Databricks • Databricks or cloud certifications Experience • 8+ years of overall experience in data engineering • 5+ years of hands on Databricks and Python/PySpark experience
Python & Databricks Developer Role Overview We are looking for a skilled Python and Databricks Developer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks platform. The role involves working closely with data engineers, data scientists, and client stakeholders to build reliable ETL pipelines and support advanced analytics use cases. ________________________________________ Key Responsibilities Data Engineering & Development • Design, develop, and maintain ETL/ELT pipelines using Python (PySpark) and Databricks • Build and optimize Spark jobs for large scale data processing • Develop reusable and modular Python code following best practices • Implement data transformations, aggregations, and validations Databricks Platform • Create and manage Databricks notebooks, jobs, and workflows • Optimize Spark performance (partitioning, caching, joins, memory tuning) • Work with Delta Lake (Delta tables, ACID transactions, schema evolution) • Manage Databricks clusters and job scheduling Data Integration • Ingest data from multiple sources: o RDBMS (Oracle, SQL Server, MySQL, PostgreSQL) o Cloud storage (ADLS, S3, Blob Storage) o APIs and streaming sources (Kafka/Event Hubs – preferred) • Ensure data quality, reliability, and consistency Cloud & DevOps • Work with Azure / AWS / GCP (based on client environment) • Integrate pipelines with CI/CD tools (Azure DevOps, GitHub, GitLab) • Use Git for version control and collaborative development Collaboration & Client Interaction • Collaborate with business analysts and data scientists to understand requirements • Participate in sprint ceremonies (planning, review, retrospectives) • Support client demos, sprint reviews, and technical discussions • Troubleshoot production issues and perform root cause analysis Required Skills & Qualifications Technical Skills • Strong hands on experience with Python • Solid experience with Databricks and Apache Spark (PySpark) • Good understanding of data warehousing concepts • Experience with Delta Lake • Strong SQL skills • Experience working with large datasets and distributed systems Cloud & Tools • Experience with at least one cloud platform: o Azure (ADF, ADLS, Azure Databricks) – preferred o or AWS (S3, EMR, Glue) • Experience with CI/CD, Git, and Agile tooling (JIRA, Azure Boards) Soft Skills • Strong problem solving and analytical skills • Good communication and documentation skills • Experience working in Agile/Scrum delivery models • Ability to work in client facing environments Good to Have • Experience with streaming frameworks (Kafka, Spark Structured Streaming) • Familiarity with data governance, security, and access controls • Exposure to machine learning workflows on Databricks • Databricks or cloud certifications Experience • 8+ years of overall experience in data engineering • 5+ years of hands on Databricks and Python/PySpark experience
Python & Databricks Developer Role Overview We are looking for a skilled Python and Databricks Developer to design, develop, and optimize scalable data engineering and analytics solutions on the Databricks platform. The role involves working closely with data engineers, data scientists, and client stakeholders to build reliable ETL pipelines and support advanced analytics use cases. Key Responsibilities Data Engineering & Development • Design, develop, and maintain ETL/ELT pipelines using Python (PySpark) and Databricks • Build and optimize Spark jobs for large scale data processing • Develop reusable and modular Python code following best practices • Implement data transformations, aggregations, and validations Databricks Platform • Create and manage Databricks notebooks, jobs, and workflows • Optimize Spark performance (partitioning, caching, joins, memory tuning) • Work with Delta Lake (Delta tables, ACID transactions, schema evolution) • Manage Databricks clusters and job scheduling Data Integration • Ingest data from multiple sources: o RDBMS (Oracle, SQL Server, MySQL, PostgreSQL) o Cloud storage (ADLS, S3, Blob Storage) o APIs and streaming sources (Kafka/Event Hubs – preferred) • Ensure data quality, reliability, and consistency Cloud & DevOps • Work with Azure / AWS / GCP (based on client environment) • Integrate pipelines with CI/CD tools (Azure DevOps, GitHub, GitLab) • Use Git for version control and collaborative development Collaboration & Client Interaction • Collaborate with business analysts and data scientists to understand requirements • Participate in sprint ceremonies (planning, review, retrospectives) • Support client demos, sprint reviews, and technical discussions • Troubleshoot production issues and perform root cause analysis Required Skills & Qualifications Technical Skills • Strong hands on experience with Python • Solid experience with Databricks and Apache Spark (PySpark) • Good understanding of data warehousing concepts • Experience with Delta Lake • Strong SQL skills • Experience working with large datasets and distributed systems Cloud & Tools • Experience with at least one cloud platform: o Azure (ADF, ADLS, Azure Databricks) – preferred o or AWS (S3, EMR, Glue) • Experience with CI/CD, Git, and Agile tooling (JIRA, Azure Boards) Soft Skills • Strong problem solving and analytical skills • Good communication and documentation skills • Experience working in Agile/Scrum delivery models • Ability to work in client facing environments Good to Have • Experience with streaming frameworks (Kafka, Spark Structured Streaming) • Familiarity with data governance, security, and access controls • Exposure to machine learning workflows on Databricks • Databricks or cloud certifications Experience • 8+ years of overall experience in data engineering • 5+ years of hands on Databricks and Python/PySpark experience
Source: HCLTech careers — Read the original posting and apply
This role is listed by the employer on its own careers site. Kaam Ki Khoj does not process applications for it.
HCLTech is an employer in the IT/Computers - Software, Software Services sector with operations in India. This is a directory listing maintained by Kaam Ki Khoj so that candidates can find the organisation; it is not an official company page and Kaam... Read More
Upload your CV once — we read it, build your profile and put you in front of every employer hiring on Kaam Ki Khoj. No forms, no fees.
