Job Title: Senior Data Engineer ETL (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. ________________________________________ Key Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues. Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills.
Job Title: Senior Data Engineer ETL (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. Key Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues. ________________________________________ Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills.
Job Title: Senior Data Engineer ETL (Databricks, PySpark & Python) Job Summary We are seeking a highly skilled Senior Data Engineer with expertise in Databricks, PySpark, Python, and Big Data technologies. The ideal candidate will have hands-on experience building scalable data platforms, developing high-performance data pipelines, and implementing modern data architectures using Databricks Lakehouse. The role requires strong knowledge of Medallion Architecture, Unity Catalog, and enterprise data governance practices. Key Responsibilities • Design, develop, and maintain scalable data pipelines and ETL/ELT solutions. • Build and optimize batch and streaming data processing solutions using PySpark and Databricks. • Implement and manage Medallion Architecture (Bronze, Silver, Gold layers) for data transformation and consumption. • Configure and manage Unity Catalog for centralized governance, access control, and data lineage. • Develop robust data engineering solutions using Python and Spark frameworks. • Optimize Databricks workloads for performance, scalability, and cost efficiency. • Collaborate with data architects, analysts, data scientists, and business stakeholders to deliver high-quality data solutions. • Ensure data quality, security, governance, and compliance across enterprise data platforms. • Support Data Lake, Lakehouse, and Data Warehouse initiatives. • Troubleshoot and resolve complex data engineering and performance issues. Required Skills & Qualifications • Strong hands-on experience with Databricks and Databricks Lakehouse Platform. • Expert-level proficiency in Python programming. • Deep expertise in PySpark and distributed data processing. • Strong understanding of Big Data concepts and modern data architectures. • Good understanding and practical implementation experience of Medallion Architecture (Bronze, Silver, Gold layers). • Experience working with Unity Catalog for data governance, security, access management, and lineage tracking. • Strong knowledge of Apache Spark performance tuning and optimization. • Experience in designing and building scalable ETL/ELT pipelines. • Hands-on experience with Delta Lake and Lakehouse architecture. • Experience with cloud platforms such as Azure, AWS, or GCP. • Strong analytical, problem-solving, and communication skills.
Source: HCLTech careers — Read the original posting and apply
This role is listed by the employer on its own careers site. Kaam Ki Khoj does not process applications for it.
HCLTech is an employer in the IT/Computers - Software, Software Services sector with operations in India. This is a directory listing maintained by Kaam Ki Khoj so that candidates can find the organisation; it is not an official company page and Kaam... Read More
Upload your CV once — we read it, build your profile and put you in front of every employer hiring on Kaam Ki Khoj. No forms, no fees.
