Azure Administration

We are looking for an experienced Azure Databricks Administrator to manage, operate, and optimize Azure Databricks platforms within an enterprise Azure ecosystem. The role is responsible for ensuring high availability, security, performance, and cost optimization of Azure Databricks environments while supporting data engineering, analytics, and AI/ML workloads.

The ideal candidate will have strong hands on experience with Azure Databricks, Azure infrastructure, security, and governance, and will work closely with cloud, data, and business teams to deliver a stable and scalable data platform.

Responsibilities

• Administer and manage Azure Databricks workspaces across Development, QA, and Production environments.
• Configure and manage interactive and job clusters, including autoscaling, auto termination, cluster pools, and policies.
• Perform environment setup, workspace configuration, and platform upgrades.
• Ensure high availability and operational stability of Azure Databricks services.
• Integrate Azure Databricks with Azure Data Lake Storage Gen2 (ADLS), Azure SQL, Synapse, Event Hub, and other Azure services.
• Manage VNet injected Databricks workspaces, private endpoints, and network security configurations.
• Work with Azure networking teams to ensure secure and optimized connectivity.
• Manage users, groups, and access controls using Azure Active Directory (Entra ID).
• Implement data governance using Unity Catalog, including catalog, schema, table, and column level security.
• Configure secure authentication using AAD SSO, Managed Identities, and Service Principals.
• Manage secrets and credentials using Azure Key Vault integration.
• Ensure compliance with enterprise security, audit, and regulatory requirements.
• Monitor Azure Databricks usage and implement cost optimization strategies, including:

o Cluster right sizing

o Autoscaling and auto termination

o Instance pool configuration

• Analyze workload patterns and recommend performance tuning best practices.
• Provide usage and cost reports to stakeholders and FinOps teams.
• Deploy and manage Azure Databricks infrastructure using Infrastructure as Code tools such as Terraform or ARM templates.
• Support CI/CD pipelines for Databricks notebooks, jobs, libraries, and workspace artifacts using Azure DevOps or GitHub.
• Automate operational tasks to reduce manual effort and improve reliability.
• Set up monitoring and alerting using Azure Monitor, Log Analytics, and Databricks metrics.
• Troubleshoot cluster failures, job issues, workspace access problems, and performance bottlenecks.
• Perform root cause analysis and implement preventive measures.
• Provide Level 2/Level 3 support within a 24×7 operational model.
• Create and maintain platform documentation, runbooks, SOPs, and architecture diagrams.
• Collaborate with data engineers, data scientists, cloud engineers, and security teams.
• Participate in platform reviews and continuous improvement initiatives.

Technical requirements

Mandatory Technical Skills

• 7+ years of experience in data platform or cloud administration, with strong focus on Azure Databricks.
• Hands on experience with Azure services including ADLS Gen2, Azure Networking, Azure Monitor, and Azure AD.
• Strong understanding of Apache Spark architecture and performance tuning.
• Experience managing secure, enterprise scale Azure Databricks environments.
• Proficiency in Python, PowerShell, or Bash scripting.
• Hands on experience with Terraform or ARM templates.

________________________________________

Preferred / Good to Have

• Experience with Unity Catalog, data lineage, and governance frameworks.
• Familiarity with Delta Lake, job orchestration, and workload isolation.
• Experience with CI/CD pipelines using Azure DevOps or GitHub Actions.
• Understanding of FinOps practices in Azure.
• Certifications:

o Azure Databricks Certified Associate/Professional

o Microsoft Azure Administrator / Solutions Architect

________________________________________

Soft Skills

• Strong analytical and troubleshooting capabilities.
• Excellent communication skills with technical and business stakeholders.
• Ability to work independently in a fast paced, 24×7 support environment.
• Proactive, detail oriented, and focused on operational excellence.

________________________________________

Nice to Have

• Experience supporting large enterprise or multi workspace Azure Databricks platforms.
• Exposure to onsite offshore delivery models and global teams.
• Experience working in regulated environments (Healthcare).

Additional responsibilities

Infosys is a global leader in next-generation digital services and consulting. We enable clients in more than 50 countries to navigate their digital transformation. With over four decades of experience in managing the systems and workings of global enterprises, we expertly steer our clients through their digital journey. We do it by enabling the enterprise with an AI-powered core that helps prioritize the execution of change. We also empower the business with agile digital at scale to deliver unprecedented levels of performance and customer delight. Our always-on learning agenda drives their continuous improvement through building and transferring digital skills, expertise, and ideas from our innovation ecosystem. Infosys provides equal employment opportunities to applicants and employees without regard to race, color, sex, gender identity; sexual orientation, religious practices and observances; national origin; pregnancy, childbirth, or related medical conditions; status as a protected veteran or spouse/family member of a protected veteran; or disability.

Preferred skills

• Technology › Infrastructure-Server Administration › Cloud Infrastructure › Azure Administration

About the role

• Role: Senior Consultant
• Experience: 7 - 9 years
• Education: Bachelor of Engineering