Senior Databricks Platform Administrator

Purpose of the Position :

Design, develop, support, and drive comprehensive data engineering solutions using Databricks.

InfoCepts is a global leader of end-to-end data and analytics solutions with nearly 20 years' experience enabling customers to derive value from a variety of data-driven capabilities.

Unique among its peers, InfoCepts operates with the scale of a global consulting firm, yet the expertise of a niche partner.

Key Result Areas and Activities :

- Implement enhancements, monitor performance, troubleshoot issues, and deliver continuous support for Databricks jobs in the production environment and perform access management, cluster management for the same.

- Conduct unit testing, integration testing, and troubleshooting of ETL processes to ensure data accuracy and reliability.

- Manage deployment of Databricks ETL processes from development to production.

- Respond to incidents reported by users or monitoring tools, ensuring swift resolution to minimize downtime.

- Maintain detailed logs of incidents, resolutions, and troubleshooting steps for knowledge sharing and future reference.

- Candidate needs to work in rotational shifts (including on-call, morning, afternoon, night and weekends shift) proving round the clock support for keeping the environments running and performing at optimal levels.

Essential Skills :

- Experience in working with Databricks with PySpark, with a strong emphasis on support and troubleshooting in production environments.

- Experience of Databricks Platform administration.

- Experience in SQL, data warehousing concepts, and relational databases like Oracle, SQL Server, etc.

- Strong understanding of data integration, ETL best practices, and batch processing.

- Knowledge of incident management tools and methodologies like ServiceNow or Jira.

Desirable Skills :

- Hands on experience on Informatica PowerCenter.

- Hands-on experience or knowledge on Alteryx.

- Good understanding of the CI/CD pipeline.

- Working experience with GitHub.

- Experience in working with Job scheduler and workload automation tools like Tidal job scheduler, Control-M, etc.

- Familiar with Unix/Linux scripting for automating tasks and managing job failures.

Qualification :

- 5+ years of relevant experience in ETL tools.

- Bachelor's degree in computer science, engineering, or related field.

- Demonstrated continued learning through one or more technical certifications or related methods.

- Relevant experience in Retail domain is a plus.

Qualities :

- Exceptional problem-solving skills, with the ability to break down issues, identify root causes, and propose solutions.

- Ability to handle multiple tasks effectively, maintain a professional attitude, and work well in a team.

- Excellent interpersonal and communication skills, with a customer-focused approach and keen attention to detail.

- Able to work with cross functional teams and clients in different time zones and in rotational shift.

- Research focused mindset.

- Champion efforts to ensure appropriate development best practices and assist with strategy and roadmap development.