Big Data DevOps Engineer
Summary
DevOps engineer building and running a secure, multi-tenant Hortonworks Hadoop big data platform on AWS for a UK client. Day-to-day work spans Linux infrastructure, CI/CD automation (Jenkins, Ansible, Terraform, Docker), and Hadoop/HDP administration with Kerberos/Knox security in an agile delivery model.
In this role you help drive a transformational big data program for a UK client, focusing on scalable data platforms and advanced visualization. You will design, deploy and secure cloud-based Hadoop infrastructure and lead CI/CD automation in a multi-tenant environment. You collaborate with cross-functional teams and stakeholders to ensure production readiness and platform reliability. This role offers hands-on work with cutting-edge big data tech and a clear impact on enterprise analytics. You’ll be part of an agile delivery model in London, contributing to a high-visibility program with senior stakeholders.
Responsibilities- Represent Cognizant as a DevOps expert to clients and partners to build trust and drive opportunities
- Design, deploy and maintain complex Linux-based infrastructure
- Set up a secure Hortonworks Hadoop cluster in AWS, including administration and capacity planning
- Automate operations with CI/CD pipelines, versioning and containerization
- Run a data platform for multiple tenants and client apps for transactional and analytical use cases
- Coordinate production readiness by integrating development outputs and supporting live systems
- Collaborate with leads and developers to deliver and guide the team toward success
- Infrastructure automation tools: Ansible, Terraform, SaltStack, Puppet, and Docker
- AWS cloud setup and administration
- Strong Linux administration skills
- Python and Java proficiency
- CI tools: Jenkins/Nexus
- Java build systems: Maven, Gradle
- GitHub for code versioning and branching
- Configuration management, backup/restore with Puppet and custom Bash scripts
- System monitoring: Zabbix, Prometheus, Nagios
- ELK stack: Elasticsearch, Logstash
- Hadoop cluster administration, HDP security with Knox and Kerberos
- Agile experience
- Excellent communication and documentation skills
- Ability to articulate tech and business trade-offs
- excellent communication
- team player
- can-do attitude
- Python
- Java
- Hadoop/HDP