Devops Data Hadoop
Posted 2
views
Summary
DevOps Engineer focused on Big Data infrastructure, responsible for CI/CD pipelines, container orchestration using Kubernetes and OpenShift, and managing Hadoop, Spark, and Kafka environments. The role involves implementing IaC with Terraform or Ansible and ensuring system scalability and security.
- DevOps and CI/CD: Design implement and manage CI/CD pipelines using tools like Jenkins and GitOps to automate and streamline the software development lifecycle.
- â Containerization and Orchestration: Deploy and manage containerized applications using Kubernetes and OpenShift ensuring high availability and scalability.
- â Infrastructure Management: Develop and maintain infrastructure as code (IaC) using tools like Terraform or Ansible.
- â Big Data Solutions: Architect and implement big data solutions using technologies such as Hadoop Spark and Kafka.
- â Distributed Systems: Design and manage distributed data architectures to ensure efficient data processing and storage.
- â Collaboration: Work closely with development operations and data teams to understand requirements and deliver robust solutions.
- â Monitoring and Optimization: Implement monitoring solutions and optimize system performance reliability and scalability.
- â Security and Compliance: Ensure infrastructure and data solutions adhere to security best practices and regulatory requirements.