Big Data Engineer (Cloudera Admin)
NewBe an early applicant- Cloudera Infrastructure Design: Design and implement Cloudera infrastructure for data storage, processing, and analytics to meet the platform’s scalability, performance, and security requirements.
- System Integrations & Automation: Manage system integrations, automation, and CI/CD pipelines to streamline operations and improve platform efficiency.
- Scalability & Availability: Ensure system scalability, availability, and performance by proactively monitoring the infrastructure and addressing potential issues.
- Resource Utilization Optimization: Monitor and optimize resource utilization across the platform to ensure efficient operation and reduce costs.
- Collaboration: Work with cross-functional teams to ensure the platform meets the data and business needs of the organization.
- Cloudera Administration: Install, configure, and manage Cloudera Hadoop clusters (CDH/CDP).
- Monitor cluster health using Cloudera Manager.
- Manage Hadoop ecosystem components: HDFS, YARN, Hive, Impala, HBase, Spark, Kafka.
- Perform cluster upgrades, patching, and capacity planning
- Handle performance tuning and troubleshooting.
- Implement and maintain data security (Kerberos, Ranger, Sentry).
- Manage backup, disaster recovery, and data replication.
Requirements
- Education: Bachelor’s degree in computer science, Engineering, or a related field.
- Experience (3-5 years) : Proven experience with Cloudera infrastructure and containerization technologies such as Kubernetes.
- Technical Skills: Strong expertise in streaming, KAFKA), and system monitoring tools such as Dynatrace and Grafana.
- Expertise in designing, implementing, and maintaining scalable infrastructure for data platforms.
- Proficient with Cloudera and CI/CD automation.
- Strong experience in system monitoring, performance optimization, and ensuring high availability.
- Ability to work with cross-functional teams to meet platform needs.