Senior Data Platform System Engineer (Data Engineering Team) (Data & AI)

You will be responsible for operating and evolving SP Group’s enterprise Data & AI platform

to support mission-critical analytics, regulatory compliance, and AI-driven innovation. This

role focuses on ensuring the platform is scalable, reliable, secure, and performant, while

enabling data engineers, scientist and analysts to deliver business value efficiently.

• Ensure high availability and reliability of the enterprise data platform (Cloudera and Talend Big Data).
• Deliver scalable and performant data infrastructure supporting batch and near real-time workloads.
• Enable secure and compliant data operations aligned with enterprise and regulatory requirements.
• Improve platform automation and operational efficiency through DevOps and Infrastructure-as-Code.
• Provide strong platform support and partnership to data engineering, analytics, and business teams.

• Design, deploy, and manage large-scale data platform environments (primarily Cloudera and Talend Big Data).
• Operate and optimize Hadoop ecosystem components (HDFS, Hive, Spark, HBase, Impala, Iceberg, Ozone).
• Manage cluster lifecycle including provisioning, scaling, patching, and upgrades.
• Ensure platform uptime, stability, and service reliability through proactive monitoring and maintenance.

• Monitor workload performance and optimize data processing efficiency.
• Perform capacity planning to support increasing data volumes and usage.
• Troubleshoot performance bottlenecks across compute, storage, and query layers.

• Plan and coordinate execution of platform upgrades, patching, and scaling activities.
• Manage deployment schedules and ensure minimal service disruption.
• Track and report execution progress against platform roadmap.
• Implement Infrastructure-as-Code using tools such as Terraform and Ansible.
• Design and maintain CI/CD pipelines for platform configuration.
• Automate platform provisioning, deployment, and operational workflows.
• Improve observability through monitoring, alerting, and logging practices.

• Implement and manage platform security controls including authentication (Kerberos), authorization (e.g. Ranger), and encryption.
• Ensure compliance with enterprise data governance, security policies, and regulatory requirements.
• Support auditability, access control, and data protection mechanisms.
• Work closely with data engineers, data scientists, and business teams to enable data solution delivery.
• Partner with infrastructure, network, and security teams to ensure platform integration and stability.
• Provide guidance, troubleshooting, and best practices to users of the platform.
• Document platform architecture, processes, and operational procedures.

• 5- 10 years of experience in data platform engineering, big data operations, or infrastructure engineering.
• Strong hands-on experience managing big data platforms (Cloudera CDP, Hadoop ecosystem).
• Deep understanding of Hadoop components: HDFS, Hive, Spark, HBase, and related services.
• Experience with cluster operations, performance tuning, and troubleshooting.
• Proficiency in automation and scripting (Python, Bash, Terraform, Ansible).
• Experience with CI/CD pipelines and DevOps practices.
• Solid understanding of data platform security and access control concepts.
• Strong analytical and problem-solving skills with a proactive mindset.
• Ability to work in a fast-paced, cross-functional environment.
• Good communication skills with ability to translate technical topics to stakeholders.
• Ownership mentality with focus on platform reliability and continuous improvement.

• Experience with Databricks, Talend, or hybrid data platforms.
• Knowledge of containerization and orchestration.
• Experience with streaming platforms (e.g. Kafka) or real-time processing.
• Familiarity with modern data lake-house concepts (e.g. Iceberg, Delta).
• Exposure to cloud or hybrid environments (Azure preferred).

• Join us to empower the future of energy .
• You will work in teams that thrive on collaboration for outstanding achievements.
• We have regular wellness programmes to promote healthy living