Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Fulcrum Worldwide Software

NewBe an early applicant

Big Data Engineer

Posted Updated

Description

Roles and Responsibilities :

  • Design, develop, test, deploy and maintain large-scale big data solutions using Hadoop, Spark, Hive and other related technologies.
  • Collaborate with cross-functional teams to identify business requirements and design scalable architectures for data processing pipelines.
  • Develop automated testing frameworks using Python to ensure high-quality delivery of big data projects on Azure/AWS platforms.
  • Troubleshoot complex issues related to big data processing workflows and provide technical guidance to junior team members.
    • Support day-to-day operations of Hadoop and Big Data platforms including HDFS, Nifi, YARN, Hive, Spark, Kafka, and ZooKeeper in non-prod and prod environments.
    • Perform routine cluster administration tasks such as service restarts, monitoring, log analysis, and basic troubleshooting.
    • Assist with Spark and Kafka pipeline support, issue triage, and incident resolution under guidance from senior engineers.
    • Cloud platforms (AWS, Azure, PCF)
    • Programming and scripting (Python, Go, Bash)
    • Monitoring and observability tools
    • DevOps and CI/CD practices
    • Incident response and troubleshooting
    • Execute standard operating procedures for upgrades, patching, certificate renewals, and access provisioning.
    • Use Unix/Linux skills (shell scripting, process analysis, filesystem troubleshooting) to resolve platform issues.
    • Contribute to automation efforts using Jenkins, Chef and/or Ansible, following existing playbooks and patterns.
    • Work closely with application teams to onboard workloads and ensure adherence to platform standards.

Job Requirements :

  • 4-5 years of experience in Big Data Engineering with expertise in Apache NiFi, HDFS, Kafka, Spark, Hive etc.
  • Strong understanding of cloud-based technologies such as Microsoft Azure or AWS.
  • Proficiency in programming languages like Python along with experience working with SRE tools like Jenkins.

Skills

Apply

See also

Data Engineering jobs by country — openings, pay and top skills →