Data Engineer

Summary

Data Engineer designing and building scalable data pipelines from production systems into analytical platforms using big data and MPP tech stacks, collaborating with engineering, product, and business teams in South Africa.

An ideal candidate will have excellent communication skills to be able to work with engineering, product, and business owners to develop and define key business questions and to build data sets that answer those questions.


You should have expertise in designing, implementing, and operating stable, scalable, solutions to flow data from production systems into analytical data platforms (big data tech stack + MPP) and into end-user-facing applications for both real-time and batch use cases.



  • The ability to do deep problem-solving and build elegant, maintainable solutions to complex problems.

  • Influence product requirements & operational plans.

  • Install best practices for development and champion their adoption, while working with the product manager to estimate and plan projects in an agile development framework. Architectural & Design Choices, Deep knowledge of one or more tech stacks, identify alternative tech choices and trade-offs.

  • Strong problem-solving skills, Identify feasible alternatives, and freeze on the optimal choice of the design approach.

  • Designing platforms as consumable data services across the organization using Big Data tech stack.

  • Do high-level design independently; Functional modeling, and break-down of a module. Build and execute data modeling projects across multiple tech stacks i.e. big data, MPP, OLAP using agile development techniques.

  • Challenge the status quo and propose innovative ways to process, model, and consume data when it comes to tech stack choices or design principles.

  • Strong engineering mindset - build automated monitoring, alerting, and self-healing (restart ability/graceful failures) features while building the consumption pipelines.

  • Translate business requirements into technical specifications (fact/dimension/filters/derivations/aggregations).

  • Lead by example, mentor and guide team members on everything from structured problem solving to the development of best practices.

  • An ideal candidate will have excellent communication skills to be able to work with engineering, product, and business owners to develop and define key business questions and to build data sets that answer those questions.

  • Above all, you should bring your passion for working with huge data sets and bringing datasets together to answer business questions and drive change.


Soft Skill Requirements



  • designing

  • implementing

  • Problem solving

  • architectural and design choices


About iBCScorp


Since 1998, iBCScorp has been serving clients with their technology needs on a project basis, outsourcing, or direct hire.

See also

Data Engineering jobs by country — openings, pay and top skills →

Tailor your CV for this role?

We couldn't check your fit for this role — add a CV to your profile to see it next time.

A new version of freehire is available