freehire launches on Product Hunt on 26 August.

Follow →

emagine

Discussion

Data Engineer – Digital Transformation

Summary

Builds large-scale data pipelines and analytics for an automotive company’s digital transformation, using PySpark/Spark and Palantir Foundry to process and model data for smarter decisions.

Summary:

The Data Engineer role focuses on developing cutting-edge data solutions that support transformation into a "Digital Enterprise" in the automotive industry. The primary objective is to collect, store, process, and analyze large sets of data to facilitate smarter decision-making and enhance service efficiency across various teams.

Main Responsibilities:

  • Collect, store, process, and analyze large data sets.

  • Implement data solutions on Palantir Foundry or for specific AI solutions.

  • Collaborate with global teams across development, manufacturing, quality, sales, and purchasing.

  • Build large-scale batch and real-time data pipelines using frameworks like PySpark and Spark.

  • Ensure data quality and perform data problem analysis.

  • Develop, construct, test, and maintain data pipelines.

  • Document data and maintain definitions.

  • Contribute to functional analysis and user needs translation into structured data pipelines.

  • Prepare data for predictive modeling and develop associated processes.

  • Conduct feasibility studies for optimal technical solutions.

  • Utilize Agile and DevOps methodologies to deliver projects timely.

  • Share expertise and facilitate knowledge transfer within the team.

Key Requirements:

  • 3-5 years of experience in data mining from various data sources.

  • Proven experience with data processing frameworks, particularly PySpark or Spark.

  • Strong understanding of data quality assurance and analysis.

  • Ability to develop and maintain data pipelines.

  • Fluency in English, both verbally and in writing.

Nice to Have:

  • Bachelor's or Master's degree in computer science engineering.

  • Experience with Big Data solutions and BI tools.

  • Hands-on experience with PySpark/Python.

  • Knowledge of PL/SQL or SQL.

  • Familiarity with Agile Software Development methodologies.

  • Excellent communication and documentation skills.

  • Problem-solving and troubleshooting skills.

  • Ability to create compelling narratives from data.

Show