Data Engineer – Big Data / Data Platform
Summary
The Data Engineer will design, build, and maintain scalable batch and streaming data pipelines using Spark, Apache Beam, and Airflow. The role involves collaborating with engineering teams to manage data transformations and quality using SQL and dbt.
Designing, building, and operating scalable batch and streaming data pipelines is a primary responsibility.
Implementing distributed data processing using frameworks like Spark and Apache Beam is crucial. Orchestrating workflows with Apache Airflow ensures efficient data management.
Developing and maintaining transformations using SQL and dbt is essential for data quality. Collaborating with Analytics Engineers and Platform Engineers is necessary for successful project execution.