Data Engineer
Posted 9
views
Summary
Design, build, and maintain data transformation pipelines and large-scale distributed systems. Core technologies include Python, Scala, Spark/Flink, AWS, and orchestration tools like Airflow or Dagster.
Duties and Responsibilities:
- Design, build, and maintain data transformation pipelines to produce business-ready datasets to stakeholders.
- Document data pipelines, data models, and data transformation processes.
- Collaborate with business stakeholders teams to identify and understand data requirements for various business needs.
Required Skills/Qualifications:
- Experience with Python and Scala.
- Experience with big data processing frameworks such as Spark, Flink, or Ray.
- Experience with dimensional modeling.
- Experience on building large scale distributed systems in AWS or a similar cloud provider.
- Experience using AI (e.g., Claude) to accelerate workflows.
- Experience with data orchestration tools like Airflow or Dagster.
- Experience with infrastructure-as-code tools (e.g., Terraform) and modern CI/CD pipelines.