Data engineer for AI products
Posted 3
views
Summary
Data engineer designing and scaling high-throughput distributed data pipelines in Databricks and AWS to support analytics and AI-driven products. Works with AI/ML and engineering teams on data architecture, quality, governance, and GenAI tooling; needs 3+ years in data engineering with strong Python and Spark/Scala.
Описание:
We’re looking for a Data Engineer to design and scale high-throughput data pipelines supporting advanced analytics and AI-driven products.
Задачи:
- Architect and maintain distributed data pipelines in Databricks and AWS
- Ingest and process large volumes of structured and unstructured data (internal, market, third-party, alternative sources)
- Collaborate with AI/ML and engineering teams to design scalable data architectures and APIs
- Optimize performance and cost using Spark and cloud-native best practices
- Implement data governance, privacy, lineage, and access controls
- Build automated validation, monitoring, and data quality frameworks
- Evaluate emerging GenAI and data tooling to enhance platform capabilities
Требования:
- 3+ years of experience in data engineering
- Strong Python and experience with Spark or Scala
- Proven experience building distributed pipelines in cloud environments
- Solid understanding of data modeling, architecture, and warehousing principles
- Innovative problem-solving mindset
- Bachelor’s or Master’s degree in Computer Science or Engineering
- Будет плюсом: Experience with graph databases
Условия:
Umowa o Pracę - full time employment 30000-38000 zl gross/month - negotiable