Senior Data Engineer
Summary
Designs and maintains the data infrastructure behind a Generative AI initiative: building data lakes, warehouses, feature stores, and reliable ETL/ELT pipelines, and collaborating with data scientists and ML engineers. Core stack includes Python, Apache Spark/Hadoop, SQL and NoSQL databases, Airflow/Luigi orchestration, and MLOps tools like MLflow and DVC.
Are you ready to build the backbone of the next generation of AI? We are looking for a detail-oriented Data Engineer to join our team and support a transformative Generative AI (GenAI) initiative.
In this role, you won't just be moving data; you will be designing and maintaining the high-performance infrastructure that powers advanced AI models for the world's leading organizations. Join a collaborative global community where you are empowered to shape your career and reimagine what's possible.
Your Role
As a Data Engineer within our AI practice, you will ensure our data workflows are efficient, our pipelines are reliable, and our systems are scalable. Your key responsibilities include:
- GenAI Optimization: Building and maintaining data lakes, warehouses, and feature stores specifically optimized for GenAI workloads.
- Pipeline Engineering: Designing and implementing robust ETL/ELT processes and automated workflows using tools like Apache Airflow or Luigi.
- Architecture Design: Developing large-scale, high-performance data architectures that support real-time data processing and streaming.
- Cross-Functional Collaboration: Working side-by-side with Data Scientists and ML Engineers to understand feature engineering and data augmentation requirements.
- Innovation: Exploring and implementing emerging tools and technologies to enhance our evolving GenAI product development.
Your Skills and Experience
We are looking for a technical expert who thrives in an Agile environment and has a passion for AI/ML infrastructure.
Technical Requirements:
- Programming: Proficiency in Python is a must. Familiarity with Scala or Java is highly valued.
- Big Data Frameworks: Hands-on experience with Apache Spark or Hadoop.
- Database Mastery: Expert knowledge of SQL and NoSQL (e.g., PostgreSQL, MySQL, MongoDB, Cassandra).
- AI/ML Context: Solid understanding of data preparation for Machine Learning, including MLOps tools and practices (MLflow, DVC).
- Version Control: Experience managing codebases and documentation via Git or GitLab.
Professional Qualities:
- Proven track record in data engineering, preferably within AI/ML environments.
- Strong problem-solving skills and a creative approach to building innovative systems.
- Excellent communication skills with the ability to translate complex technical concepts for non-technical stakeholders.
- Education: Bachelor's degree in Computer Science, Data Science, IT, or a related field.
Why Join Us?
Join a thriving company where technology is used to build a more sustainable and inclusive world. You'll gain the experience and connections needed to shape your future while working on projects that have a tangible global impact.