Data Engineer
Posted Updated
Description
A technology company is seeking a Data Engineer in Bengaluru, India. The role involves designing and managing scalable data pipelines for AI-driven SaaS applications in the Consumer Packaged Goods sector. Candidates should have over 3 years of experience, strong skills in Python, PySpark, and SQL, and hands-on experience with AWS services. The job also requires developing data workflows, collaborating with various teams, and optimizing pipeline performance. Comprehensive knowledge of data modeling and ETL processes is essential.
Build and optimize scalable data pipelines for AI platforms.
Develop and orchestrate data pipelines using Airflow/Kubeflow.
Write Python, PySpark, and SQL for data processing.
Own the end-to-end pipeline lifecycle including monitoring.
Design and document data architectures and technical solutions.
Collaborate with AI/ML, MLOps, DevOps, and Product teams.
Optimize data ingestion and pipeline performance.
3+ years of experience in Data Engineering.
Strong coding in Python, PySpark, and SQL.
Hands-on experience with Apache Spark in production environments.
Experience building scalable ETL pipelines.
Familiarity with Airflow/Kubeflow orchestration tools.
Experience with AWS cloud services.
Skills
Sign in to read and join the discussion.