Senior Data Engineer
Summary
Senior Data Engineer building scalable, high-performance data pipelines and data lake solutions on Google Cloud Platform using BigQuery, Airflow (Composer), Dataflow, and Pub/Sub, with Python or Java for ETL/ELT development, query optimization, data quality, and CI/CD. Requires 8+ years experience; based in Chennai, Pune, Coimbatore, or Bangalore.
Job Summary
We are looking for a skilled Data Engineer with a strong background in building scalable, high-performance data pipelines on Google Cloud Platform (GCP).
The ideal candidate will have hands-on experience with BigQuery, Airflow, and cloud-based ETL workflows, along with solid programming expertise in Java or Python.
Experience: 8+ Years
Location: Chennai, Pune, Coimbatore and Bangalore
Primary Skills
- Google Cloud Platform (GCP): Hands-on experience with BigQuery, Cloud Storage, Dataflow, Pub/Sub, and Airflow (Composer).
- BigQuery: Strong SQL skills with expertise in query optimization, partitioning, clustering, and cost management.
- Airflow (Composer): Proven ability to build, schedule, and monitor ETL/ELT pipelines using Apache Airflow.
- Programming: Proficiency in Python or Java for data processing, automation, and pipeline development.
- ETL & Data Integration: Designing and implementing end-to-end ETL/ELT workflows for structured and unstructured data.
- Database Management: Strong command of SQL; working knowledge of NoSQL databases like MongoDB or Firestore.
- Version Control & DevOps: Experience with Git, CI/CD practices; familiarity with Docker/Kubernetes is a plus.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and data lake solutions on GCP.
- Build and automate data ingestion and transformation workflows using Airflow and BigQuery.
- Ensure data quality, performance, and security across all cloud environments.
- Collaborate with cross-functional teams to translate business needs into efficient data models.
- Optimize queries, manage schema evolution, and maintain high data availability.
- Implement data governance, version control, and CI/CD best practices.
- Monitor and troubleshoot data pipelines for continuous reliability and efficiency improvements.
Why Join KANINI?
- Join KANINI’s award-winning Data Engineering Team, recognized as the "Outstanding Data Engineering Team" at DES 2025
- Work on real-world problemsat the intersection of data, AI, and product development
- Be part of a collaborative, innovative, and growth-driven environment.
- Access to cutting-edge tools, leadership support, and continuous learning opportunities.