Data Engineer

Job Description :

We are looking for 5+ years of experience in data engineering and data science roles across fintech, e-commerce, and consumer platforms for Bangalore Location.

Work from Office opportunity - Client Interaction Involved.

Key Responsibilities :

- Architect and deploy scalable data pipelines to ensure seamless data ingestion and transformation across diverse enterprise environments.

- Implement and manage modern lakehouse architectures to provide a unified, performant interface for both batch and real-time analytics.

- Develop and optimize streaming applications using Apache Flink and Kafka to support low-latency data processing requirements.

- Automate data transformation workflows using dbt to ensure high-quality, consistent, and well documented data models for downstream consumption.

- Manage the lifecycle of data stored in Apache Iceberg tables to improve query performance and data reliability for analytical workloads.

- Collaborate with cloud infrastructure teams to provision and maintain secure, cost-effective environments that support high-volume data processing.

Skills Required :

Programming Languages :

- Proficient in Python and SQL.

Big Data & Data Engineering :

- Expert in building large-scale data pipelines and lakehouse architectures using PySpark, DBT, Apache Kafka / Apache Flink, and Apache Iceberg.

Cloud Infrastructure :

- Strong experience with AWS, including cost optimization (e.g., reducing EMR costs) and building cloud-agnostic MLOps platforms.

Data Orchestration & Workflow :

- Hands-on expertise with Apache Airflow for automating and scaling production pipelines.

Data Warehousing & Storage :

- Experience with Snowflake (Snow pipes, DBT models), Presto, and Ceph RADOS.

MLOps & Data Science :

- Skilled in deploying NLP/ML models (such as sentence transformers and collaborative filtering), building end-to-end MLOps solutions, and feature engineering.

Analytics & Visualization :

- Proficient in creating insights through Tableau and Python-based Dash.

Data Governance :

- Experience evaluating and implementing tools like Immuta, Collibra, Atlan, and CastorDoc for cataloging and security.