Lead Data Engineer

Job Summary (Lead Data Engineer Data Transformation & Consumption)

- Lead and drive data modernization, transformation, governance, and analytics consumption initiatives.
- Build and manage scalable ETL/ELT pipelines for diverse data types (structured, semi-structured, unstructured).
- Develop batch and real-time data processing solutions using modern cloud and big data technologies.
- Design and implement data models, data marts, and analytics-ready datasets for BI, AI, and ML.
- Lead cloud data engineering projects across AWS, Azure, or GCP.
- Ensure data quality, governance, lineage, security, and compliance throughout the data ecosystem.
- Collaborate with business stakeholders, data architects, data scientists, and BI teams to deliver data-driven business value.
- Modernize legacy data warehouses and support data lakehouse architectures and platform transformations.
- Establish and enforce engineering best practices, CI/CD pipelines, Infrastructure as Code (IaC), and automation.
- Mentor and lead data engineering teams; conduct architecture reviews and provide technical governance.
- Optimize platform performance, scalability, reliability, and manage cloud costs.

Core Technical Skills
- Proficient in: SQL, Python, PySpark, Spark
- Data modeling (Star Schema, Snowflake, Data Vault)
- ETL/ELT tools: Airflow, DataStage, Informatica, Talend, DBT
- Cloud data warehouses: BigQuery, Snowflake, Synapse, Redshift
- Event streaming: Kafka, Pub/Sub, Event Hub
- DevOps: Git, CI/CD, Terraform, Docker, Kubernetes
- Cloud platforms: AWS, Azure, GCP

Preferred Skills
- Experience with Data Lakehouse (Databricks, Delta Lake)
- Master Data Management (MDM)
- Data Mesh/Data Product Architecture
- AI/ML & Generative AI data platforms
- Domain experience in BFSI, Insurance, Retail, or Healthcare