Data Engineer/Lead - Big Data Ecosystem

Data Engineer Lead to design, build, and manage scalable data infrastructure and pipelines for enterprise-grade data platforms across cloud and big data ecosystems.

The ideal candidate will lead a team of data engineers and drive end-to-end data architecture, ensuring high-performance, secure, and reliable data solutions that support analytics, AI/ML, and business intelligence initiatives.

Responsibilities :

- Lead design and development of scalable data pipelines and data engineering solutions.

- Architect and implement end-to-end data platforms (batch and real-time processing).

- Build and optimize ETL/ELT workflows using modern data engineering tools.

- Develop robust data models for analytics, reporting, and machine learning use cases.

- Work with big data frameworks like Hadoop, Spark, and distributed computing systems.

- Design cloud-native data solutions on AWS / Azure / GCP.

- Implement data governance, data quality, and data security standards.

- Collaborate with Data Scientists, Analysts, and Business stakeholders.

- Manage streaming data pipelines using Kafka / real-time ingestion tools.

- Monitor, troubleshoot, and optimize data systems for performance and scalability.

- Lead and mentor a team of data engineers.

- Ensure documentation and adherence to best engineering practices.

Exp (Years) :

- 10 to 18 Years

Required Skills :

- Strong experience in Python, SQL, and distributed data systems.

- Expertise in Apache Spark, Hadoop, Kafka, Airflow, Databricks.

- Experience with Machine Learning data pipelines.

- Knowledge of DevOps tools (Docker, Kubernetes, CI/CD).

- Familiarity with BI tools (Power BI / Tableau).

Education :

- B.E / B.Tech / M.Tech / MCA / M.Sc (Computer Science / IT / Data Science / Statistics)