Engineering Manager
Summary
Data Engineering Manager (12–16 yrs experience) in Gurugram, hybrid, leading and mentoring a team of data engineers while staying hands-on. Core stack: PySpark, Apache Spark, and Apache Airflow, building large-scale ETL/ELT pipelines and data platforms on GCP (BigQuery, GCS, Dataflow, Dataproc, Pub/Sub).
Job Description – Data Engineering Manager
Experience: 12–16 Years
Location: Gurugram
Work Mode: Hybrid
Industry Preference: Product-Based Companies Only
Mandatory Cloud Platform: Google Cloud Platform (GCP)
About the Role
We are looking for an experienced and technically strong Data Engineering Manager to lead and drive our data engineering initiatives. The ideal candidate should have extensive hands-on experience in designing, developing, and managing scalable data engineering solutions.
The candidate must possess strong expertise in PySpark, Apache Spark, Apache Airflow, and Google Cloud Platform (GCP). This role requires a combination of technical leadership, architectural understanding, team management, and hands-on problem-solving capabilities.
Candidates must have prior experience working with product-based organizations.
Key Responsibilities
- Lead, manage, and mentor a team of data engineers to deliver scalable and high-quality data solutions.
- Design, develop, and implement robust data pipelines and data processing frameworks.
- Drive data engineering initiatives using PySpark, Apache Spark, and Apache Airflow.
- Architect and manage scalable data platforms and solutions on Google Cloud Platform (GCP).
- Oversee the development and optimization of ETL/ELT workflows and data integration processes.
- Ensure data quality, reliability, scalability, and performance across data platforms.
- Collaborate with product managers, architects, data scientists, and other engineering teams.
- Identify opportunities to improve data processing efficiency and optimize cloud infrastructure costs.
- Establish engineering best practices, coding standards, and data governance practices.
- Take ownership of end-to-end data engineering delivery, technical decisions, and project execution.
Required Technical Skills
1. Data Engineering
- Strong hands-on experience in Data Engineering.
- Expertise in building large-scale data pipelines.
- Strong understanding of ETL/ELT processes.
- Experience with structured and unstructured data processing.
- Strong understanding of data warehousing and data architecture.
2. Big Data Technologies
- Excellent hands-on experience with PySpark.
- Strong knowledge of Apache Spark.
- Experience with distributed data processing and large-scale data transformations.
3. Workflow Orchestration
- Hands-on experience with Apache Airflow.
- Experience designing, scheduling, monitoring, and managing complex data workflows.
4. Cloud Platform – GCP (Mandatory)
- Strong hands-on experience with Google Cloud Platform.
- Experience with GCP-based data engineering solutions.
- Knowledge of services such as BigQuery, Google Cloud Storage (GCS), Dataflow, Dataproc, and Pub/Sub.
- Experience designing and deploying scalable cloud-based data solutions.
5. Leadership & Management
- Experience managing and mentoring data engineering teams.
- Strong technical decision-making and problem-solving capabilities.
- Ability to drive projects independently and manage multiple priorities.
- Excellent communication and stakeholder management skills.
Candidate Requirements
- 12–16 years of overall relevant experience in Data Engineering.
- Must have strong hands-on experience in PySpark, Apache Spark, and Airflow.
- GCP experience is mandatory.
- Must have worked with product-based companies.
- Previous experience in leading or managing data engineering teams.
- Strong understanding of modern data engineering architectures and cloud-based data platforms.
Job Details
ParameterDetails
Position
Data Engineering Manager
Experience
12–16 Years
Location
Gurugram
Work Mode
Hybrid
Cloud
GCP (Mandatory)
Primary Skills
PySpark, Spark, Airflow
Company Background
Product-Based Companies Only
Important Hiring Criteria
- GCP experience is mandatory.
- Candidates must have experience working in product-based companies only.
- Strong hands-on expertise in PySpark, Spark, and Airflow is required.
- Candidates should have relevant technical leadership and team management experience.
Interested candidates can share their updated resumes at: sachin@questhiring.com
Skills
Sign in to read and join the discussion.