Technical Manager/ Data Project Manager (Lead Data engineer)
Summary
Lead data engineering teams to build and migrate data lakes, warehouses, and lakehouses using PySpark, Python, and cloud platforms like AWS/Azure/GCP.
Responsibilities
- Lead and manage large-scale Data Engineering and Data Modernization projects.
- Drive end-to-end delivery of Data Lake build and migration initiatives.
- Lead PySpark migration and optimization projects, ensuring performance and scalability.
- Design and implement modern data architectures, including Data Lakes, Data Warehouses, and Lakehouse solutions.
- Collaborate with business stakeholders, architects, and engineering teams to define data strategies and roadmaps.
- Provide technical leadership and mentorship to Data Engineers and Developers.
- Ensure best practices around data governance, security and compliance, and performance optimization.
- Lead data platform modernization initiatives across cloud environments.
- Review solution designs, architecture documents, and implementation approaches.
- Manage project planning, resource allocation, risks, and delivery timelines.
- Drive Agile delivery and ensure successful project execution.
Requirements
- 8+ years of experience in Data Engineering and Data Platform delivery.
- Minimum 3+ years of experience in a Technical Manager or Lead role.
- Proven experience in delivering:
- Data Lake implementation projects
- Data Lake migration programs
- PySpark migration projects
- Large-scale data transformation initiatives
Technical Skills
- Strong expertise in:
- Python
- PySpark
- Spark SQL
- SQL
- ETL/ELT frameworks
- Experience with:
- Hadoop ecosystem
- Data Lakes and Lakehouse architectures
- Distributed data processing frameworks
- Strong understanding of:
- Data modeling
- Data integration patterns
- Batch and real-time processing
- Experience with cloud platforms such as:
- AWS
- Azure
- GCP
- Hands‑on experience with:
- Data migration strategies
- Performance tuning and optimisation
- CI/CD and DevOps practices for data platforms
Must Have
- Hands‑on experience in managing Data projects end-to-end — effort estimation, scoping, project plan, timelines, team allocation, stakeholder management.
- Hands‑on experience with modern Data technologies like PySpark, SQL, CML, Python on any cloud; preferred GCP.
- Transform business requirements into Data solutions, manage risks, issues and dependencies.