Data Engineer Lead
Posted Updated
Description
Job Summary
We are looking for an experienced Databricks Lead to lead and manage data engineering initiatives. The ideal candidate should have strong hands-on experience with Databricks, data engineering, and cloud-based data platforms, along with the ability to guide a team, drive technical decisions, and ensure successful project delivery.
Key Responsibilities
- Lead and manage Databricks-based data engineering projects.
- Design, develop, and implement scalable data pipelines using Databricks.
- Provide technical guidance and mentorship to team members.
- Work on data ingestion, transformation, processing, and integration.
- Develop and optimize ETL/ELT workflows using PySpark and Spark SQL.
- Handle performance tuning and optimization of Databricks workloads.
- Collaborate with business stakeholders, architects, and cross-functional teams.
- Identify and resolve technical issues and ensure smooth project delivery.
- Follow best practices for data quality, security, and governance.
- Participate in technical discussions, code reviews, and solution design.
Mandatory Skills
- Strong hands-on experience with Databricks.
- Good knowledge of Apache Spark, PySpark, and Spark SQL.
- Experience in data engineering and ETL/ELT pipelines.
- Strong programming knowledge in Python and SQL.
- Experience with cloud platforms such as AWS, Azure, or GCP.
- Understanding of data lake and lakehouse architecture.
- Experience in team leadership and technical coordination.
- Strong problem-solving and communication skills.
Skills
Sign in to read and join the discussion.