Lead Data Engineer
Posted 7
views
Summary
Leads design, development, and optimization of ETL/ELT pipelines, data warehouses, and data models — writing and tuning advanced SQL (Snowflake), Python ETL scripting, and cloud-based orchestration workflows, while mentoring associate engineers and ensuring pipeline reliability, performance, and cost efficiency.
Duties & Responsibilities
Lead the design, development, and optimization of ETL/ELT pipelines and workflows
Write, tune, and optimize advanced SQL queries for large-scale data processing
Build and maintain data warehouses and data models (relational, dimensional, star schema)
Translate business requirements into robust, well-architected, and reusable data solutions
Partner with analysts, data scientists, and stakeholders to deliver trusted, actionable datasets
Implement and maintain unit tests, integration tests, and validation frameworks to ensure pipeline reliability
Document workflows, and design decisions to support knowledge sharing and operational continuity
Apply coding standards, CI/CD practices, version control, and peer code reviews to ensure high-quality deliverables
Proactively monitor, optimize, and troubleshoot pipelines for performance, scalability, and cost efficiency
Support deployments and handle post-production monitoring and incident resolution
Mentor associate engineers, providing technical guidance and feedback
Requirements
Basic Qualifications
Bachelor’s degree in computer science, Information Systems, Engineering, or a related field
6-10 years of hands-on experience in data engineering or related fields
Expertise in writing complex SQL queries, optimizing, using advanced functionals and turning the performance on cloud data warehouse environment.
Strong experience with data warehousing concepts, design, and implementation
Minimum 6 years of hands-on experience with Snowflake or modern cloud data warehouse
Minimum 3 years of hands-on experience in data modeling
Minimum 2 year of hands-on Python development (ETL/ELT scripting, OOP, automation)
Strong knowledge of at least one major cloud platform (AWS, Azure, or GCP)
Experience with orchestration tools (e.g., Airflow, ADF, Luigi) for workflow management
Demonstrated ability to lead and mentor teams while managing multiple priorities
Strong communication and stakeholder management skills
Strong hands-on experience with data warehousing concepts, design, and implementation
Preferred Qualifications
Hands-on experience with DBT (Data Build Tool) for data transformations
Familiarity with DevOps and CI/CD best practices in data engineering
Exposure with real-time/streaming platforms (Kafka, Spark Streaming, Flink)
Exposure to the e-commerce domain or large-scale B2B/B2C environments
Has the ability to break down problems and estimate time for development tasks
Experience mentoring and guiding junior engineers
Understands the technology landscape, up to date on current technology trends and new technology, brings new ideas to the team
Learns organization vision statement and decision-making framework. Able to understand how team and personal goals/objectives contribute to the organization vision