Data Engineer Lead dl
Summary
Lead a remote team to design and migrate scalable data pipelines and warehouses using Databricks, Spark, and AWS, while aligning technical decisions with business goals.
π What We Do
- Leveraging our expertise, we build modern Machine Learning systems for demand planning and budget forecasting.
- Developing scalable data infrastructures, we enhance high-level decision-making, tailored to each client.
- Offering comprehensive Data Engineering and custom AI solutions, we optimize cloud-based systems.
- Using Generative AI, we help e-commerce platforms and retailers create higher-quality ads, faster.
- Building deep learning models, we enhance visual recognition and automation for various industries, improving product categorization, quality control, and information retrieval.
- Developing recommendation models, we personalize user experiences in e-commerce, streaming, and digital platforms, driving engagement and conversions.
π Our Partnerships
- Amazon Web Services
- Astronomer
- Databricks
π Our Values
- π We are Data Nerds
- π€ We are Open Team Players
- π We Take Ownership
- π We Have a Positive Mindset
Responsibilities π€
- Lead end-to-end data migration initiatives, ensuring a smooth, secure, and business-aligned transition.
- Define and communicate data architecture decisions, evaluating impacts, risks, and performance considerations across cloud data platforms such as AWS Redshift, Databricks, and related technologies.
- Coordinate the discovery and documentation of dependencies, schema inventories, data flows, and critical processes involved in migration efforts.
- Own migration and cutover planning, ensuring all activities, validations, and contingency measures are properly defined and executed.
- Serve as the primary liaison between engineering, analytics, product, and business stakeholders, facilitating clear communication and alignment throughout the project lifecycle.
- Translate technical changes in data pipelines, data warehouses, and platform architecture into clear business impact assessments.
- Define and document assumptions, requirements, and acceptance criteria to ensure successful migration outcomes.
- Oversee data validation processes to guarantee the accuracy, consistency, and integrity of migrated datasets against source-of-truth systems.
- Proactively identify, assess, and mitigate migration-related risks, escalating blockers and dependencies when necessary to maintain project timelines and quality standards.
- Drive problem-solving and decision-making in fast-paced, ambiguous environments, helping cross-functional teams stay focused on priorities and deliverables.
- Create, maintain, and improve project documentation, including migration runbooks, testing strategies, rollback plans, and operational procedures.
- Provide regular project status updates to technical and executive stakeholders, ensuring transparency around progress, risks, and next steps.
- Promote data governance, quality, and operational best practices throughout the migration lifecycle.
- Align engineers, analysts, and product stakeholders around priorities, timelines, and delivery expectations to ensure successful project execution.
Required Skills π»
- Experience in Data Engineering, including building and optimizing data pipelines.
- Experience working with Databricks, Python, and Spark, building batch and streaming.
- Intermediate English Level.
- Familiarity with a BI visualization tool like Looker or Tableau.
π Perks
- Remote-first culture β work from anywhere! π
- AWS, DBT, Google Cloud, Azure & Databricks certifications fully covered
- In-Company English Lessons.
- Birthday off + an extra vacation week (Mutt Week! ποΈ)
- Referral bonuses β help us grow the team & get rewarded!
- Maslow: Monthly credits to spend in our benefits marketplace.
- βοΈποΈ Annual Mutters' Trip β an unforgettable getaway with the team.
- πΆ Monthly Childcare Reimbursement β Because supporting families matters too
Originally posted on Himalayas