Senior Data Engineer – Mexico/India (Remote)
Posted Updated
Summary
Senior data engineer leading technical design and delivery of cloud data platforms and pipelines (AWS/Azure, Databricks/Cloudera/Snowflake, Python/Scala/PySpark) while mentoring a team of 5+. Fully remote from Mexico, India, or Sri Lanka, with duties spanning architecture, code quality, data governance, and on-time project delivery.
Location: 100% Remote – Mexico, India or Sri Lanka
Experience: 8+ years of hands‑on experience in delivering data engineering solutions
Required Skills & Qualifications
- Experience in analyzing, identifying automation opportunities in a Data Platform modernization project – including accelerating ingestion frameworks, transformation stage, data quality and metadata.
- Experience leading a team of 5+ team members, responsible for technical solution, design review, code reviews and mentoring
- Strong hands‑on experience in AWS/Azure, with experience in data engineering related services and technologies likeDatabricks / Cloudera / Snowflake (Prior deep experience with Cloudera is an added advantage)
- Expertise in database concepts, SQL and relational modelling
- Good understanding and experience working with NoSQL databases
- Strong hands‑on experience in implementing data pipelines using Python, Scala and PySpark
- Good understanding of data lake and data warehouse concepts, modern data architecture patterns and industry trends in data engineering space
- Good understanding and experience implementing data quality management solutions
- Good understanding of metadata management for data engineering solutions and data platforms
- Good understanding of data governance concepts – including data stewardship, data security and data privacy management
- Experience with version control tools like Git for managing version control for source code
- Experience with DevOps processes, including experience implementing CI/CD pipelines for data engineering solution components
- Strong analytical and problem-solving skills
- Excellent communication and collaboration skills
Responsibilities:
- Lead technical design efforts for data engineering solution components, automation and accelerators
- Design and implement scalable, secure and resilient data engineering solutions on AWS/Azure, with a good understanding of data architecture patterns
- Evaluate and select appropriate technology stack for data engineering solution components – cloud native services, open-source tools, data storage, data processing frameworks etc
- Lead, mentor and manage a team of dynamic data engineers for development, testing and implementation of data pipelines ensuing quality, performance and on‑time delivery
- Responsible for detailed technical design and data mapping specification
- Ensure adherence to industry best practices, coding standards and guidelines driving continuous improvement in code quality and delivery excellence
- Optimize data solutions for performance, cost‑efficiency and scalability
- Ensure data quality and integrity for downstream systems and applications
- Plan, prioritize and manage data engineering projects, ensuring timely delivery with quality
- Identify and mitigate risks, troubleshoot issues, handle customer escalations
- Ensure alignment with business requirements and priorities, working closely with stakeholders – both internal and customer
- Maintain comprehensive technical documentation, including architecture diagrams, high level / low level data integration design, data mapping specifications, database design etc
- Working closely with peers, BI engineers and data scientists to enable advanced analytics, machine learning models and reporting solutions