Senior Data Engineer - Spark & Databricks, Global Projects
Posted Updated
Description
EPAM Systems in Mexico is seeking a seasoned data engineer to design scalable ingestion and transformation frameworks for analytics, and to build batch, near real-time, and streaming pipelines using Databricks and Apache Spark.
You will lead data governance, mentor engineers, collaborate with cross-functional teams, and help deliver end-to-end data products while advancing platform standards across the organization.
Design scalable data ingestion and transformation frameworks for analytics use cases. Build batch, near real-time, and streaming pipelines using Databricks and Apache Spark. Define Databricks best practices for Medallion layers, performance, and operations. Deliver end-to-end data products from raw ingestion through governed consumption layers. Develop trusted analytical datasets and semantic models for reporting and self-service. Implement data quality, lineage, monitoring, and observability across pipelines. Lead proof-of-concepts and evaluate new capabilities within the Databricks ecosystem. Enable AI-assisted engineering patterns and reusable accelerators for faster delivery. Coordinate with engineering and analytics stakeholders to align on delivery plans. Mentor engineers and promote consistent platform standards across teams. 3+ years of data engineering with modern data platforms. Strong leadership to drive engineering standards and mentor engineers. Proven end-to-end delivery of enterprise analytics solutions. Advanced Spark and PySpark for large-scale processing. Advanced Databricks in production, with performance tuning. Strong SQL for transformation logic and analytics datasets. Knowledge of Lakehouse and Medallion patterns. Hands-on CI/CD and Git-based SDLC. Healthcare benefits Paid time off and sick leave Upskilling and certification courses Global career opportunitiesSkills
Sign in to read and join the discussion.