Senior Data Engineer: PySpark, Databricks & Lakehouse
Summary
Senior Data Engineer at Y-Axis in Dubai who designs, builds and optimizes scalable batch and real-time data pipelines, using Python, PySpark and Databricks within a Lakehouse architecture. Focuses on data quality, security, governance and tuning Spark jobs and cloud resources while collaborating with data scientists and analysts.
Y-Axis in Dubai is seeking a Senior Data Engineer to design, develop and optimize scalable data pipelines for batch and real-time workloads. You will build ingestion frameworks and transform data using Python, PySpark, and Databricks within a Lakehouse architecture.
You will collaborate with data scientists, analysts and stakeholders to translate business needs into robust data solutions, ensuring quality, security and governance while optimizing Spark jobs and cloud resources.