Hybrid Data Engineer - PySpark, Databricks & Cloud
Posted Updated
Description
Acesoft Labs (India) Pvt Ltd in Dubai invites a skilled Data Engineer to design, develop and maintain scalable data pipelines using PySpark, Spark, Python and SQL. You will build ETL/ELT processes on cloud data platforms and Databricks, with exposure to data lakes and lakehouses.
The role requires hands-on data engineering experience, strong problem solving, and collaboration with data scientists and analysts in a multicultural environment.
Design, develop and maintain scalable data pipelines and ETL/ELT workflows. Develop high-performance data processing solutions using PySpark and Apache Spark. Build data pipelines to ingest data from databases, APIs, files and other sources. Perform data cleansing, transformations, aggregations and validation. Develop and optimize Spark SQL and PySpark jobs. Work with structured, semi-structured and unstructured data. Implement data pipelines using Databricks and cloud platforms. Optimize Spark jobs for performance, scalability, memory and time. Collaborate with Data Architects, Data Scientists and Analysts. Troubleshoot production data pipelines and perform root-cause analysis. Maintain technical documentation for data pipelines and architecture. 4–8 years of experience in Data Engineering. Hands-on with PySpark and Apache Spark; strong Python and SQL. Experience developing ETL/ELT pipelines and Databricks. Experience with cloud data platforms (Azure/AWS/GCP).Skills
Sign in to read and join the discussion.