Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

HCL Technologies Limited

Senior Technical Lead

Posted Updated

Description

HCL Technologies Middle East FZLLC Dubai Branch is seeking a Senior PySpark Developer to design, develop, and maintain large-scale data processing systems using PySpark to ingest, transform, and analyze massive data volumes.

You will build scalable ETL/ELT pipelines, optimize Spark jobs, shape data models, and integrate Spark apps with AWS S3, Azure ADLS, and GCP Dataproc, while applying CI/CD practices and unit testing.

Design and build scalable ETL/ELT pipelines to move and transform data. Develop PySpark scripts for large-scale data processing (DataFrames, Datasets, Spark SQL). Tune Spark jobs for performance, partitions, caching, and shuffles. Model data schemas and storage formats (Parquet, Delta Lake, ORC). Integrate Spark with cloud storage (S3/ADLS/Blob) and databases. Automate data workflows with CI/CD practices and testing. Extensive PySpark ETL/ELT pipeline experience. Strong SQL and data modeling skills. Cloud data platform experience (AWS/Azure/GCP). Familiarity with Parquet and Delta Lake formats. Hands-on CI/CD and data workflow automation.

Skills

Apply

See also