Python PySpark Data Engineer

Interested candidate please share the below details.

Pan card number:(Mandatory)

DOB:

Total Exp:

Notice period or LWD:

Updated resume:

Please Find the Job description below:

Mandatory Skill sets

• Strong Python Programming: Core Python, OOP, multithreading/multiprocessing, memory management, exception handling, debugging, and performance optimization.
• PySpark & Apache Spark: DataFrames/RDDs, transformations, joins, partitioning, shuffle, Spark internals (DAG, Catalyst, AQE), and performance tuning
• Advanced SQL: Complex joins, window functions, CTEs, query optimization, indexing, execution plans, stored procedures, and database design.

Good to have

• ETL: ETL/ELT pipeline development, data modelling, batch & streaming processing, data lakes, Parquet/Delta Lake, and data quality frameworks.
• Cloud & Big Data Ecosystem: AWS (S3, EMR, Glue, IAM), Databricks, Airflow, Kafka, CI/CD, Git, Docker, and Infrastructure-as-Code (Terraform/CloudFormation).
• System Design & Production Support: Designing scalable data pipelines, troubles
• hooting production issues, monitoring, performance tuning, root cause analysis, and best practices for distributed systems.