Senior Data Engineer (Python, PySpark, Kafka)
**Key Responsibilities:**
- Develop and maintain robust Python code for data manipulation, scripting, and application development
- Design and build high-performance data transformation jobs using PySpark on large-scale datasets
- Build and maintain real-time data streaming pipelines using Apache Kafka, including Kafka Connect and Kafka Streams implementations
- Work with the Hadoop ecosystem, including HDFS, YARN, Hive, and related technologies
- Extract, query, and manage data using SQL and relational databases (e.g., PostgreSQL, SQL Server)
- Apply data warehousing concepts and dimensional modeling principles to optimize data architecture
- Manage code repositories and collaborate with teams using version control systems (e.g., Git, GitHub)
B.Tech, M.Tech
6 to 9 Years