Lead Pyspark Developer
Posted Updated
Description
luxoft information technology (singapore) pte. ltd. seeks an experienced data engineering lead to build and enhance a scalable surveillance data platform. You will own PySpark-based pipelines, optimize performance, and guide a development team while coordinating with backend and data teams.
The role requires deep Python, Spark architecture knowledge, and strong SQL/dataframe experience to deliver compliant, high-quality software in a regulatory environment.
Develop software applications from business requirements and maintain them. Participate in analysis, design, development, testing, and deployment activities. Lead code reviews and propose reliable, scalable solutions. Translate business needs into technical architecture for data pipelines. Collaborate with teams, ensure coding standards and best practices. 15+ years of experience in data-intensive PySpark development and performance optimization. Proven leadership of a development team and core Python expertise. Strong understanding of data pipelines, transformations, and parallel processing. Experience with Pandas/Numpy dataframes and SQL data loading. Knowledge of distributed computing concepts and Spark architecture.Skills
Sign in to read and join the discussion.