Hands-On Data Engineer with Spark Skills
Summary
Build and optimize cloud-based Spark/PySpark pipelines for large financial datasets using Parquet and Iceberg to power portfolio management systems.
Step into the role of Data Engineer with Total Fund Management's Portfolio Management Technology team. Focus on optimizing cloud-based data systems utilizing Spark/PySpark and modern storage techniques.
You'll develop and optimize Spark workloads tailored for cloud environments, working with significant datasets in formats like Parquet and Iceberg. The role emphasizes translating requirements into effective engineering solutions while ensuring system reliability and performance.
Key Responsibilities:
• Optimize and develop Spark workloads in cloud setups
• Work with large datasets using Parquet and Iceberg technologies
• Translate specifications into strong engineering solutions
• Guarantee performance, reliability, and maintainability standards
• Participate in monitoring, documentation, and deployment processes
Requirements:
• Strong knowledge of Python and PySpark
• Proficient with Apache Spark, ideally in cloud environments
• Experience with Databricks
• Expertise in large-scale data systems
• Previous experience in client-facing roles
Utilize your expertise to drive innovative data solutions with Total Fund Management.