Senior Data Engineer – Python, Spark, AWS
Summary
Senior Data Engineer builds and maintains scalable data pipelines and lakehouse infrastructure using Python, Spark, AWS, and Delta Lake to enable data ingestion, transformation, and distribution for internal and external stakeholders.
We’re looking for a Senior Data Engineer to join a highly skilled team responsible for building a scalable, multitenant data platform that powers data ingestion, transformation, cataloging, and distribution for internal and external stakeholders. This role is central to designing robust data pipelines, integrating modern data lakehouse components, and supporting the company’s AWS-based infrastructure.
- 5+ years of experience in Python and strong software engineering skills
- Solid knowledge of AWS cloud services and best practices
- Experience building scalable Spark pipelines in PySpark or Scala
- Practical experience with Spark Streaming for low-latency pipelines
- Familiarity with Delta Lake and modern data lakehouse architectures
- Hands-on experience with Kubernetes and container orchestration
- Experience with MongoDB and other NoSQL data stores
- Understanding of microservices and event-driven architecture
- Excellent communication skills and ability to work in a collaborative, global team
- Commitment to high standards of ethics, quality, and delivery
Nice to Have:
- Familiarity with Machine Learning, AI models, or Data Science workflows