Senior Data Engineer
Summary
Build and maintain a Databricks-powered data platform using Spark, Delta Lake, and Python to power analytics and AI/ML workflows for a company migrating from AWS-native services.
Are you passionate about building modern data platforms that power analytics, artificial intelligence, and machine learning initiatives?
We are partnering with an innovative organization undergoing a major data transformation and seeking a Senior Data Engineer to join their growing team on a 6-month contract. This is an exciting opportunity to help shape the future of a modern enterprise data ecosystem as the organization transitions from AWS-native services to a Databricks-powered platform.
What You Will Be Doing:
- Design, build, and maintain scalable cloud-based data platforms and pipelines
- Develop and optimize ETL/ELT processes using modern data engineering practices
- Build and support Databricks-based solutions leveraging Spark, SparkSQL, and Delta Lake
- Enable machine learning and MLOps workflows through robust data pipelines and feature engineering processes
- Collaborate closely with Data Science, Analytics, Architecture, and Platform teams to deliver production-ready solutions
- Implement data quality, monitoring, observability, and governance best practices
- Contribute to CI/CD automation and deployment processes for data and analytics workloads
- Troubleshoot and resolve complex data and platform-related issues
- Mentor junior team members and promote engineering excellence across the organization
- Help define data architecture standards, patterns, and best practices as the platform evolves
Your Experience:
- 5+ years of experience in Data Engineering, Data Platform Engineering, or a related discipline
- Advanced proficiency with Python and SQL
- Strong experience building and supporting cloud-based data platforms within AWS environments
- Hands-on experience with Databricks, Apache Spark, Delta Lake, and modern data processing frameworks
- Experience supporting analytics, AI, machine learning, or MLOps initiatives
- Strong understanding of data modeling, data architecture, and performance optimization
- Experience implementing data quality, monitoring, and observability solutions
- Knowledge of CI/CD pipelines, Git, and modern software development practices
- Experience working with structured and semi-structured data formats including JSON, XML, and CSV
- Strong communication skills with the ability to collaborate across technical and business teams
Nice to Have
- Experience with AWS SageMaker and machine learning lifecycle management
- Database Administration (DBA) experience
- Salesforce (SFDC) data integration experience
- Experience supporting large-scale cloud migration or modernization initiatives