Senior Data Engineer (Databricks & PySpark)

We are seeking an experienced Senior Data Engineer to design, build, and operate our Sustainability Data Foundation (SDF). The SDF supports trusted, scalable, and auditable sustainability reporting and analytics through a modern data platform built on Databricks.

The focus of this role is to deliver data products and pipelines that enable sustainability reporting, regulatory compliance, and advanced analytics rather than enterprise-wide IT solutions. This position requires a combination of deep technical expertise, strong delivery ownership, and a pragmatic approach to engineering.

You will join a Sustainability Data & AI team that values reliability, operational excellence, and practical solutions over unnecessary complexity. We are looking for someone who is proactive, solution-oriented, and comfortable taking ownership from design through production support. The ideal candidate independently validates outputs, identifies root causes, and ensures high-quality delivery.

• Design and implement scalable data products using Databricks, Delta Lake, and PySpark.
• Develop data models, transformation frameworks, and reusable engineering patterns.
• Build and maintain data pipelines processing complex datasets from ERP, procurement, sustainability, and external systems.
• Optimize workloads for performance, scalability, and cost efficiency.
• Design, develop, deploy, and support production-grade data solutions.
• Build Databricks-based tools, dashboards, and applications that enhance user interaction with data and streamline reporting processes.
• Implement automated data quality controls throughout the data lifecycle.
• Proactively identify and resolve data issues before they impact reporting.
• Ensure data accuracy, completeness, and trustworthiness.
• Take ownership of data quality, not just technical delivery.
• Implement CI/CD pipelines, automated deployments, testing frameworks, and Infrastructure as Code (IaC).
• Support platform governance, security controls, access management, and operational standards.
• Contribute to platform reliability, monitoring, and continuous improvement initiatives.
• Partner with sustainability experts, business analysts, product owners, and reporting teams.
• Support requirements gathering, solution design, testing, and production releases.
• Communicate risks, dependencies, and delivery status clearly and proactively.
• Ensure solutions meet business, performance, and quality expectations.
• Take full ownership of deliverables and outcomes.
• 5+ years of experience designing, developing, and operating data platforms and pipelines.
• Extensive hands-on experience with Databricks in production environments.
• PySpark
• SQL
• Strong knowledge of Git-based development and version control practices.
• Experience with automated testing, deployment pipelines, and release management.
• Solid understanding of:
• Data lineage and metadata management
• Data governance and auditing
• Data validation and reconciliation frameworks
• Strong troubleshooting and root-cause analysis skills.
• Ability to balance engineering excellence with pragmatic delivery.
• Demonstrated ownership and accountability for production services and data products.
• Proactive, self-driven, and solution-oriented mindset.
• Preferred Qualifications
• Experience with sustainability reporting, ESG data, and regulatory reporting requirements.
• Experience developing Databricks applications, dashboards, or user-facing data tools.
• Knowledge of procurement, supplier, finance, and ERP data domains.
• Experience supporting executive reporting, disclosures, or audit-related data processes.