Senior Data Platform Engineer
Summary
Senior Data Platform Engineer owning the design and operation of a custom metadata/catalog platform on GCP, building Python REST connectors, BPMN governance workflows, and GenAI pipelines using Collibra, BigQuery, Cloud Run, and Vertex AI.
Hybrid working model
Do you turn complex and ambiguous business requirements into scalable technical solutions?
We are looking for a Senior Data Platform Engineer to take ownership of the design and operation of a custom catalog and metadata platform. You’ll build the workflows, connectors, and pipelines that keep a modern metadata ecosystem running, while acting as a technical mentor and escalation point for engineers around you.
What you’ll be doing:
- Design and optimize enterprise governance workflows using BPMN 2.0 / Flowable Engine with Java/Groovy
- Build Python-based REST API connectors and serverless microservices on GCP Cloud Run to synchronize metadata between Collibra and source systems
- Lead impact assessments on schema and asset model changes to ensure downstream systems remain stable
- Develop metadata extraction pipelines using Dataplex Knowledge Catalog and Gemini APIs to classify unstructured content such as PDFs and wikis
- Manage CI/CD processes using GitHub Actions
- Query and optimize Collibra Insights in BigQuery for performance and cost efficiency
- Act as the escalation point for complex debugging across GCP and Collibra
- Translate technical trade-offs into clear solutions for business architects and data stewards
What you bring:
- Proven experience delivering complex, cloud-native solutions in evolving environments
- Strong analytical and communication skills, with the ability to translate business goals into scalable software solutions
- Experience mentoring developers and leading technical discussions and code reviews
Technical expertise:
- Strong GCP experience (BigQuery, Cloud Run, Logging/Monitoring, IAM)
- Advanced Python (OOP) and SQL skills Basic Java knowledge for troubleshooting
- Strong understanding of REST APIs and OAuth 2.0
- Experience with BPMN 2.0 and event-driven process design
- Experience building GenAI applications using Python, Vertex AI, LangChain, and RAG workflows
- Experience with Docker and GitHub Actions
Nice to have:
- Groovy / Flowable Engine experience within Collibra
- Knowledge of GDPR/CCPA/PCI and zero-trust IAM practices (Secret Manager, HashiCorp Vault)
- Exposure to Model Context Protocol (MCP)
Screening:
A background screening process is part of the application procedure to assess candidate reliability and integrity.