Staff Engineer- Data Scientist
Posted 13
views
Description
Role & responsibilities
- 7-10 years of experience. Hands-on Azure Databricks depth: Delta Lake, medallion architecture, PySpark, cluster and job management, Unity Catalog, and performance optimisation (partitioning, caching, broadcast joins, AQE, OPTIMIZE / Z-ORDER).
- The wider Azure data stack: ADLS Gen2, Azure Data Factory, Synapse, with strong SQL and ETL/ELT design including incremental ingestion patterns. SQL Server and Key Vault exposure expected.
- Unstructured and siloed data sourcing: ingestion and remediation of unstructured content from distributed sources (scanned or flattened documents, Excel-to-PDF snapshots that resist standard OCR, SharePoint repositories, attachments) using OCR, document intelligence and AI techniques. CI/CD and environment ownership: Git, Azure DevOps, YAML pipelines, secrets and service principal management, and multi-environment promotion across Dev/QA/UAT/Prod.
- Power BI to a build-and-own standard: semantic models, Power Query/M, DAX, and deployment pipelines, including where a direct lake-to-BI connection is unavailable.
- Expert Python and applied machine learning: at least two of time series forecasting, classification, computer vision or predictive modelling, plus practical LLM/GenAI experience.
- Production data quality and governance discipline: validation frameworks, data lineage, audit logging, and role-based security. Demonstrable hands-on delivery ownership: within a centrally governed enterprise platform, with the financial acumen to size and defend the business impact of what you build.
Skills
- AI
- Azure
- Azure Data Factory
- Azure DevOps
- CI/CD
- Computer Vision
- Data Lineage
- Data Quality
- Databricks
- Delta Lake
- DevOps
- ELT
- ETL
- Generative AI
- Git
- LLM
- Machine Learning
- Medallion Architecture
- Power Query
- Power BI
- Predictive Modeling
- PySpark
- Python
- SharePoint
- SQL
- SQL Server
- Time Series
- Unity
- Unity Catalog
- Vault
- YAML
Sign in to read and join the discussion.