Senior Data Scientist: Bayesian Analytics & AI Platform
Posted Updated
Description
Intellias is seeking a skilled data platform engineer to design and operationalize enterprise-grade pipelines on Databricks and AWS. You will handle pseudonymisation, entity resolution, and scalable image processing, while implementing robust MLOps and monitoring capabilities.
Responsibilities include platform setup with client teams, secure ingestion pipelines, feature engineering at scale, and privacy-focused governance with thorough documentation.
Platform setup: configure Databricks, compute, secrets and access for clients. Secure data ingestion: parsing, pseudonymisation, data quality checks. Feature engineering at scale: embedding jobs, linkage keys, pysuo. Modeling support: feature selection, clustering, comparisons and tests. Automation: orchestrate end-to-end, idempotent workflows. Monitoring: build data-quality and drift monitoring with alerts. Privacy engineering: enforce access controls and data provenance. Documentation: prepare runbooks and conduct live walkthroughs. Expertise in PySpark and Spark SQL with complex data transforms. Hands-on Databricks experience including Unity Catalog and Delta lake. Cloud and security basics in AWS: IAM, KMS, Secrets Manager. Experience with pseudonymisation and data privacy controls. Proficiency in entity resolution and blocking strategies. Ability to process images at scale using GPU pipelines. Knowledge of mixed-data clustering and Bayesian analysis basics. Strong feature engineering and model evaluation practices. Experience with MLOps, monitoring and lineage in data platforms. Clear documentation and runbook creation for handover.Skills
Sign in to read and join the discussion.