Senior AI Solution Architect
Summary
Designs and architects AI/ML solutions integrating batch and real-time inference using Spark, Kafka, and Jupyter Enterprise Gateway within a data lakehouse.
Follow us on LinkedIn to get job related updates.
Our client is a custom software development company that helps businesses streamline operations and accelerate growth through modern technology solutions. They specialize in building scalable applications tailored to client needs.
Role Introduction
Senior AI Solution Architect will designs AI/ML solution patterns that integrate into the existing data platform, leveraging Jupyter Enterprise Gateway, Spark, and Kafka for both real-time and batch inference scenarios.
Features
- Fulltime
- Onsite
Requirements
- Design end-to-end AI/ML solution architecture that plugs into the existing Iceberg/Trino/Spark lakehouse.
- Define patterns for both batch inference (via Spark) and real-time inference (via Kafka-based streaming).
- Architect the integration between Jupyter Enterprise Gateway-based development environments and production inference infrastructure.
- Evaluate and select ML serving, feature engineering, and model lifecycle tooling appropriate for the platform.
- Partner with the Data Architect to ensure AI/ML workloads don't conflict with core data platform SLAs or resource allocation.
- Define reference architectures and reusable patterns for common AI use cases relevant to the client's domain.
- Set standards for model governance, monitoring, and explainability appropriate to the deployment context.
- Present architectural recommendations to both technical teams and client stakeholders.
Specifications
- 8+ years in software/data engineering with at least 5 years specifically in ML/AI solution architecture (not just model building).
- Hands-on experience designing both batch and real-time inference architectures using Spark and Kafka respectively.
- Familiarity with Jupyter Enterprise Gateway or similar multi-tenant notebook infrastructure in an enterprise setting.
- Strong understanding of how AI/ML workloads integrate with an existing data lakehouse (Iceberg/Trino/Spark) without disrupting core platform operations.
- Experience with at least one major ML framework (PyTorch or TensorFlow) at an architectural/integration level.
- Strong stakeholder communication skills - this role bridges data platform, ML engineering, and client leadership.
Expertise
Skills: Batch Inference Patterns, Jupyter Enterprise Gateway, Kafka, Spark
About TalentHue
TalentHue provides scalable, reliable Tech Recruitment, Corporate Recruitment and Consulting (Strategy, Operations, Performance) services. Our Recruitment and HR consultants will work alongside your team to meet the unique needs of your business.