Ai data engineer
Summary
Designs and maintains AI-ready data pipelines and feature stores to power ML models and generative AI solutions, ensuring data quality, lineage, and compliance.
Job Purpose
The Manager: AI Data Engineer is responsible for designing, building, and managing data pipelines and AI-ready data products that support artificial intelligence and advanced analytics initiatives across the organization.
The role focuses on enabling feature engineering, curated datasets, real‑time and batch data pipelines, and data quality controls that support machine learning models, generative AI solutions, and AI-enabled business services.
The position works closely with AI Platform Engineering, Data Platform teams, MLOps, AI Application Engineers, Security & Risk, and business stakeholders to ensure AI solutions are powered by accurate, secure, compliant, and production‑ready data.
Key Performance Areas (KPAs) 1. AI Data Architecture & Pipelines Design and implement scalable data pipelines to support: Model training Feature generation Inference and real‑time decision-making Build and maintain: Batch data pipelines Streaming data pipelines Data ingestion from internal and external sources Ensure data architectures align with: Enterprise AI reference architectures Organizational data and platform standards 2. Feature Stores & AI Data Products Design, build, and manage feature stores that support: Reuse of engineered features Consistency between model training and inference Create curated AI-ready datasets for: Data Scientists AI Engineers Product Teams Analytics Teams Improve discoverability and reuse of AI data assets across the organization. 3. Data Quality, Lineage & Observability Implement automated validation for: Data accuracy Completeness Timeliness Consistency Maintain end-to-end data lineage and traceability. Build observability into AI data pipelines to: Detect data drift Identify anomalies Support root cause analysis Collaborate with Site Reliability Engineering (SRE) and Platform teams to ensure operational stability. 4. Privacy, Security & Regulatory Compliance Enforce data privacy, sovereignty, and protection requirements. Implement appropriate access controls, masking, and encryption where required. Ensure AI datasets comply with: Applicable regulatory requirements Organizational security and governance standards Support internal and external audit activities related to AI data usage. 5. Enablement of AI & Analytics Use Cases Partner with AI Application and MLOps teams to: Enable rapid experimentation Accelerate production deployment of AI solutions Reduce data preparation effort for AI initiatives Support: Generative AI solutions Machine Learning workloads Advanced Analytics initiatives Balance innovation with strong governance and operational discipline. 6. Continuous Improvement & Standardization Standardize AI data engineering practices, patterns, and tooling. Contribute to enterprise AI and data platform roadmaps. Drive continuous improvement in: Pipeline reliability Data freshness Scalability Performance Reusability of AI data products Job Requirements Education Master’s Degree in Computer Science, Data Science, Artificial Intelligence, Big Data, Information Systems, Engineering, or a related discipline. Experience Minimum of 5 years’ experience in Data Engineering or Data Platform roles. Hands‑on experience designing and implementing: Large‑scale batch and streaming data pipelines Feature engineering pipelines Experience working with: Cloud data platforms (Azure is essential) Structured and unstructured data Exposure to Artificial Intelligence, Machine Learning, or Advanced Analytics environments is preferred. Experience working within highly regulated industries is advantageous. Technical Competencies Data pipeline architecture and orchestration Feature Store design and implementation Streaming and batch data processing Data quality frameworks Data lineage and observability Cloud-native data platforms Data modelling techniques Security and privacy‑by‑design principles AI data lifecycle management Skills Strong analytical and problem‑solving abilities Excellent data modelling capabilities Technical documentation and communication skills Cross‑functional collaboration with AI, platform, engineering, and business teams Ability to balance innovation with governance Continuous improvement mindset Behavioural Competencies Detail‑oriented with a strong focus on quality Accountable and delivery‑driven Structured and methodical approach to work Curious with a passion for learning emerging technologies Collaborative and respectful team player Comfortable working across multiple business units and large-scale environments