Ai data engineer
Posted Updated
Job Purpose
The Manager: AI Data Engineer is responsible for designing, building, and managing data pipelines and AI-ready data products that support artificial intelligence and advanced analytics initiatives across the organization. The role focuses on enabling feature engineering, curated datasets, real‑time and batch data pipelines, and data quality controls that support machine learning models, generative AI solutions, and AI-enabled business services. The position works closely with AI Platform Engineering, Data Platform teams, MLOps, AI Application Engineers, Security & Risk, and business stakeholders to ensure AI solutions are powered by accurate, secure, compliant, and production‑ready data. Key Performance Areas (KPAs)
1. AI Data Architecture & Pipelines
Design and implement scalable data pipelines to support:
Model training Feature generation Inference and real‑time decision-making
Build and maintain:
Batch data pipelines Streaming data pipelines Data ingestion from internal and external sources
Ensure data architectures align with:
Enterprise AI reference architectures Organizational data and platform standards
2. Feature Stores & AI Data Products
Design, build, and manage feature stores that support:
Reuse of engineered features Consistency between model training and inference
Create curated AI-ready datasets for:
Data Scientists AI Engineers Product Teams Analytics Teams
Improve discoverability and reuse of AI data assets across the organization. 3. Data Quality, Lineage & Observability
Implement automated validation for:
Data accuracy Completeness Timeliness Consistency
Maintain end-to-end data lineage and traceability. Build observability into AI data pipelines to:
Detect data drift Identify anomalies Support root cause analysis
Collaborate with Site Reliability Engineering (SRE) and Platform teams to ensure operational stability. 4. Privacy, Security & Regulatory Compliance
Enforce data privacy, sovereignty, and protection requirements. Implement appropriate access controls, masking, and encryption where required. Ensure AI datasets comply with:
Applicable regulatory requirements Organizational security and governance standards
Support internal and external audit activities related to AI data usage. 5. Enablement of AI & Analytics Use Cases
Partner with AI Application and MLOps teams to:
Enable rapid experimentation Accelerate production deployment of AI solutions Reduce data preparation effort for AI initiatives
Support:
Generative AI solutions Machine Learning workloads Advanced Analytics initiatives
Balance innovation with strong governance and operational discipline. 6. Continuous Improvement & Standardization
Standardize AI data engineering practices, patterns, and tooling. Contribute to enterprise AI and data platform roadmaps. Drive continuous improvement in:
Pipeline reliability Data freshness Scalability Performance Reusability of AI data products
Job Requirements
Education
Master’s Degree in Computer Science, Data Science, Artificial Intelligence, Big Data, Information Systems, Engineering, or a related discipline. Experience
Minimum of
5 years’ experience
in Data Engineering or Data Platform roles. Hands‑on experience designing and implementing:
Large‑scale batch and streaming data pipelines Feature engineering pipelines
Experience working with:
Cloud data platforms (Azure is essential) Structured and unstructured data
Exposure to Artificial Intelligence, Machine Learning, or Advanced Analytics environments is preferred. Experience working within highly regulated industries is advantageous. Technical Competencies
Data pipeline architecture and orchestration Feature Store design and implementation Streaming and batch data processing Data quality frameworks Data lineage and observability Cloud-native data platforms Data modelling techniques Security and privacy‑by‑design principles AI data lifecycle management Skills
Strong analytical and problem‑solving abilities Excellent data modelling capabilities Technical documentation and communication skills Cross‑functional collaboration with AI, platform, engineering, and business teams Ability to balance innovation with governance Continuous improvement mindset Behavioural Competencies
Detail‑oriented with a strong focus on quality Accountable and delivery‑driven Structured and methodical approach to work Curious with a passion for learning emerging technologies Collaborative and respectful team player Comfortable working across multiple business units and large-scale environments
The Manager: AI Data Engineer is responsible for designing, building, and managing data pipelines and AI-ready data products that support artificial intelligence and advanced analytics initiatives across the organization. The role focuses on enabling feature engineering, curated datasets, real‑time and batch data pipelines, and data quality controls that support machine learning models, generative AI solutions, and AI-enabled business services. The position works closely with AI Platform Engineering, Data Platform teams, MLOps, AI Application Engineers, Security & Risk, and business stakeholders to ensure AI solutions are powered by accurate, secure, compliant, and production‑ready data. Key Performance Areas (KPAs)
1. AI Data Architecture & Pipelines
Design and implement scalable data pipelines to support:
Model training Feature generation Inference and real‑time decision-making
Build and maintain:
Batch data pipelines Streaming data pipelines Data ingestion from internal and external sources
Ensure data architectures align with:
Enterprise AI reference architectures Organizational data and platform standards
2. Feature Stores & AI Data Products
Design, build, and manage feature stores that support:
Reuse of engineered features Consistency between model training and inference
Create curated AI-ready datasets for:
Data Scientists AI Engineers Product Teams Analytics Teams
Improve discoverability and reuse of AI data assets across the organization. 3. Data Quality, Lineage & Observability
Implement automated validation for:
Data accuracy Completeness Timeliness Consistency
Maintain end-to-end data lineage and traceability. Build observability into AI data pipelines to:
Detect data drift Identify anomalies Support root cause analysis
Collaborate with Site Reliability Engineering (SRE) and Platform teams to ensure operational stability. 4. Privacy, Security & Regulatory Compliance
Enforce data privacy, sovereignty, and protection requirements. Implement appropriate access controls, masking, and encryption where required. Ensure AI datasets comply with:
Applicable regulatory requirements Organizational security and governance standards
Support internal and external audit activities related to AI data usage. 5. Enablement of AI & Analytics Use Cases
Partner with AI Application and MLOps teams to:
Enable rapid experimentation Accelerate production deployment of AI solutions Reduce data preparation effort for AI initiatives
Support:
Generative AI solutions Machine Learning workloads Advanced Analytics initiatives
Balance innovation with strong governance and operational discipline. 6. Continuous Improvement & Standardization
Standardize AI data engineering practices, patterns, and tooling. Contribute to enterprise AI and data platform roadmaps. Drive continuous improvement in:
Pipeline reliability Data freshness Scalability Performance Reusability of AI data products
Job Requirements
Education
Master’s Degree in Computer Science, Data Science, Artificial Intelligence, Big Data, Information Systems, Engineering, or a related discipline. Experience
Minimum of
5 years’ experience
in Data Engineering or Data Platform roles. Hands‑on experience designing and implementing:
Large‑scale batch and streaming data pipelines Feature engineering pipelines
Experience working with:
Cloud data platforms (Azure is essential) Structured and unstructured data
Exposure to Artificial Intelligence, Machine Learning, or Advanced Analytics environments is preferred. Experience working within highly regulated industries is advantageous. Technical Competencies
Data pipeline architecture and orchestration Feature Store design and implementation Streaming and batch data processing Data quality frameworks Data lineage and observability Cloud-native data platforms Data modelling techniques Security and privacy‑by‑design principles AI data lifecycle management Skills
Strong analytical and problem‑solving abilities Excellent data modelling capabilities Technical documentation and communication skills Cross‑functional collaboration with AI, platform, engineering, and business teams Ability to balance innovation with governance Continuous improvement mindset Behavioural Competencies
Detail‑oriented with a strong focus on quality Accountable and delivery‑driven Structured and methodical approach to work Curious with a passion for learning emerging technologies Collaborative and respectful team player Comfortable working across multiple business units and large-scale environments