AI Data Engineer
Posted 1
view
Summary
Senior data engineer building and operating scalable batch and streaming data pipelines plus RAG/knowledge-base data solutions (embeddings, vectorisation) for a Singapore enterprise's AI transformation, using Python, SQL, Apache Spark, and cloud data platforms like Databricks, Kafka, and Delta Lake.
We're looking for a
Senior AI Data Engineer
to join a leading Singapore-based enterprise undergoing one of the region's most ambitious AI transformations.
RESPONSIBILITIES: Design, develop, and operate scalable data ingestion, transformation, and serving solutions across cloud and hybrid data environments. Build and optimise batch and streaming data pipelines to support analytics, machine learning, and AI use cases. Develop data transformation and processing solutions using technologies such as Python, SQL, and Apache Spark. Integrate data from diverse sources including APIs, databases, files, and streaming platforms. Build and operationalise knowledge bases and Retrieval-Augmented Generation (RAG) solutions to support Generative AI and agentic AI applications. Develop and maintain solutions for knowledge storage, embeddings, vectorisation, and knowledge lifecycle management. Ensure data pipelines and AI data solutions are scalable, reliable, secure, observable, and maintainable. Monitor and troubleshoot production data workflows, perform incident triage and root-cause analysis, and drive continuous improvement. Contribute to automation, CI/CD, version control, and deployment processes across data and AI platforms. Collaborate with engineers, architects, product teams, business stakeholders, and source-system owners to translate requirements into effective technical solutions. Provide technical guidance and mentorship to engineers and delivery partners, promoting reusable components, engineering best practices, and production readiness. Maintain technical documentation, metadata, data lineage, operational procedures, and solution designs. Apply data governance, security, access control, and enterprise compliance standards throughout solution design and implementation. Evaluate emerging data and AI technologies and contribute to proof-of-concepts and innovative solutions.
REQUIREMENTS: 5+ years of experience in
Data Engineering, Data Platform Engineering, Cloud Data Engineering , or a related field. Bachelor's degree in
Computer Science, Engineering, Data Science , or a related discipline. Strong hands-on experience with
Python and SQL . Experience with
Apache Spark/PySpark
and large-scale data processing. Proven experience designing, developing, and operating production-grade
batch and/or streaming data pipelines . Experience with modern cloud data platforms and technologies such as
Databricks, Kafka, Delta Lake, Microsoft Fabric , or equivalent. Experience working with
Generative AI, Retrieval-Augmented Generation (RAG), knowledge bases, embeddings, or vector databases . Understanding of data ingestion, transformation, orchestration, data quality, monitoring, and observability practices. Experience integrating data from APIs, databases, files, and other enterprise data sources. Familiarity with
CI/CD, Git, DevOps, and DataOps
practices. Understanding of enterprise data architecture, data governance, security, access control, and compliance requirements. Experience supporting production environments, including troubleshooting, incident management, root-cause analysis, and performance optimisation. Strong analytical and problem-solving skills with the ability to work through complex technical challenges. Excellent written and verbal communication skills, with the ability to collaborate effectively across technical and non-technical stakeholders. Ability to work independently, lead technical discussions, and provide guidance to other engineers. Self-motivated, adaptable, and passionate about emerging data and AI technologies.
Please contact Ella Godinez at (HIDDEN TEXT) for a confidential discussion. EA License no: 16S8066
Senior AI Data Engineer
to join a leading Singapore-based enterprise undergoing one of the region's most ambitious AI transformations.
RESPONSIBILITIES: Design, develop, and operate scalable data ingestion, transformation, and serving solutions across cloud and hybrid data environments. Build and optimise batch and streaming data pipelines to support analytics, machine learning, and AI use cases. Develop data transformation and processing solutions using technologies such as Python, SQL, and Apache Spark. Integrate data from diverse sources including APIs, databases, files, and streaming platforms. Build and operationalise knowledge bases and Retrieval-Augmented Generation (RAG) solutions to support Generative AI and agentic AI applications. Develop and maintain solutions for knowledge storage, embeddings, vectorisation, and knowledge lifecycle management. Ensure data pipelines and AI data solutions are scalable, reliable, secure, observable, and maintainable. Monitor and troubleshoot production data workflows, perform incident triage and root-cause analysis, and drive continuous improvement. Contribute to automation, CI/CD, version control, and deployment processes across data and AI platforms. Collaborate with engineers, architects, product teams, business stakeholders, and source-system owners to translate requirements into effective technical solutions. Provide technical guidance and mentorship to engineers and delivery partners, promoting reusable components, engineering best practices, and production readiness. Maintain technical documentation, metadata, data lineage, operational procedures, and solution designs. Apply data governance, security, access control, and enterprise compliance standards throughout solution design and implementation. Evaluate emerging data and AI technologies and contribute to proof-of-concepts and innovative solutions.
REQUIREMENTS: 5+ years of experience in
Data Engineering, Data Platform Engineering, Cloud Data Engineering , or a related field. Bachelor's degree in
Computer Science, Engineering, Data Science , or a related discipline. Strong hands-on experience with
Python and SQL . Experience with
Apache Spark/PySpark
and large-scale data processing. Proven experience designing, developing, and operating production-grade
batch and/or streaming data pipelines . Experience with modern cloud data platforms and technologies such as
Databricks, Kafka, Delta Lake, Microsoft Fabric , or equivalent. Experience working with
Generative AI, Retrieval-Augmented Generation (RAG), knowledge bases, embeddings, or vector databases . Understanding of data ingestion, transformation, orchestration, data quality, monitoring, and observability practices. Experience integrating data from APIs, databases, files, and other enterprise data sources. Familiarity with
CI/CD, Git, DevOps, and DataOps
practices. Understanding of enterprise data architecture, data governance, security, access control, and compliance requirements. Experience supporting production environments, including troubleshooting, incident management, root-cause analysis, and performance optimisation. Strong analytical and problem-solving skills with the ability to work through complex technical challenges. Excellent written and verbal communication skills, with the ability to collaborate effectively across technical and non-technical stakeholders. Ability to work independently, lead technical discussions, and provide guidance to other engineers. Self-motivated, adaptable, and passionate about emerging data and AI technologies.
Please contact Ella Godinez at (HIDDEN TEXT) for a confidential discussion. EA License no: 16S8066
Skills
- Agentic AI
- AI
- Analytics
- API
- Automation
- CI/CD
- Cloud
- Data Engineering
- Data Governance
- Data Ingestion
- Data Lineage
- Data Pipelines
- Data Quality
- Data Science
- Databricks
- DataOps
- Delta Lake
- DevOps
- Embeddings
- Generative AI
- Git
- Kafka
- Machine Learning
- Microsoft Fabric
- Observability
- PySpark
- Python
- RAG
- Root Cause Analysis
- Solution Design
- Spark
- SQL
- Vector Databases
- Version Control