Data Engineer - Artificial Intelligence/Machine Learning

Work Experience:

5 - 8 years of experience in data engineering, data platform engineering, or cloud-scale analytics solution delivery, with demonstrated ownership of production pipelines and platform components.

Must-Haves:

- Mandatory Skills: AZURE, Databricks, Kafka, PySpark, SQL, Delta Lake, Cloud Data Platforms, ETL.

- Bachelor's degree in Computer Science, Engineering, or a related field.

- Strong understanding of enterprise data architecture, cloud security, access control, CI/CD, release management, and production operations for data and AI platform solutions.

- Proven ability to independently design, build, optimise, and operate production-grade batch or streaming data pipelines, including orchestration, observability, error handling, performance tuning, and operational support.

- Hands-on experience with Python and SQL for data transformation and validation.

- Familiarity with Apache Spark (especially PySpark) and large-scale data processing concepts.

- Experience with implementing knowledge base and RAG solutions for agentic AI use cases.