Python Data Engineer - Data bricks & LLM

Job Summary

We are looking for a skilled Python Data Engineer with experience in Python, SQL, data engineering, ETL pipelines, Databricks, and LLM integration. The ideal candidate should have strong data processing skills and experience integrating APIs and working with modern AI/LLM-based solutions.

Key Responsibilities

• Develop and maintain data pipelines and ETL workflows using Python, SQL, and Databricks.
• Process, transform, and analyze large datasets using Pandas and NumPy.
• Build and optimize SQL queries for data extraction and transformation.
• Integrate REST APIs and external data sources into data pipelines.
• Work on LLM-based applications and integrate AI capabilities into existing systems.
• Ensure data quality, performance, scalability, and reliability of data workflows.
• Collaborate with data engineers, data scientists, and application development teams.

Mandatory Skills

• Strong hands-on experience in Python and SQL.
• Experience in Data Engineering and ETL pipeline development.
• Working knowledge of Databricks.
• Experience with Pandas and NumPy for data manipulation.
• Hands-on experience with API integration, preferably REST APIs.
• Understanding of LLM concepts and integration.

Good to Have

• Experience with PySpark and cloud-based data platforms.
• Knowledge of LLM frameworks, prompt engineering, and Generative AI.
• Experience with data pipeline optimization and workflow orchestration.