AI Engineer

AI Developer – Azure Databricks, Python, Java, Microservices & AI (LLM/Copilot or Equivalent )

Job Summary

We are looking for a AI Developer with strong expertise in Python, Azure Databricks (ADB), Delta Lake, and AI/LLM technologies to design and build Databricks jobs, reusable scripts, and cloud-native microservices. The role requires hands-on experience with Copilot agentic mode (or equivalent), building LLM-driven AI use cases, and the ability to work independently by taking full ownership of deliverables without the need for micromanagement.

Roles and Responsibilities

Azure Databricks & Data Engineering

• Design, develop, and manage Azure Databricks jobs, workflows, and notebooks.
  
• Build reusable Python/PySpark scripts for data ingestion, transformation, and analytics.
  
• Implement Delta Lake–based lakehouse architectures with ACID transactions and versioning.
  
• Optimize Databricks workloads for performance, scalability, and cost efficiency.
  
• Implement batch and streaming pipelines using Databricks.
  

Microservices & Application Development

• Develop Python-based microservices to expose data and AI capabilities via APIs.
  
• Build and integrate RESTful services for data access, AI inference, and orchestration.
  
• Ensure secure, scalable, and resilient service design using cloud-native patterns.
  
• Drive development tasks independently, from design through deployment, with minimal supervision.
  

AI, LLMs & Copilot Agentic Mode

• Design and build LLM-powered AI use cases such as copilots, intelligent assistants, and automation agents.
  
• Implement Copilot agentic mode workflows, including tool calling, task planning, and multi-step reasoning.
  
• Develop prompt engineering strategies, system prompts, and guardrails for enterprise use.
  
• Integrate LLMs with Databricks, data lakes, and microservices for real-time and batch inference.
  
• Build RAG (Retrieval-Augmented Generation) pipelines using enterprise data.
  
• Evaluate, fine-tune, and monitor LLM performance, accuracy, and cost.
  

Architecture, Governance & Best Practices

• Contribute to solution architecture and technical design decisions.
  
• Ensure data security, governance, and compliance in data and AI solutions.
  
• Implement logging, monitoring, and observability for Databricks jobs and AI services.
  
• Perform code reviews and enforce engineering best practices.
  

Leadership & Collaboration

• Mentor peer developers and provide technical guidance when needed.
  
• Collaborate effectively with data scientists, architects, DevOps teams, and business stakeholders.
  
• Clearly communicate technical concepts, progress, and risks through excellent verbal and written communication skills.
  
• Take full ownership of assigned workstreams and drive them to completion without requiring micromanagement.
  

Required Technologies & Skills

Programming & Frameworks

• Python (advanced)
  
• PySpark / Spark SQL
  
• REST APIs and microservices frameworks (FastAPI, Flask, etc.)
  

Azure & Data Platforms

• Azure Databricks (ADB)
  
• Delta Lake
  
• Azure Data Lake Storage (ADLS Gen2)
  
• Azure cloud services and security fundamentals
  

AI / LLM & Copilot

• Hands-on experience with LLMs (Azure OpenAI, OpenAI, or similar)
  
• Experience building Copilot solutions and agentic workflows
  
• Prompt engineering and orchestration frameworks (LangChain, Semantic Kernel, etc.)
  
• Experience with RAG architectures, vector databases, and embeddings
  
• Understanding of AI safety, guardrails, and responsible AI practices
  

DevOps & Engineering

• CI/CD pipelines (Azure DevOps, GitHub Actions)
  
• Git version control
  
• Containerization (Docker preferred)
  
• Infrastructure-as-code concepts (Terraform/Bicep – preferred)
  

Preferred Qualifications

• 8 to 10 years of experience in Python, data engineering, or backend development
  
• Proven experience delivering production-grade AI/LLM solutions
  
• Experience in enterprise-scale cloud environments
  
• Exposure to streaming platforms (Kafka, Event Hubs – preferred)
  

Soft Skills

• Excellent verbal and written communication skills
  
• Strong sense of ownership and accountability
  
• Ability to work independently and proactively drive outcomes
  
• Strong problem-solving and analytical skills
  
• Collaborative mindset with leadership capabilities