Senior Data Engineer
Posted 49
views
Description
Job Summary
We are looking for a Data Engineer with 36 years of experience with strong hands-on in cloud, data engineering, event streaming, software development, and emerging AI-assisted engineering practices to build and support scalable real-time data platforms.
Duties and Responsibilities - Event Streaming: Apache Kafka, Azure Event Hubs event processing.
- Azure Cloud: Azure Storage, Cosmos DB, App Service, Functions, Key Vault
- Data Engineering: Databricks, Apache Spark/PySpark, ETL/ELT pipelines, data transformation and data integration.
- Databases: SQL Server, NoSQL databases such as Cosmos DB/MongoDB, and Time-Series databases like Kusto / clickhouse.
- Programming: .NET Core / C#, Python, REST APIs and JSON.
- Containers & Kubernetes: Docker, AKS (Azure Kubernetes Service), ACR (Azure Container Registry) and containerized application deployment is must.
- DevOps & CI/CD: Azure DevOps, Git, CI/CD pipelines, build/release automation and deployment management.
- Architecture: Knowledge of Distributed systems, microservices, event-driven architecture and cloud-native applications.
- AI-Assisted Engineering: Exposure to GitHub Copilot / GitHub Agent, AI-assisted coding, code generation, debugging, test generation and developer productivity tools is a plus.
- AI & Data Engineering: Understanding of Generative AI, LLMs, AI agents, RAG, vector databases and AI-enabled data applications is an advantage.
- 36 years of relevant experience in Data Engineering / Software Engineering.
- Strong SQL and programming fundamentals.
- Hands-on experience with Azure cloud technologies.
- Practical experience with event streaming and real-time data processing.
- Experience with CI/CD and DevOps practices.
- Good understanding of NoSQL and distributed data systems.
- Exposure to AI-assisted software development and modern developer productivity tools is preferred.
- Strong analytical, debugging and problem-solving skills.
- Ability to work with high-volume, scalable and production-grade data platforms.
- Primary Technology Stack: Azure | Kafka / Event Hubs | Databricks | NoSQL | Time-Series DB | .NET Core | C# | Python | AKS | ACR | Docker | Azure DevOps | CI/CD | SQL | GitHub Copilot / Agent | GenAI
Skills
- Agentic AI
- AI
- AKS
- API
- Automation
- Azure
- Azure DevOps
- CI/CD
- ClickHouse
- Cloud
- Cloud Native
- Cosmos Db
- C#
- Data Engineering
- Databricks
- DevOps
- Distributed Systems
- Docker
- .NET
- ELT
- ETL
- Event Driven Architecture
- Generative AI
- Git
- GitHub
- Github Copilot
- JSON
- Kafka
- Kubernetes
- LLM
- Microservices
- MongoDB
- NoSQL
- PySpark
- Python
- REST
- Spark
- SQL
- SQL Server
- Time Series
- Vault
- Vector Databases