Senior Data Engineer
Posted 1
view
Summary
Senior data engineer (Senior FDE) who designs Databricks/Spark lakehouse solutions, leads data/cloud migration and modernization projects, builds batch and streaming pipelines, and contributes to GenAI/RAG and MLOps work while engaging directly with customers. Core stack: Databricks, Spark/PySpark, Delta Lake, Unity Catalog, AWS/Azure/GCP, and Python.
We are hiring Senior FDE professionals with strong expertise in Databricks, Apache Spark, Data Engineering, Cloud, Data Migration and GenAI/RAG.
Required Skills :
- 10+ years overall experience with 7+ years in Data Engineering.
- Strong hands-on experience with Databricks.
- Strong expertise in Apache Spark, PySpark & Spark SQL.
- Hands-on experience with Data Migration / Cloud Migration / Data Modernization.
- Strong Data Architecture / Solution Architecture experience.
- Experience with AWS, Azure or GCP.
- Strong understanding of Delta Lake & Unity Catalog.
- Experience with batch & streaming data processing.
- Strong Python/PySpark coding skills.
- Experience with CI/CD and DevOps.
- Exposure to GenAI, LLM, RAG and MLOps/MLflow.
- Strong production troubleshooting, debugging and RCA skills.
- Customer-facing experience with strong stakeholder communication and technical solutioning.
Key Responsibilities :
- Design and implement scalable Databricks & Spark-based data solutions.
- Lead data migration and modernization initiatives.
- Design enterprise Lakehouse and data architectures.
- Develop production-grade data pipelines and integrations.
- Optimize Spark workloads and troubleshoot complex production issues.
- Implement batch and streaming data solutions.
- Work with Unity Catalog, Delta Lake and Databricks ecosystem.
- Contribute to GenAI/RAG and MLOps solutions.
- Engage with customers and stakeholders to understand requirements and provide technical solutions.
- Drive architecture discussions, technical design and end-to-end implementation.