Senior Data Engineer
Posted Updated 18
views
Description
- 8+ years in data engineering/data warehousing; 3+ years delivering Azure solutions and production
- Microsoft Fabric experience
- Advanced T-SQL, Spark SQL, Python/PySpark; working knowledge of KQL and DAX
- Deep experience with Delta Lake, Spark tuning, medallion/lakehouse patterns, CDC and incremental loading
- Strong dimensional modeling and working knowledge of Data Vault 2.0
- Practical CI/CD, Git, automated testing, infrastructure as code, DataOps, governance and security experience
- Professional-services/client-facing delivery experience
- Authorization to work in the US and ability to travel up to 15%
- Preferred: DP-700, DP-600, DP-203; streaming/event-driven, RAG/GenAI, Fabric IQ/data agents, and industry experience
- Design, build and maintain production-grade batch, incremental, real-time and near-real-time data pipelines across Microsoft Fabric and Azure
- Implement CDC, watermarking, merge/upsert, late-arriving-data and idempotent reprocessing patterns
- Develop PySpark, Spark SQL and T-SQL transformations across bronze, silver and gold Delta Lake layers
- Build Fabric solutions using OneLake, Lakehouse, Warehouse, Data Factory, Dataflows Gen2, Spark notebooks, Real-Time Intelligence and Power BI
- Implement dimensional and Data Vault models, data-quality testing, reconciliation, observability, logging, alerting and SLA monitoring
- Establish Git-based CI/CD and automated deployment using Azure DevOps or GitHub, Fabric deployment pipelines, REST APIs, Bicep and Terraform
- Apply Purview catalog/lineage, least-privilege access, sensitivity controls, encryption and secure connectivity
- Support migrations from SQL Server, SSIS, SSAS, legacy warehouses and Synapse to Fabric/Azure, including conversion, reconciliation and cutover
- Prepare governed data for AI/ML and RAG workloads, including ingestion, chunking, embeddings, indexing and telemetry
- Lead engineering workstreams, estimates, risk management, client communication, mentoring, reusable accelerators and operational handover
Skills
- AI
- API
- Azure
- Azure Data Factory
- Azure DevOps
- Bicep
- CI/CD
- Data Engineering
- Data Pipelines
- Data Quality
- Data Vault
- Data Warehousing
- DataOps
- Delta Lake
- DevOps
- Dimensional Modeling
- Embeddings
- Event Driven Architecture
- Generative AI
- Git
- GitHub
- Infrastructure as Code
- KQL
- Lakehouse
- Machine Learning
- Microsoft Fabric
- Observability
- Power BI
- PySpark
- Python
- REST
- Spark
- SQL
- SQL Server
- SSAS
- SSIS
- Terraform
- Test Automation
- Vault