Senior Data Engineer at Pi Techniques Pvt Ltd
Description
Position Overview
We are seeking a visionary Senior Data Engineer to architect and build a next-generation distributed database infrastructure. In this role, you will lead the strategic design and implementation of a highly scalable, multi-database architecture utilizing Microsoft Fabric and Azure SQL. Your primary focus will be designing for future enhancement, building modular database structures that support evolving business needs, and establishing high-performance analytical and transactional capabilities from the ground up.
Core Responsibilities
- Distributed Database Architecture: Design and build a scalable, modular multi-database topology using Azure SQL and Microsoft Fabric optimized for future applications and business expansion.
- Modern Fabric Ecosystem Implementation: Architect a unified data fabric using OneLake, Synapse Data Warehousing, Lakehouses, and Data Factory Gen2 to enable seamless cross-database data integration.
- Future-Proof Data Modeling: Develop highly flexible data structures (relational, star schema, and Delta Lake formats) designed to easily adapt to new features, high concurrency, and massive data growth.
- Data Virtualization & Performance: Leverage Fabric's Shortcuts and DirectLake functionalities to deliver instant, high-performance reporting and cross-database querying without building rigid data silos.
- Automation & Enterprise Governance: Set up automated CI/CD deployment pipelines for all database objects and implement robust, future-ready data security, row-level access controls, and compliance standards.
Required Technical Skills & Experience
- Experience: 5+ years in Data Engineering or Database Architecture, with a strong background in designing distributed cloud database systems.
- Microsoft Fabric Mastery: Hands-on experience with Microsoft Fabric, including SaaS data
warehousing, Lakehouse architectures, Delta-Parquet optimization, and cross-workspace orchestration.
- Azure SQL Expertise: Deep proficiency in Azure SQL Database, advanced T-SQL optimization, distributed queries, and database sharding/partitioning strategies.
- Advanced Data Modeling: Expert-level knowledge of domain-driven design, transactional vs. analytical processing patterns, and building decoupled data layers.
- Orchestration: Experience building complex, modular pipelines using Fabric Data Factory or Azure Data Factory.
