Senior Data Engineer

*This is a hybrid role with an on-site requirement in Wilmington, MA; candidates must reside within commuting distance to the greater Boston area*

We are seeking a highly skilled and experienced Senior Data Engineer specializing in Microsoft SQL Server, Azure data services, and Microsoft Fabric. The ideal candidate will design, develop, and maintain enterprise data warehouses, lakehouses, and data pipelines, ensuring data reliability, performance, quality, and security. The Senior Data Engineer will collaborate closely with software developers, data analysts, and other stakeholders to deliver trusted data for reporting, analytics, and business processes.

Database Design and Development\:

• Design, implement, and maintain database structures and dimensional data models, including tables, views, indexes, and schemas using Microsoft SQL Server, Azure SQL, and Microsoft Fabric Warehouse and Lakehouse.
• Develop complex SQL queries, stored procedures, functions, and triggers to support application development and data processing.
• Optimize database performance through indexing, query optimization, and other performance tuning techniques.

Data Integration and ETL Processes\:

• Design and develop ETL and ELT processes using SQL Server Integration Services (SSIS), Azure Data Factory, and Microsoft Fabric Data Factory pipelines and notebooks to ingest and transform data from multiple sources.
• Ensure data quality, consistency, and integrity across different systems and environments.
• Monitor and troubleshoot data pipelines, implement logging and error handling, and ensure timely and accurate data processing.

Query Optimization\:

• Analyze and optimize SQL queries to improve execution times and reduce resource consumption.
• Implement indexing strategies to speed up data retrieval operations.
• Rewrite complex queries to make them more efficient and easier to maintain.
• Collaborate with developers to optimize queries embedded within applications.

Azure and Microsoft Fabric Integration\:

• Design and maintain cloud data solutions using Azure SQL Database, Azure Data Lake Storage, and Microsoft Fabric OneLake, Lakehouse, and Warehouse.
• Develop and maintain data integration solutions connecting SQL Server, Azure data services, and Microsoft Fabric, including incremental loads and bronze, silver, and gold data layers.
• Apply best practices for Azure and Microsoft Fabric, including workspace organization, access controls, monitoring, and performance and cost optimization.

Collaboration and Support\:

• Work closely with software developers, data analysts, and other IT team members to understand requirements and provide database solutions.
• Provide technical support and troubleshooting for database-related issues.
• Mentor and guide junior data engineers and other team members.

Documentation and Compliance\:

• Create and maintain comprehensive documentation for database designs, processes, and procedures.
• Ensure compliance with data governance, security policies, and industry regulations.