Data Engineer
NewBe an early applicantSummary
Senior data engineer at Quarry Consulting building enterprise data integration and transformation pipelines on Microsoft Fabric and Azure data services. Day-to-day work centers on PySpark, Python, SQL/T-SQL, pipeline tuning, and Git/Azure DevOps deployment automation.
- 7+ years of experience in data engineering and enterprise data integration.
- Strong hands-on experience with Microsoft Fabric and related Azure data services.
- Strong experience with PySpark, SparkSQL, Python, and SQL/T-SQL.
- Experience building ingestion and transformation pipelines from APIs, files, databases, and third-party platforms.
- Experience with curated data layers, data modeling, and analytics-ready datasets.
- Experience tuning data pipelines for scale, resiliency, and cost optimization.
- Working knowledge of security, access controls, and data governance practices.
- Experience with Azure DevOps and Git-based deployment automation.
Preferred Skills
- Experience with learning, HR, or talent data domains.
- Familiarity with ADF, Synapse, Databricks, Azure Functions, or Logic Apps.
- Exposure to AI tools such as GitHub Copilot and Claude to accelerate engineering productivity and solution design.
- Experience with data governance tooling such as Purview.
- Exposure to AI/ML-ready data preparation and feature engineering practices.