Senior Data Engineer - Microsoft Fabric
Summary
Senior Data Engineer designing, building and optimising scalable data solutions on Microsoft Fabric: ingestion, transformation and integration pipelines, OneLake/Lakehouse/Warehouse, and Power BI semantic models. Writes production-quality code in C# and Python, plus shell scripts, SQL and PySpark for data processing.
WearelookingforanexperiencedSeniorDataEngineerwithstronghands-onexperienceinMicrosoftFabric,dataengineeringandsoftwaredevelopment.ThesuccessfulcandidatemustbecomfortablewritingcodeandhavepracticalprogrammingexperienceusingC#andPython.
Thecandidatemusthaveastrongunderstandingofmoderndataengineeringprinciplesandbeabletodesign,develop,optimiseandsupportscalabledatasolutionsusingMicrosoftFabric.
Responsibilities
- Design,developandmaintainmoderndataengineeringsolutionsusingMicrosoftFabric.
- Developandmaintaindataingestion,transformationandintegrationpipelines.
- WorkwithOneLake,LakehouseandFabricWarehouse.
- DevelopandoptimiseDataPipelinesandDataflowsGen2.
- Writeandmaintainproduction-qualitycodeusingPythonandC#.
- DevelopandmaintainShellscriptstoautomatedataprocessing,filetransfers,jobschedulingandoperationaltasks.
- UseSQLand,whereapplicable,PySpark/Sparkfordataprocessingandtransformation.
- DevelopdatamodelsandsupportPowerBIandsemanticmodels.
- Monitor,troubleshootandoptimisedatapipelinesanddataworkloads.
- Investigateandresolvedataquality,performanceandintegrationissues.
- Ensuredatasolutionsarereliable,scalable,secureandfitforpurpose.
- Translatebusinessrequirementsintotechnicaldatasolutions.
- CollaboratewithDataEngineers,BIDevelopers,Architectsandbusinessstakeholders.
- Supporttheimplementationofbestpracticesacrossdataengineering,developmentanddeployment.
Requirements
- 5+years'experienceinDataEngineeringorarelatedtechnicaldatarole.
- 3+years'stronghands-onexperiencewithMicrosoftFabric.
- ValidMicrosoftCertified:FabricDataEngineerAssociate(DP-700)certification–Essential.
- Strongprogrammingandsoftwaredevelopmentexperience.
- Provenhands-onexperiencewritingcodeusingC#andPython.
- Hands-onexperiencewithShellScripting.
- StrongSQLskills.
- ExperiencewithOneLake,LakehouseandFabricWarehouse.
- ExperiencedevelopingFabricDataPipelinesandDataflowsGen2.
- StrongunderstandingofETL/ELT,datamodellinganddatawarehousing.
- ExperiencewithPowerBIandsemanticmodels.
- ExperiencewithAzure/cloud-baseddataplatforms.
- Strongtroubleshootingandproblem-solvingskills.
- Abilitytoworkindependentlyandengageeffectivelywithtechnicalandbusinessstakeholders.
Certification–Essential
- MicrosoftCertified:FabricDataEngineerAssociate(DP-700)–Validcertificationrequired.
Advantageous
- AzureDataEngineeringexperience.
- PySpark/Spark.
- AzureDataFactory,ADLSorSynapse.
- CI/CDandDevOpsexperience.
- Datagovernance,securityanddataquality.
- ExperienceworkinginAgileenvironments.