Data Engineer

The Castillian is working with a digital marketing sector client and is currently focused on the following key activities:

• Improving the code base and data model of the data estate.
  

• Assisting with maintenance tasks and daily data operations.
  

• Monitoring the data loading process chain for the Datawarehouse ETL/ELT and conducting analysis when alarms are raised.
  

• In-charge of the migration of Central Backend (Excel sheet data source) to the Central repository (Airtable source).
  

• Involved heavily in the migration of the client’s architecture to Microsoft Fabric while collaborating closely with a Microsoft consultant.
  

• Involved in a new web scripting data analytics project to collect article data by author, including publication dates and update history.
  

• Working on data pipeline enhancements by updating sub-affiliate calculation logic and implementing revenue adjustment tracking through a new audit table to monitor changes over time.
  

• Developing and testing new ETL data for dimension adjustments from multiple pipelines and integrating the data quality pipeline into the affiliate data pipeline.
  

• Supporting production issues across pipelines as part of daily operational checks and carrying out ad hoc data analysis tasks to assist business stakeholders.
  

• Improving knowledge on data and BI estate, as well as business knowledge, continuously through training and self-development