Data Engineer

Key Responsibilities and Accountabilities:

• Based on demand build ETL architecture transferring structured and non-structured data into a Hitachi Content Platform (HCP) Object store as the back end for archiving
• Build bespoke archives of data to satisfy point business solutions such as regulation and compliance
• Build a generic framework to ingest decommissioned application data into searchable archives such that applications may present their data from e.g. a database in a pre-determined format to be ingested in a searchable form
• Deliver a standard interface for searching such archives securely
• Anticipate demand and innovate existing solutions
• Code, test, document, and maintain standardised, automated data pipelines
• Help design and build the infrastructure required for optimal extraction, transformation, and loading of data from a variety of data sources
• Build analytics tools that utilise the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics
• Responsible for testing, documentation and handover to the customer.
• Delivery to meet aggressive timescales.
• Communication of progress and risk to the team

Business Knowledge/ Technical Skills:

• Experience building and optimizing big data data pipelines, architectures and data sets.
• Strong analytic skills related to working with both structured and non-structured datasets.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, combining, processing and extracting value from large disconnected datasets.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Knowledge of SIARD (Software Independent Archiving of Relational Databases)
• We are looking for a candidate with experience or a strong aptitude in a Data Engineering role, and ideally have experience using the following software/tools:

Must have

• In-depth experience of developing Python or PowerShell scripts across Windows and Linux platforms
• Experience of UI development, ideally with React
• Experience with creating playbooks in Ansible Automation Platform
• Working knowledge and experience working with MSSQL and Oracle databases
• Knowledge of Hitachi Content Platform
• Knowledge of Solr

Personal Attributes:

• Ability to influence effectively at different organizational levels;
• Courage and determination to challenge current practices;
• Strong analytical and conceptual skills;
• Clear written and verbal communication skills in English.
• Ability to deal with people, across a complex organisation with different cultures and geographies
• Demonstrates professional acumen, integrity, discretion, credibility and trustworthiness
• Self-motivated, with the ability to work effectively under pressure and results driven
• Ability to respond swiftly to changing demands of the business
• Strong interpersonal and client relationship/influencing skills