DataOps/Cloud Data Engineer
Summary
Design and maintain cloud data pipelines, data models, and warehouse solutions using Azure SQL, Synapse, and Oracle, with a focus on CI/CD and performance tuning.
- Participate in the review of project, work request and maintenance release artefacts - Use Case, System Requirement Specification, etc.
- Deliver detailed system design documents and any other supporting documentation conforming to Ministry standards.
- Design, develop and optimize data/database CI/CD pipelines.
- Design logical and physical data models using Sybase Power Designer.
- Prepare database change in DACPAC and develop Azure CI/CD release pipeline to promote DACPAC.
- Promote database changes through Development, Integration Testing, QA, UAT, Production environments.
- Provision data in cloud environment according to requirements.
- Design, develop data loading, data conversion, data migration program.
- Design, develop and implement data conversation of VLD (Very large Data) of Online analytical processing (OLAP) and online transaction processing (OLTP) environments to Cloud Software-as-a-service (SaaS), Platform-as-a-service (PaaS) and Infrastructure-as-a-service (IaaS) environments.
- Design, develop and implement fact/dimension model, data mapping, data warehouse, data lake and data lakehouse.
- Plan, configure, maintain, administrate, tune performance of Oracle, Microsoft Azure SQL database and Synapse database.
- Support cloud-based heterogeneous data replication solution.
- Develop complex Oracle PL/SQL program, SQL server stored procedure to fulfill project, work request, maintenance requirements.
- Optimize performance of PL/SQL program, SQL server stored procedure.
- Monitor and analyze incident, provide timely resolution.
- Perform reference data administration as required.
- Build and test database backup/recovery scripts.
- Build and test database automation scripts.
- Perform database administration, including patching/upgrade, monitoring, performance tuning, partitioning, capacity planning, OEM upgrade/enhancement, achieving, etc
- Document and resolve any database problems/issues.
- Provide effective knowledge transfer to Ministry staff at all stages of this assignment.