[Perm] Data Engineer (Azure DevOps)
Posted 4
views
Summary
Permanent Data Engineer role in Singapore focused on building and optimising cloud-based data pipelines, data models, and governance on an Azure data platform with DevOps/CI-CD automation. Requires 10+ years of data engineering experience with strong Python and SQL skills.
Data Management & Transformation
- Design, develop, and maintain scalable data pipelines integrating data from a variety of business applications, databases, and external sources.
- Implement data quality controls and validation processes to ensure data accuracy, consistency, completeness, and reliability.
- Perform data transformation, cleansing, aggregation, filtering, and enrichment to support business and analytical requirements.
- Establish and maintain appropriate governance practices for data, analytical models, and automated decision‑making processes.
- Develop and maintain logical and physical data models, including data structures, relationships, and business rules.
- Ensure data platforms and solutions adhere to relevant security, governance, and compliance standards.
- Develop and maintain automated deployment and integration processes for data solutions and cloud-based infrastructure.
- Optimise data pipelines, processing workflows, and data models for performance, scalability, reliability, and cost efficiency.
- Monitor and optimise data processing and storage capacity, including capacity planning and forecasting.
- Identify and resolve system and query performance issues through appropriate tuning, optimisation, caching, indexing, and other techniques.
- Monitor platform utilisation and implement appropriate housekeeping and cost‑management practices.
- Support the deployment, integration, and ongoing operation of advanced analytics and AI/ML solutions within the data platform.
- Continuously assess and improve the reliability, maintainability, and operational efficiency of the data environment.
- Work closely with data leads, technology teams, and business stakeholders to establish and manage data as a strategic business asset.
- Translate business requirements into scalable and sustainable data solutions.
- Provide guidance to business users on the development and maintenance of reports, dashboards, and analytical solutions.
- Promote good data management, governance, and engineering practices across relevant stakeholder groups.
- Bachelor’s degree in Computer Science, Data Science, Software Engineering, Information Systems, or another relevant quantitative or technical discipline. A Master’s degree is advantageous.
- At least 10 years of relevant experience in data engineering, data management, data integration, data modelling, data quality, data platform optimisation, or related disciplines.
- Strong experience designing and implementing modern cloud‑based data architectures and data platforms.
- Strong programming skills in Python and SQL. Experience with distributed data processing frameworks is highly desirable.
- Experience working with both relational and non‑relational databases. Familiarity with enterprise or legacy database technologies is advantageous.
- Experience with software engineering and DevOps practices, including source control, CI/CD, automated deployment, and workflow orchestration.
- Experience with data governance, metadata management, data cataloguing, and access‑control frameworks.
- Familiarity with business intelligence and data visualisation tools is advantageous.
- Experience supporting the deployment and operationalisation of AI/ML solutions, including areas such as feature engineering and production inference, is desirable.
- Strong understanding of data security, governance, scalability, performance, and operational best practices.
- Strong analytical, problem‑solving, communication, and stakeholder management skills.