Data Engineer (Python/SQL)
Summary
Builds and maintains scalable data pipelines, ETL processes, and data models using Python, SQL, and big data tools like Spark/Hadoop to ensure data quality and accessibility for business stakeholders.
Company Description
NCS is a leading AI Tech Services company. With a 15,000-strong team across the Asia Pacific, NCS scales its platforms and capabilities to provide clients with greater agility and AI expertise across a range of industries. Embracing a strong ecosystem of global partners, NCS transforms technology services delivery combining AI with digital resilience to drive real business impact. NCS is a subsidiary of the Singtel Group.
Duties and Responsibilities
Design, build, and maintain scalable, secure data pipelines and storage systems; ensure data quality through ETL processes and regular checks.
Implement policies and practices to control, optimize, and secure data assets, ensuring data integrity and accessibility.
Develop and maintain data models, structures, and databases to meet business needs; communicate data architecture effectively.
Develop, test, and maintain scripts and programs to automate data processing and pipelines, adhering to industry standards.
Create and operationalize data visualization solutions to simplify complex data for stakeholders and decision-making.
Work with cross-functional teams to gather data requirements, optimize existing processes, and deliver ad hoc reports and insights
Knowledge
Knowledgeable in the following:
Comprehensive understanding of data manipulation tools such as pandas, dplyr, and Spark.
In-depth knowledge of big data frameworks and tools like Apache Spark and Hadoop.
Familiarity with data warehousing services like AWS Redshift, Snowflake, or similar solutions.
Proficiency in AWS Cloud Services, particularly AWS Glue and AWS Lake Formation.
Familiarity with business intelligence tools such as Tableau, Power BI, QuickSight, or Google Data Studio.
Awareness of data visualization libraries and packages like Dash, Plotly, Matplotlib, ggplot, and Folium.
Understanding of machine learning libraries and tools (e.g., scikit-learn, caret, MATLAB) is a plus.
Knowledge of data governance, quality control, and security best practices.
Skills
Problem-Solving: Ability to address technical challenges and deliver efficient data solutions.
Communication: Clear and concise communication skills for collaborating with team members and stakeholders.
Collaboration: Ability to work effectively within a team environment to achieve shared goals.
Time Management: Capacity to manage tasks and meet deadlines in a structured and timely manner.
Attention to Detail: Careful and accurate handling of data to ensure quality and integrity.
Client-Focus: Ability to understand business needs and align data solutions to support decision-making and strategic objectives.
Adaptability: Flexible and open to learning new tools, technologies, and processes in a rapidly changing environment.