Data Engineer
Summary
Builds and maintains data pipelines, warehouses, and ETL processes to turn raw data into clean, accessible datasets for analysts and scientists, using SQL, Python, and cloud platforms like GCP/AWS.
Data Engineer is responsible for supporting the process of framework construct or build systems that collect, manage, and convert raw data into usable information for data scientists to interpret. Their ultimate goal is to make data accessible so that organizations can use it to evaluate and optimize their performance.
1.Data warehousing, make sure the data kept and ready for use
Create data models and speak to the tradeoffs of different modeling approaches
Build and maintain the data pipelines, databases, and storage systems.
Integrate data from various sources, such as databases, APIs, and third-party systems and transform it into a usable format.
Evaluate and implement emerging tools and technologies to enhance data engineering capabilities.
2.Data quality and governance
Implement data cleansing, validation, and data governance practices.
Implement compliance with data governance and security policies.
Explore ways to enhance data quality and reliability.
Ensure users or data scientists can connect to the warehouse, access the needed data in the convenient format, and start getting valuable insights from it.
POSITION REQUIREMENTS
Min. Diploma Degree (D3) in any major with relevant certification in Data Science/Engineering will also be considered
Experience
2-3 years experience in relevant roles
Experienced in a startup, e-commerce, technology or similar environment will be better
Experience in Data warehousing or Big Data Management
Portfolio of previous work and projects
Technical Competency
Basic knowledge and skills in developing data pipelines, data processing systems, and ETL processes
Basic knowledge of data modeling and data architecture design principles
Familiarity with data security and compliance standard
Expertise in database technologies such as SQL and NoSQL database
Familiarity with cloud-based data platforms such as GCP (BigQuery, Notebook, Google Storage, Google Environment), AWS Environment.
Having basic programming knowledge and skill, such as Python & SQL is a must. Golang, Java, Scala are a plus.
Strong analytical mind and good business sense
Which of the following types of qualifications do you have?
How many years' experience do you have as a Data Engineer?
How many years' experience do you have using SQL queries?
Information Technology Services 101-1,000 employees
One-Stop Solution for Bill Payment Services
Alterra Bills provides bill payment services from upstream to downstream that connect all bill issuers, payment channels, and end-users.
One-Stop Solution for Bill Payment Services
Alterra Bills provides bill payment services from upstream to downstream that connect all bill issuers, payment channels, and end-users.