Data Engineer
Summary
Build and maintain data pipelines, warehouses, and ETL processes on GCP/BigQuery to support analytics and investment insights for a mutual-fund platform.
Bibit.id is a platform to help you to start your investment in mutual fund in a very simple way.
Job Description
We are looking for a Data Engineer to develop, construct, test and maintain architectures, such as databases and large-scale processing systems. The data engineers will need to recommend and sometimes implement ways to improve data reliability, efficiency, and quality. This position will need to ensure that the architecture that is in place supports the requirements of the data scientists and the stakeholders, the business.
Responsibilities
- Construct and maintain efficient data pipelines and warehouses for data collection
- Develop necessary scripts/mini-apps to run data analysis
- Conduct tests on large scale data platforms
- Provide recommendations to improve data quality
- Ensure and support the data architecture utilized by data scientists and analysts
- Development of data processes for data modeling, mining, and data production
Qualifications
- Working experience of at least 2 years with exposure to Big Data, Real-time data pipelines, ETL processes and supporting tools/framework such as Airflow
- Experience in Cloud Service environment, especially GCP + BigQuery. Skill in other Cloud Services is a big plus
- Experience with third party behaviour analytics tools such as Firebase, Moengage, CleverTap
- Experience in working with RDBMS & Non-RDBMS datastores
- Excellent skills in writing efficient querying language especially SQL
- Experience in utilizing popular Visualization Tools such as Tableau, Holistics, Metabase
- Skilled in Python programming
- Skilled in Pig Latin is a plus
- Understand how to analyze & utilize data in order to provide value/insights for company decision making