Data Engineer
Summary
Data Engineer building and maintaining scalable data pipelines and ETL processes using SQL, MongoDB, Python, Kafka, Spark, and dbt for a financial/insurance/banking client in Jakarta, full on-site.
MSBU which stands for Managed Service Business Unit was founded in 2019. We deliver IT Talent Solutions for the Future Scalable recruiting and managed solutions. Our competitive advantages of SLA, client-success delivery, and community-based recruitment model are proven through our exemplary track record in filling the most challenging IT positions for startup and enterprise clients alike.
Requirement:
- Bachelor's degree in Computer Science or related fields, or equivalent professional experience
- Minimum 3 years experience as Data Engineer
- Have at least 3 years hands on experience with SQL
- Proven to be skilled at ETL - SSIS / Talend / Kettle / Pentaho Debezium, Kafka, Flink, Spark Postgresql, Oracle Database, SQL, NoSQL, Gsql dan TG, Python, MongoDB (MongoDB is a must)
- Have knowledge of ETL, CDC, Event Based Streaming, GraphDB, REST API, Phyton, Basic Programming
- Good in English proficiency
- Proficiency in dbt for data transformation and modelling
- Understanding CI/CD Concept, practice and git-based version control and pipelines
- Understanding of EKS or Containerised environment is a plus
- Preferably available to join ASAP (As soos as possible)
- OPEN TO CONTRACT EMPLOYMENT
- OPEN TO WORK IN FINANCIAL/INSURANCE/BANKING INDUSTRY
Responsibilities:
- Design, develop, and maintain scalable and efficient data pipeline for ingestion, processing, and storage of structured and unstructured data
- Implement ETL processes to ensure data quality and integrity
- Develop and optimize database structure for performance and scalability
- Implement indexing, partitioning, and other optimization techniques for efficient data storage and retrieval
- Design and implement solutions for procession and analyzing large volumes of data
- Collaborate with data modelers to translate data requirements into effective data structures
- Ensure alignment with data modeling standards and best practices
- Implement monitoring and alerting systems to ensure the reliability of data pipelines