Big Data Engineer Blockchain
Summary
Build and maintain blockchain data pipelines, aggregate on-chain data into a data lake, and ensure data quality with Python, SQL, and ETL workflows.
You collect, transform and store data from multiple sources in the data lake, aggregate and process data from blockchains, develop reliable live extraction processes, implement data-quality checks and continuously improve data-gathering workflows while resolving data issues.
Responsibilities
- Collect, transform and store data from various sources in the data lake
- Aggregate data from various blockchains
- Develop reliable live data-extraction processes
- Define and implement data-quality checks
- Improve data-gathering processes
- Handle data-related problems
Requirements
- Excellent knowledge of Python
- Experience writing robust scripts
- Experience with RESTful APIs
- Knowledge of database design and SQL
- Extensive experience building ETL pipelines
- Basic knowledge of blockchains considered an advantage
- Spark experience considered an advantage
- Experience with AWS considered an advantage
Benefits
- Competitive salary and bonuses
- Flexible and remote working options
- Learning Hub
- Free assorted healthy snacks and fresh fruits in the office
- Weekly gourmet breakfast meet-ups
- Regular team events and parties