Senior Data Engineer
Summary
The Senior Data Engineer will build and maintain robust ETL/ELT pipelines to process data from payment networks and internal systems using Python, SQL, and orchestration tools like Airflow. The role involves collaborating with stakeholders to ensure data accuracy, security, and performance optimization.
Key Responsibilities:
- Data Pipeline Development: Build and maintain robust ETL/ELT pipelines using Python, SQL, and cloudnative tools to ingest and transform data from 40+ payment networks and internal systems.
- Data Integration: Join and harmonize data across diverse sources including transactional databases, accounting systems, and CRM platforms.
- Collaboration & Governance: Work closely with developers, analysts, and business stakeholders to ensure data accuracy, security, and accessibility.
- Monitoring & Optimization: Implement data quality checks, performance tuning, and monitoring for pipelines handling tens of millions of rows monthly.
- Documentation & Standards: Maintain clear documentation and follow best practices for data architecture, version control, and deployment.
Job Experience and Skills Required:
- Degree in a quantitative field (e.g. Information Systems, Computer Science, Mathematics, Statistics, or Finance).
- 8+ years relevant experience in data engineering or related roles.
- Proven experience in python-based data engineering projects.
- Strong proficiency in SQL and Python for data transformation and automation.
- Experience with data orchestration tools (e.g., Airflow).