Senior Data Engineer

Responsibilities

• Participate in understanding business needs to ensure value delivery
• Design architecture and evaluate available technologies to solve problems
• Implement data pipelines for data democratization within a Data Mesh architecture
• Develop and deliver reusable components such as connectors, APIs, SDKs, and CLIs
• Test and automate processes wherever possible
• Validate and adapt components based on user feedback
• Collaborate on product quality and technological innovation

Requirements

• Large-scale data processing knowledge (Apache Spark, Hadoop, Hive, etc.)
• Experience with AWS Data & Analytics services (Glue, EMR, Athena, Quicksight)
• Proficiency in Python
• Mastery of AWS Cloud services (S3, Lambda, SQS, DynamoDB, API Gateway)
• Practical experience with relational (SQL) and NoSQL databases
• Proficiency in DevOps practices and Infrastructure as Code (Terraform, CodePipeline, etc.)

Preferred Qualifications

• Previous experience with Data Mesh initiatives
• Experience with Generative AI or automation tools applied to data pipelines
• Knowledge of Observability tools (Cloudwatch, Prometheus, Grafana, etc.)
• Experience developing solutions for financial institutions

About the Company

Zup Innovation uses technology to transform lives, empower talent, and position underprivileged regions as high-quality technology hubs.