FullStack Data Engineer
Summary
Build and maintain a cloud-based big data platform using Python, SQL, and AWS services like Lambda and Glue.
We are looking for a Full Stack Data Engineer to contribute to the development and operation of data-driven solutions on an AWS-based big data platform. The role combines software development, data processing and cloud deployment activities while collaborating on architecture and data flow design.
Main Responsibilities
- Develop end-to-end features for a cloud-based big data platform.
- Participate in architecture discussions and the definition of data processes and flows.
- Create proof-of-concepts to validate proposed solutions.
- Develop, test and deploy solutions across development, quality and production environments.
- Monitor deployed solutions and perform troubleshooting and maintenance activities.
- Document developed functionality using Confluence.
- Support data extraction activities and exploratory data analysis.
- Experience with Python for data manipulation and processing.
- Experience with SQL.
- Experience in software development and debugging.
- Interest in data science.
Additional Valued Experience
- AWS services such as S3, Lambda, Athena, EC2, Glue, CDK/CloudFormation, Step Functions, Batch and Fargate.
- Spark / PySpark.
- Git.
- Jupyter Notebooks.
- Experience developing applications and/or working in production environments.
- Proactive approach and initiative.
- Team collaboration skills.
- Ability to independently identify and solve problems.
- Interest in learning new skills.
Location / Work Model
- Lisbon, Portugal
- Hybrid work model