Data Lakehouse Engineer
Data Lakehouse Engineer at N Consulting Ltd
Location: Warsaw (Poland)
Job Summary:
We are seeking a skilled AWS Data Lake Engineer to design, build, and maintain scalable data lake solutions on AWS. The ideal candidate will have strong experience in cloud-based data engineering, big data processing, and modern data architecture.
Key Responsibilities:
- Design and develop data lake solutions using AWS services such as IAM, Lambda, EKS, S3, EMR, and MWAA
- Build and optimize data pipelines using Python and PySpark
- Implement and manage Apache Iceberg tables for efficient data storage and querying
- Develop and maintain CI/CD pipelines and infrastructure using Terraform
- Ensure data quality, security, and governance across the platform
- Orchestrate workflows and automate data processing tasks
Required Skills:
- Strong experience with AWS data services and cloud architecture
- Proficiency in Python and PySpark
- Hands-on experience with Apache Iceberg
- Experience with CI/CD tools and Infrastructure as Code (Terraform)
- Knowledge of data quality, data security, and workflow orchestration