Senior Data Engineer
Summary
LanceSoft Europe is hiring a Senior Data Engineer in the Netherlands to design, build, and optimize complex production-grade data pipelines and manage large-scale distributed batch processing. The core stack includes Python, SQL, PySpark, Apache Spark, Airflow, dbt, Docker, Kubernetes, and cloud data platforms with GCP/BigQuery preferred.
- At least 8 years of relevant experience in data engineering, cloud engineering, data analytics or a closely related technical field.
- Proven hands-on experience designing and building complex production-grade data pipelines using ETL and ETL patterns.
- Strong programming skills in Python and SQL, with practical experience using PySpark for distributed data processing.
- Hands-on experience with Apache Spark and workflow orchestration tools such as Apache Airflow.
- Experience with cloud data platforms, preferably Google Cloud Platform and BigQuery, and working knowledge of Azure or AWS.
- Strong understanding of data modelling, normalization and denormalization, data integration, data lakes and data warehouses.
- Experience with relational and non-relational data stores, including technologies such as MS SQL, Hive and dbt, plus object storage such as Amazon S3.
- Experience with containerization, orchestration and automated delivery tooling, including Docker, Kubernetes and CI/CD pipelines.
- Experience managing and optimizing distributed systems and clusters for large-scale batch processing.
- Good command of software engineering practices, including modular design, testing, code review, documentation, security and production support.