data engineer in Databricks

Описание:

Deloitte provides business consulting and services to clients.

Задачи:

• Design, build, and optimize data pipelines in Databricks;
• Develop, optimize, and operate ETL/ELT pipelines in Databricks using PySpark, SparkSQL, and Scala;
• Run migration programmes to Databricks, including legacy ETL processes into scalable data pipelines;
• Build robust data pipelines on Databricks for real-time and batch processing;
• Configure and manage Databricks clusters and jobs for cost and performance efficiency;
• Apply CI/CD to automate testing and deployment of notebooks, jobs, and pipelines;
• Drive adoption of Databricks Lakehouse architecture and data engineering practices across clients;
• Design and build reliable ETL/ELT pipelines in Databricks;
• Apply medallion architecture and Delta Lake best practices;
• Optimize clusters for performance and cost;
• Embed data quality to deliver reliable data products;
• Work closely with cloud platform engineers to ensure data solutions are secure, scalable, and easy to operate.

Требования:

• Hold a technical Master’s degree, preferably in Data Science, Big Data, Econometrics, Physics, Mathematics, Computer Science, Artificial Intelligence, or a related field;
• Have at least 4 years of work experience in data engineering;
• Have strong hands‑on experience with Databricks and Apache Spark, including PySpark and Spark SQL;
• Have a deep understanding of Delta Lake and medallion architecture;
• Be proficient in SQL and one or more data engineering‑related programming languages, such as Python or Scala;
• Have excellent command of English and Dutch;
• Nice to have: consulting experience, relevant Databricks certifications such as Data Engineer Associate/Professional, Spark Developer Associate, or Data Analyst Associate.

Условия:

• Profit‑sharing in addition to a fixed salary;
• Flexible working hours;
• Part‑time work for 32 hours or full‑time work for 40 hours;
• 26 Vacation days per year for a full‑time employment and the option to purchase additional days;
• The option to exchange 3 national holidays for 3 non‑national holidays;
• Time‑for‑time arrangement;
• The possibility to take at least 2 months of sabbatical leave;
• Mobility scheme with options including a lease car, public transport, cash budget, or a combination;
• Allowance for an ergonomically equipped home office and internet allowance;
• Up to 55 hours of childcare service per calendar year for children aged 12 or younger;
• Laptop and iPhone, with private use of the iPhone permitted;
• Pension scheme with a 2% employee contribution;
• Participation in a collective healthcare cost scheme;
• Tax‑advantaged benefits such as company fitness, bicycle purchase, or bicycle leasing;
• Monthly flexible benefits budget for options such as additional vacation days or bicycle plan financing;
• Six weeks of fully paid birth leave for traditional and rainbow families.