Senior Data Engineer
Summary
Senior Data Engineer designing and building scalable, cloud-native data platforms with event-driven ingestion pipelines, primarily using Spark/PySpark, Databricks/Snowflake, and a major cloud provider (Azure preferred).
- Design and build scalable, cloud-native data platforms from greenfield to production
- Implement near-real-time ingestion pipelines using event-driven patterns
- Define and enforce platform standards, including Data Lake / Lakehouse principles, medallion architecture, and data contracts
- Refactor and optimise existing Spark and PySpark scripts for performance and maintainability
- Introduce best practices for code quality, testing, and CI/CD across data pipelines
- Drive adoption of AI tooling and agentic workflows within the data engineering team
- Ensure data quality, observability, and reliability across all pipelines and platforms
- Develop self-service tooling and microservices to simplify platform usage for other teams
- 5+ years of professional experience in Data Engineering
- Strong Python and SQL development skills for pipeline development and optimisation
- Proficiency in Apache Spark / PySpark, including query optimisation and performance tuning
- Hands-on experience with Databricks (preferred) or Snowflake
- Experience with at least one major cloud provider: Azure (preferred), AWS, or GCP
- Experience with stream processing technologies (Kafka, Spark Structured Streaming)
- Solid understanding of ETL/ELT patterns, data modelling (dimensional, Data Vault), and data warehousing
- Experience with orchestration tools (Apache Airflow, Azure Data Factory, or equivalent)
- Knowledge of Infrastructure as Code (Terraform or equivalent)
- Understanding of production-grade system requirements: reliability, scalability, observability, and performance
- Upper-Intermediate English level
- WILL BE A PLUS
- Familiarity with RAG pipeline design and LLM integration patterns
- Knowledge of data governance frameworks and tools (Unity Catalog, Apache Atlas, or similar)
- Experience with dbt for data transformation and modelling
- Familiarity with MLflow, Feature Stores, or ML platform integration
PERSONAL PROFILE
- Self-driven and proactive in identifying improvements
- Comfortable working in a fast-paced, innovative environment
- Strong problem-solving mindset with attention to detail
- Open to experimenting with emerging technologies and approaches