Data Engineering Team Lead
Summary
Leads a team of data engineers to build and maintain reliable, high-velocity data pipelines on GCP/BigQuery, ensuring data quality and governance for trading, AI/ML, and compliance systems.
We're not hiring a team lead to manage Jira tickets and run standups. We're hiring someone to build the environment where data engineers ship reliable pipelines fast — and keep getting faster.
Why This Matters
Deriv's mission is Trading for Anyone, Anywhere, Anytime. Millions of traders, multiple regulatory regimes, around the clock. Every trade, every compliance check, every AI model depends on data arriving on time, in the right shape, with full traceability.
Data engineering at Deriv isn't a support function. It's the foundation that product analytics, compliance, trading systems, and AI/ML all stand on. When a pipeline breaks, traders get bad data. When governance fails, regulators ask questions. The stakes are real, and so is the ownership.
Why Deriv
We're in production, not planning.
- Dozens of fraud detection models running continuously on data infrastructure we built
- AI resolving 65%+ of customer enquiries — powered by pipelines that feed the right data at the right time
- Finance automation processing real transactions, not spreadsheet exports
- 400+ internal users on our workflow orchestration platform
We use AI to build, not just to talk about building. AI coding assistants are part of the daily workflow. Automation isn't aspirational — it's how the team already operates. You'll raise that bar, not introduce the concept.
We share openly. Deriv is where we write about what we're shipping, what breaks, and what we figured out the hard way.
What You’ll Do
This is a management role. You'll lead a data engineering team and own their delivery, growth, and operating standard. The Tech Lead owns technical direction; you own the environment in which great engineering happens.
Own delivery velocity as a real metric
- Track cycle time, automation coverage, and governance incident reduction — not just sprint burndown
- Set OKRs tied to pipeline reliability, delivery throughput, and data quality outcomes
- Remove blockers before your team has to escalate them: process, tooling, dependencies, ambiguity
- Own the reliability of data flowing to internal systems, external platforms, and AI/ML workloads
Build a team that operates with autonomy
- Hire engineers who identify problems and act on them without waiting for assignment
- Coach through clear expectations and timely feedback. Handle performance gaps directly — not three quarters late
- Create the conditions where strong engineers grow into technical leaders
Make governance an engineering practice
- Own data quality frameworks, lineage tracking, anomaly detection, and SLA management at team level
- Ensure pipelines are reliable and secure by design, not by heroic intervention
- Turn data contracts, SLAs, and SLOs into things the team builds and monitors — not things they promise in meetings
Drive AI and automation adoption
- Set standards for AI coding assistant usage across the team. Measure the productivity shift, not just the adoption
- Automate what shouldn't need human attention: scheduling, quality checks, deployment, alerting
- Improve output through better engineering leverage, not more hours
Translate business needs into delivery
- Work with product, finance, compliance, and leadership to turn requirements into an executable roadmap
- Communicate progress, risks, and trade-offs with honesty. No surprises
- Partner with the Tech Lead to keep technical direction and delivery priorities aligned
Who You Are
You've led data engineering teams — not just worked in them
- 10+ years in data engineering, with 4+ years in an engineering leadership role. You've managed delivery velocity over multiple quarters. You know the difference between a team that ships and a team that's busy.
You know cloud data stacks from the inside
- GCP, BigQuery, Airflow, Python — or the equivalent. You've worked with dbt or Dataform, built pipelines using Kafka or Pub/Sub, and applied data modelling techniques like Kimball or Data Vault. You understand these tools well enough to hold a high technical bar without owning every decision.
You've built governance frameworks, not just worked within them
- Data quality, lineage, anomaly detection, SLA management — you've owned these as engineering deliverables. You've set and enforced CI/CD standards for data pipelines: version control, testing, automated deployment. You've used pipeline observability tooling to catch problems before stakeholders do.
You drive AI tooling into daily practice
- You've embedded AI coding assistants into a team's workflow and measured the productivity outcomes. This isn't a side interest — it's part of how you think about engineering leverage.
You communicate without hedging
- Delivery progress, risks, trade-offs — you share these clearly with technical and non-technical stakeholders across product, finance, compliance, and leadership. You coach and develop engineers across levels, not just manage them.
You balance competing forces
- Delivery speed, reliability, cost, governance, and team capacity — you've navigated all of these at once and made the trade-offs stick.
Tech Stack
- Cloud: GCP, BigQuery
- Orchestration: Airflow (or equivalent)
- Transformation: dbt, Dataform
- Streaming: Kafka, Pub/Sub
- Languages: Python, SQL
- CI/CD: Version control, automated testing, deployment pipelines for data workflows
- Observability: Lineage tracking, alerting, anomaly detection tooling
- Good to have: Exposure to low-code integration tools like Fivetran or RudderStack. Background in financial services, fintech, or regulated environments.
The Honest Reality
This is demanding work. You'll lead through ambiguity — balancing delivery speed against reliability, governance against velocity, team growth against immediate output. You'll manage expectations from stakeholders who all believe their pipeline is the priority. You'll make trade-offs that not everyone agrees with and stand behind them.
Some weeks you'll spend more time unblocking than building. That's the job.
But you'll own a team's delivery and growth in a company where data engineering is critical infrastructure, not a cost centre. You'll ship governance and quality frameworks that actually protect real trading systems. And you'll build the kind of team you'd want to join.
If you want a role where someone else sets the pace, this isn't it. If you want to set the standard your team operates by, it might be.