Data Engineer
TE-1521- Data Engineer
Location: Hyderabad, India
Department: Software Development
Experience: 8-12
- Warehouse architecture and modelling — BigQuery (or equivalent) as the single source of truth, designed to serve both analytics and activation use cases.
- dbt project ownership — model layering (staging → intermediate → marts), naming conventions, tests, documentation, exposures, and the promotion workflow from dev to production.
- Ingestion pipelines — event streams from the CDP (Segment or equivalent) landing cleanly in the warehouse, plus structured loads from the product database, billing / payments, ad platforms and third-party SaaS systems.
- Identity resolution in the warehouse — stitching anonymous, edge-level and identified user records into one canonical key, and publishing that key to downstream consumers.
- Core marts — revenue, orders, subscriptions, cohorts, LTV, funnel conversion, channel performance, clinical or operational metrics as the business requires.
- Reverse-ETL / activation models — warehouse tables purpose-built to feed ad-platform conversions APIs, lifecycle marketing, audiences and experimentation tools.
- Data quality and observability — dbt tests, freshness monitors, row-count alerts, reconciliation checks between systems (for example: warehouse orders vs payment processor, warehouse conversions vs ad-platform reports).
- Data governance — PII / PHI handling, column-level masking, row-level security where required, access policies, and the documentation that makes those policies auditable.
- Orchestration and CI/CD — scheduled runs, retries, incremental strategies, pull-request-based dbt reviews, automated tests in CI.
- Partnership with external data partners and the sister MarTech Engineering function — clear ownership lines, shared conventions, joint on-call for pipeline incidents.
- 8–12 years in data engineering or analytics engineering at consumer-facing digital businesses — ideally including at least one direct-to-consumer, e-commerce, subscription or marketplace product.
- Deep, hands-on experience designing and maintaining a warehouse on BigQuery, Snowflake or Redshift. You can explain partitioning, clustering, materialised views, and cost / performance trade-offs from memory.
- Production experience with dbt — Core or Cloud — including model layering, incremental strategies, snapshots, tests, documentation, exposures, and managing a multi-contributor repo through pull requests.
- Fluent SQL — advanced window functions, CTEs, pivoting, session stitching, slowly-changing-dimension patterns. You can read a 200-line query and tell us what is wrong with it.
- Experience designing identity resolution in the warehouse — merging anonymous, cookie-level, logged-in and server-generated identifiers into one canonical customer key, and publishing that key for downstream systems to join on.
- Hands-on experience building marts for revenue, orders, subscriptions, cohorts, LTV and funnel analysis — and reconciling those marts against authoritative systems (payment processors, billing, ad platforms).
- Python for data work — transformations, orchestration DAGs, small internal tools, and scripting around the warehouse. You do not need to be a backend engineer.
- Working knowledge of at least one workflow orchestrator — Airflow, Prefect, Dagster, or equivalent — and the operational discipline that goes with it (idempotency, backfills, retries, alerting).
- Comfort integrating warehouse output into reverse-ETL tooling (Polytomic, Hightouch or Census) so that audiences, conversion events and customer traits can be activated in ad platforms, lifecycle tools and experimentation systems.
- A bias for data contracts and tests over heroics — you treat a broken dashboard as a platform failure to be prevented, not a one-off to be patched.
- Experience operating under ambiguity — a platform migration, re-platforming, or a major tracking overhaul where the "right" schema had to be invented mid-flight.
| Category | Tools |
| Data Warehouse | BigQuery (required), Snowflake or Redshift. Strong grasp of partitioning, clustering, cost controls. |
| Transformation | dbt Core or dbt Cloud (required), dbt tests, snapshots, exposures, packages. |
| Orchestration | Airflow, Prefect, Dagster, or an equivalent DAG runner. dbt Cloud scheduling for dbt-only pipelines. |
| Ingestion | Segment (CDP → warehouse), Fivetran or Airbyte (SaaS → warehouse), custom Python ingestion where needed. |
| Reverse-ETL / Activation | Polytomic (preferred), Hightouch, Census — mapping marts to ad-platform, lifecycle and experimentation destinations. |
| Product Analytics (consumer) | Mixpanel, Amplitude, or Heap — you will not own these tools, but you will land their events and feed their user properties from the warehouse. |
| BI / Visualisation | Omni, Looker, Mode, Metabase, Tableau — warehouse-native semantic modelling preferred. |
| Languages | SQL (advanced), Python (fluent), Jinja (dbt); shell scripting as needed. |
| Storage & Cloud | Google Cloud Platform primary (Cloud Storage, IAM, Pub/Sub, Cloud Run / Cloud Functions). |
| Governance & PII | Column-level security, masking policies, row-level access, data-classification tagging, audit logging. |
| Dev workflow | Git, GitHub / GitLab, pull-request reviews, dbt CI, unit and schema tests, dbt docs. |
| Collaboration | Linear, Slack, Google Docs & Sheets, Confluence or Notion. |
- Healthcare, fintech, or other privacy-sensitive domain experience — designing a warehouse that respects PII / PHI boundaries and survives audit.
- Experience operating a dbt repo shared between an internal team and an external data partner (or vendor) — conventions, code review, ownership boundaries.
- Experience with Segment Unify, identity-graph services, or custom identity-resolution frameworks.
- Hands-on with experimentation data — building the exposures, assignment and outcome tables that power A/B test readouts and feeding them to an experimentation platform.
- Hands-on with LTV or propensity modelling (regression, survival or tree-based), and with getting those predictions into the warehouse as first-class tables.
- Familiarity with streaming patterns — Pub/Sub, Kafka, Kinesis — and when to prefer them over batch.
- Prior early-hire data platform experience at a growth-stage product company, where you set the conventions rather than inherited them.
- You treat the warehouse as a product — versioned, documented, tested, with clear ownership of every model and every downstream exposure.
- You are as comfortable debugging a BigQuery cost regression as you are explaining an attribution or LTV model to senior leadership.
- You write dbt tests before you write dashboards. You would rather catch the problem in CI than in a Monday standup.
- You draw scope lines honestly — you know where your layer ends and the MarTech, product or ML layer begins, and you document the hand-off.
- You reconcile aggressively — warehouse vs payment processor, warehouse vs ad platforms, warehouse vs the product UI. Anything that does not tie out is a ticket, not a footnote.
- You write things down. The team should not need to re-learn a modelling decision you have already made.
- Location: On-site at our Hyderabad (Softobiz) office. This is not a remote role.
- Reporting: Reports to the Engineering Owner, with regular working interactions across product, analytics, growth and external data partners.
- Compensation: Competitive base salary plus performance bonus and standard benefits.
- Schedule: Full-time, with some overlap expected with partner teams in other time zones.
Skills
- AI
- Airbyte
- Airflow
- Amplitude
- Analytics
- API
- Bash
- BigQuery
- CI/CD
- Cloud
- Cloud Run
- Confluence
- Dagster
- Data Engineering
- Data Governance
- Data Modeling
- Data Quality
- Data Warehousing
- dbt
- E-commerce
- ETL
- Fintech
- Fivetran
- GCP
- Git
- GitHub
- GitLab
- IAM
- Kafka
- Kinesis
- Lifecycle Marketing
- Looker
- Machine Learning
- Metabase
- Mixpanel
- Notion
- Observability
- Prefect
- Python
- Redshift
- SaaS
- Slack
- Snowflake
- SQL
- Tableau