Data Engineer – Oil & Gas
Summary
Senior data engineer building batch and streaming pipelines that move well, production and sensor data into a cloud lakehouse for an oil & gas upstream operator, using Spark, Kafka, Airflow and dbt on Databricks or Snowflake (Azure/AWS). Remote full-time contractor role open to LATAM, working Houston business hours.
Client: oil and gas company (upstream operator, Houston HQ) · Location: remote, LATAM, Houston business hours · Level: senior · Engagement: full-time contractor
You will build the pipelines that move well, production and sensor data from the field into a cloud lakehouse used by engineers and analysts.
What you will do
Ingest time-series data from SCADA and historians (OSIsoft PI, OPC UA) alongside ERP and production accounting sources
Build batch and streaming pipelines on Databricks or Snowflake with Spark, Kafka and Airflow
Model data in a medallion architecture for production, drilling and maintenance analytics
Enforce data quality, lineage and access controls
Work with reservoir and production engineers to turn their questions into datasets
Optimize pipeline cost and performance
What you bring
5+ years in data engineering
Strong SQL and Python, plus Spark in production
Experience on Azure or AWS data stacks
Orchestration with Airflow or similar, and dbt for transformations
Experience with high-volume time-series or IoT data
Advanced English
Nice to have
Oil and gas domain knowledge (WITSML, PPDM, OSDU)
PI System integration experience
Spanish or Portuguese for work with LATAM operations