Data Engineer
Description
Mission
Execute massive-scale asset rationalization, pipeline migrations, and programmatic metadata implementations using enterprise query, visualization, and telemetry tools to streamline enterprise data operations.
Core Responsibilities
Asset Rationalization: Execute the migration, consolidation, or retirement of hundreds of legacy enterprise dashboards (Tableau, Power BI) and data pipelines based on strict usage telemetry.
Programmatic Definitions: Write code that pushes metric definitions, dimensions, and owner mapping programmatically into central semantic catalogs via APIs.
Data Health Enforcement: Monitor real-time observability dashboards and resolve automated escalations regarding pipeline freshness lags or volume drops within strict SLAs.
Tech Stack & Technical Depth
Core Technologies
Query & Databases: Hive, Advanced SQL, distributed datasets (Hive / Presto / Trino).
Programming & Orchestration: Python (for data manipulation), Airflow (or equivalent orchestration tools).
Visualization & Telemetry: Tableau / Enterprise Visualization tools, telemetry logging.
Requirements
Technical Depth
Proven hands-on experience querying massive distributed datasets.
Strong capability to trace complex data lineage manually through code.
Good to Have / Bonus Skills
Experience with Databricks workflows and PySpark optimization.
Practical experience integrating automated testing within dbt (data build tool).
Familiarity utilizing AI coding assistants for rapid SQL script generation and optimization.