Data Engineer - Level 2
Posted Updated
Description
Digital Room (Philippines), Inc. is seeking a Data Engineer II to own end-to-end data pipelines, from ingestion to publishing and monitoring. You will build batch, incremental, and near real-time pipelines, implement CDC, and optimize storage formats for analytics and AI systems.
You will design the orchestration layer, apply data quality checks, and mentor junior engineers while collaborating with stakeholders to translate business questions into robust data solutions.
Build and maintain batch, incremental, and near-real-time pipelines from various sources to databases, data lakes, and warehouses. Implement change data capture on high-volume source tables; handle schema drift and replays. Write idempotent, restartable jobs to avoid duplicates or data loss. Choose and justify load patterns (full refresh, watermark-based, upserts, SCD). Write and tune complex SQL; read execution plans to optimize performance. Encapsulate business rules in a testable layer; avoid ad-hoc scripts. Design orchestration layer with dependencies, backfills, and observability. Model data with clear grain, keys, and measures; discuss storage formats." Design automated lineage capture and maintain dataset catalogs; ensure auditability. Provide production support for pipelines; participate in on-call rotation. Translate stakeholder questions into data requirements; mentor Level 1 engineers. Bachelor’s degree in CS/IT or equivalent practical experience. At least 3 years building and operating production data pipelines. Strong Python at production quality with tests and logging. Advanced SQL with window functions and performance tuning. Hands-on AWS data services including S3, Glue, Lambda, Redshift. Experience with non-transactional stores and data lake concepts. Orchestration frameworks and CI workflows experience. Data modeling including grain/keys and slowly changing dimensions. Clear communication and ability to mentor junior staff.Skills
Sign in to read and join the discussion.