Senior Data Engineer - (NPW)

Role summary

Owns the Enterprise Knowledge Ingestion Framework — pipeline hardening, chunking, metadata extraction, indexing, versioning, and synchronization extension to new sources; owns integration with the client's operational data platform API and the Planisware field expansion for an upcoming release, and the dual-source retrieval architecture with source attribution and fallback protocol; and owns the enterprise ingestion RAG pipeline and Databricks Enterprise Vector Store integration, supporting the Knowledge Discovery and Tribal Intelligence Suite.

Key responsibilities

• Harden the Enterprise Knowledge Ingestion Framework pipeline
• Implement chunking, metadata extraction, indexing and versioning
• Extend synchronization to Teams, Smartsheet, and the client's action-tracking system
• Build and maintain the integration with the client's operational data platform API
• Deliver the Planisware field expansion for an upcoming release and verify the operational data platform API's completeness
• Build the dual-source retrieval architecture with source attribution and a defined fallback protocol
• Build and maintain the enterprise ingestion RAG pipeline
• Own Databricks Enterprise Vector Store integration
• Build knowledge-base pipelines supporting the Knowledge Discovery and Tribal Intelligence Suite

Required skills & experience

• 5+ years data engineering experience, including ingestion pipeline design
• Experience with chunking / metadata-extraction strategies for RAG pipelines
• Vector store indexing, versioning and synchronization experience
• Familiarity with Microsoft 365 (Teams/SharePoint) and Smartsheet integration a plus
• Senior-level data engineer with strong Python, REST API integration and ETL pipeline design skills
• Experience integrating enterprise planning or operational systems
• Experience designing fallback / source-attribution protocols for dual-source retrieval
• Hands-on Databricks experience
• Embeddings and vector-store pipeline management experience
• RAG integration experience

Preferred qualifications

• Experience enumerating and scoping ambiguous 'structured database' source classes
• Prior Smartsheet or Teams API integration experience
• Direct Planisware integration experience
• Experience enumerating query classes in scope for an ambiguous dual-source integration
• Databricks Enterprise Vector Store production experience specifically
• Experience supporting a Should-tier, conditionally-committed deliverable

Education

B.Tech/M.Tech in Computer Science, Data Engineering, or a related field, or equivalent practical experience.