Lead Data Engineer

We are looking for a Lead Data Specialist to own and manage the entire data lifecycle for our algorithmic trading systems. This role is critical to the reliability of our research, backtesting, and live trading environment. You will lead data engineering, ETL pipelines, data quality processes, and vendor integrations. This is a high-ownership role working directly with quants, traders, and tech leads.

Responsibilities:

• Data Pipeline Ownership: Design, build, and maintain scalable ETL pipelines for tick, OHLC, fundamentals, and alternative data. Develop automated ingestion systems for both real-time and historical market data. Ensure fault-tolerant, high-availability data workflows.
• Data Quality and Integrity: Detect and fix data anomalies: stale ticks, bad prints, missing data, and timestamp issues. Implement automated QA, validation checks, and reconciliation workflows. Own daily data quality reporting.
• Data Architecture and Storage: Manage storage layers. Optimize data retrieval for quants and live trading systems. Implement data versioning and point-in-time correctness for backtesting.
• Vendor and Exchange Data Management: Integrate and manage feeds from Bloomberg, Refinitiv, FactSet, Tick data providers, NSE/BSE, and global exchanges. Be the primary contact for vendor issue resolution. Ensure compliance with vendor licensing and data usage policies.
• Cross-Team Collaboration: Work closely with quant researchers to understand data needs and create feature-ready datasets. Support engineering teams in live feed integration, API access, and latency monitoring. Work with product and compliance on data governance and documentation.
• Leadership and Mentorship: Lead a small team of data engineers/analysts. Review code, enforce best practices, and drive technical excellence. Build a roadmap for data capabilities and automation.

Requirements:

• 6-10 years of experience in market data, data engineering, or financial data operations.
• Strong understanding of tick data, corporate actions, exchange market structures, and vendor feed behaviors.
• Hands-on experience with Python (mandatory), SQL, and distributed databases, ETL frameworks, and message queues (Kafka/Pulsar a plus).
• Strong debugging skills for live and historical data issues.

Good to Have:

• Experience in a quant / HFT / prop trading environment.
• Familiarity with low-latency market data feeds.
• Understanding of microstructure, order book data, and corporate action adjustments.
• Exposure to ML pipelines (feature stores, labeling).