Lead Data Engineer

Responsibilities:

• Design and maintain scalable ETL pipelines for real-time and historical market data.
• Build reliable, fault-tolerant data ingestion and processing systems.
• Ensure data quality through validation, reconciliation, and monitoring frameworks.
• Optimize data storage and retrieval for research, backtesting, and live trading.
• Manage market data vendor integrations (Bloomberg, Refinitiv, FactSet, exchanges, etc. ).
• Collaborate with quant researchers and engineers to build feature-ready datasets.
• Support live data feeds, APIs, and production monitoring.
• Lead and mentor a small team while driving engineering best practices and technical excellence.

Requirements:

• 5-8 years of experience in data engineering, market data engineering, or financial data operations.
• Strong expertise in Python, SQL, ETL frameworks, and Kafka (Pulsar is a plus).
• Solid understanding of tick data, corporate actions, exchange market structure, and vendor feeds.
• Experience building scalable, high-performance data pipelines and troubleshooting production systems.
• Prior experience in quantitative trading, HFT, proprietary trading, or financial markets.
• Familiarity with low-latency market data, market microstructure, and order book data.
• Exposure to ML data pipelines, feature engineering, and data preparation.

Nice to Have:

• Experience with distributed data processing and cloud platforms.
• Knowledge of real-time streaming architectures and event-driven systems.
• Background in quantitative research platforms or large-scale financial data infrastructure.