Lead Data Engineer

Job Description

Experience Level: 5 – 7+ Years

Preferred Industry: High-Frequency Trading (HFT) or Hedge Funds

Key Responsibilities:

• Handle the entire data delivery process, including data procurement, vendor

discussions, process setup, pipeline design and deployment (with logging and alerts),

and database insertion, optimizing for minimal data delivery cycles.

• Build applications that simplify data querying for quantitative developers.

• Work with multiple datasets to deliver standardized data outputs.

• Break down large, abstract tasks into manageable deliverables within set deadlines.

• Identify business requirements, communicate them effectively to end users, and clarify

technical details as needed.

• Distinguish between user needs and wants to prioritize effectively.

• Proactively explore and propose new ideas.

• Quickly assume responsibilities from senior staff to support the team.

• Adhere to safe data handling practices.

Essential Skills:

• Strong command of Python, Bash, and SQL in a Linux environment.

• Understanding of low-level system design for efficient data delivery applications.

• Ability to explain technical concepts to non-technical stakeholders.

• Hands-on experience with ClickHouseDB and its integrations with Python, Bash, and

C++.

• Familiarity with Jira and Agile methodologies.

• Proficiency with Git and version control workflows.• Comfortable developing over SSH.

• Knowledge of market, alternative, and supporting datasets with strong technical

comprehension.

• Experience leading a team of at least 4 people and delivering robust data solutions.

Ideal Skills:

• Experience handling 1+ TB of daily data processing.

• Ability to build highly efficient and resource-optimized data pipelines.

• Up to date with hardware and software trends relevant to data engineering.

• Ability to write efficient and maintainable code without relying on LLMs.

• Experience working in fast-paced and high-availability proprietary trading

environments.

Nice-to-Have Skills:

• Experience with serialization, lock-free queues, and multithreading in C++.

• Knowledge of multiprocessing and asynchronous programming in Python.

• Familiarity with various data processing frameworks.

• Experience with Azure data services.

Needful Qualities:

• Well-organized, detail-oriented; able to multi-task and keep track of various deadlines

in a fast-paced, high-demand environment.

• Highest level of integrity, empathy, and character.

• Mature and thoughtful, with the ability to operate in a collaborative, team-oriented

culture.

Why Join Us?

• Be part of a fast-growing, innovative firm at the forefront of financial technology.

• Work on challenging, high-impact projects with opportunities for rapid learning and

career growth.

• Competitive compensation package with performance-based incentives.

• A dynamic work environment with exposure to cutting-edge technologies in trading.