Senior Data Engineer

Company Overview

hermeneutic research is a best-in-class proprietary trading firm. It deploys research-driven discretionary and systematic strategies as well as makes strategic long-term investments. The partners' decade-long history of success in trading and business building and a firm-wide cultural emphasis on alpha generation, open debate, relentless iteration, and teamwork are key to the firm's continued expansion in a challenging market environment that has hamstrung competitors. A hard-wired emphasis on risk management and opportunistic market participation ensures that hermeneutic research will continue its growth trajectory in the coming decades.

Job Overview

We are looking for a Senior Data Engineer to help us architecting, implementing and operating the complete data infrastructure pipeline for our Research and Trading operations. This role will be crucial in building a scalable, reliable, and cost-efficient system for handling vast amounts of market trading data, real-time news feeds and a variety of internal and external data sources. The ideal candidate will be a hands‑on professional who understands the entire data lifecycle and can drive innovation while collaborating across research and engineering teams to meet their needs.

Key Responsibilities

• Design, build, and optimize scalable pipelines for ingesting, transforming, and integrating large-volume datasets (market data, news feeds and various unstructured data sources).
• Ensure data quality, consistency, and real-time monitoring using tools like DBT, 3rd party libraries that can facilitate data validation processes.
• Develop processes to normalize and organize our data warehouse for use across different departments.
• Apply advanced data management practices to ensure the scalability, availability, and efficiency of data storage.
• Ensure the infrastructure supports trading and research needs while maintaining data integrity, security, and performance at scale.
• Collaborate with research and analytics teams to understand their data needs and build frameworks that empower data exploration, analysis, and model development.
• Create tools for overlaying data from multiple sources.
• Ensure that data storage, processing, and management are done in a cost-effective manner, optimizing both hardware and software resources.
• Implement solutions that balance high performance with cost control.
• Stay ahead of the curve by continuously evaluating and adopting the most suitable technologies for the organization's data engineering needs.
• Ensure that company's systems align with the latest best practices in data management.

Must-Haves

• Strong problem-solving and analytical thinking
• Clear communication skills for cross-functional collaboration
• Proficiency in building robust data quality checks for ingested data
• Experience identifying anomalies in ingested data.
• Strong proficiency in writing complex SQL (and similar) queries and optimize performance.
• Proficiency in Python or Java / Scala.
• Experience building and maintaining complex ETL pipelines with tools like Apache Airflow, dbt, or custom scripts.
• Strong understanding of dimensional modeling, star/snowflake schemas, normalization/denormalization principles, etc.
• Proven experience with platforms like Snowflake , Redshift , BigQuery , Synapse , etc.
• Expert knowledge of Apache Spark , Kafka , Flink , or similar.
• Strong understanding of data security and privacy standards

Preferred Qualifications

• A degree in Computer Science, Engineering, Mathematics, or a related field.
• Familiarity with one of the major cloud platforms ( AWS , GCP , Azure ) and their data services (e.g., BigQuery, Redshift, S3, Dataflow, etc.), proven by certifications (e.g., Google Professional Data Engineer, AWS Big Data Specialty or Snowflake's SnowPro Data Engineer )
• Experience with data quality frameworks (e.g., Great Expectations, Deequ or others)
• Experience with Git/GitHub or similar for code versioning.
• Experience with infrastructure-as-code tools (e.g., Terraform, CloudFormation).
• Exposure to containerization/orchestration (Docker, Kubernetes).
• Familiarity with data governance , data lineage , and catalog tools (e.g., Apache Atlas, Amundsen).
• Hands-on with observability and monitoring tools for data pipelines (e.g., Monte Carlo, Datadog).
• Knowledge of machine learning pipelines.
• Prior experience in a trading or financial services environment.

Interview Process

CV Screening - We will review your application based on the qualifications and experience outlined above.

HR Interview - To understand your experience, overall skill set, and cultural fit.

Technical Interview - Interviews with multiple stakeholder teams to assess your skill set and cross-functional collaboration.

CTO Interview - Interview with the CTO to discuss your experience, technical expertise, problem-solving approach, and overall fit for the role.

Co-Founder Interview - A concluding discussion to evaluate cultural fit, strategic alignment, and how you resonate with the firm's valu