Principal Software Engineer
Summary
A principal engineer designs and maintains market abuse surveillance models and the data pipelines behind them, working across an in-house SQL-based solution, a third-party vendor platform, and AWS data infrastructure. Core stack: Python, Java, SQL, pipeline frameworks (Airflow/dbt/Spark), and AWS services for a financial-services compliance surveillance team.
This role will be responsible for developing various Market Abuse Models and ensuring that these Models are effective. The role will also require work to ensure the reliability, scalability, and compliance of data pipelines that support the surveillance systems. The role will involve collaborating closely with internal Business Analysts our QA team, with a third-party vendor surveillance solution and working on an in-house SQL based solution.
Role Responsibilities
- Design, build and maintain Models in the in-house Market Abuse Surveillance system
- Identify opportunities to leverage AI solutions to improve the quality and effectiveness of the solutions delivered to Compliance Surveillance stakeholders.
- Re-engineer the data mapping layer to the vendor solution
- Write unit, integration, and infrastructure tests
- Working with Data Engineers in the team, contribute to the design and build of data pipelines and workflows between the source data points and target destinations.
- Ensure solutions delivered are in compliance with regulatory, legal, and security protocols.
- Work closely with surveillance analysts, compliance officers, and engineering teams to translate business rules into technical specifications.
- Partnering closely with stakeholders and subject matter experts such as the Cloud Infrastructure team to optimise performance and costs.
- Stay updated on industry trends and emerging tech to ensure continuous improvement
Experience / Competences
- Experience of building business rules within SQL
- Strong proficiency in Python and Java, SQL, and data pipeline frameworks (e.g., Airflow, dbt, Spark), with solid experience with the AWS ecosystem.
- Strong experience ETL/ELT data pipeline builds from design, to implementation, to maintenance in relation to financial market messaging platforms, and trade & order systems.
- Experience with some of EKS, Lambda, EventBridge, Step Functions, S3, DynamoDB, AWS Glue, Snowflake, Terraform and Transfer Family.
- Knowledge of streaming technologies (Kafka, Kinesis) and API integrations, and hands-on experience with monitoring tools (e.g. Grafana) and observability practices.
- Excellent problem-solving skills and ability to work in a fast-paced environment.
- Bachelor’s degree in Computer Science, Data Science, Engineering, or related field.
- Strong communication and collaboration skills to engage with technical and non-technical stakeholders.
- Strong experience with Agile software delivery.
- Experience building solutions using Amazon Kiro
- Experience with market data ingestion, metadata extraction, and event-driven architectures.
- Familiarity with security best practices, IAM, and VPN configuration.
- Experience with regulatory compliance and data security in financial services.
- Knowledge of financial markets and trading platforms.
- Experience with GitLab, Qliksense & Alation