Senior Software Engineer / Tech Lead - Message Security Detection
Summary
Lead the architecture of high-throughput threat detection systems using Spark, Airflow, AWS, and EKS.
The Mission
We are building the high-throughput, low-latency infrastructure that powers our real-time threat detection capabilities. This isn't just about maintaining stable backend systems; it’s about architecting a cutting-edge data platform at massive scale to turn raw data into high-fidelity, actionable intelligence for our ML models and security researchers.
As a Tech Lead (SWE-3) on this team, you will be our technical anchor and force multiplier. You will own the architectural blueprint for our distributed data pipelines (Spark/Airflow) and cloud-native infrastructure (AWS/EKS), bridging the gap between complex system design and strategic execution. Beyond traditional engineering, you will lead by example spearheading our AI productivity initiatives to supercharge how our entire engineering team builds, tests, and ships software. If you are passionate about deep technical ownership, mentoring the next generation of engineers, and defining the future of our data ecosystem, this is your mission.
What You Will Do
- Architect & Scale: Design and maintain high-throughput detection systems and backend services, ensuring low latency and high reliability.
- Data Orchestration: Lead the development of robust data pipelines using Spark and Airflow to supply detection teams with high-fidelity data for ML model training and evaluation.
- Technical Leadership: Partner with Engineering Managers to define technical roadmaps, mentor junior/mid-level engineers, and drive best practices in code quality and system design.
- AI-Driven Innovation: Lead the team in adopting AI productivity initiatives, leveraging tools like Claude, OpenAI, and GitHub Copilot to accelerate the development lifecycle and automate internal workflows.
- Cross-functional Collaboration: Work closely with TPMs, Product Managers, Data Scientists, and Security Researchers to translate complex business needs into scalable technical solutions.
- Operational Excellence: Manage and optimize cloud-native infrastructure on AWS/EKS, ensuring systems are cost-effective, secure, and performant.
Must Haves
- 8+ years of professional experience: Proven track record in production-level backend development with Python or Golang.
- Data Engineering Expertise: Expert-level proficiency in building and optimizing distributed data pipelines using Spark and Airflow (or equivalent orchestration tools).
- Cloud & Infrastructure: Extensive experience managing and scaling cloud-native applications on AWS (highly preferred), GCP, or Azure, with a proven track record of hands-on container orchestration using EKS.
- Leadership & Mentorship: Experience leading technical projects, mentoring engineers, and contributing to the long-term technical strategy of a team.
- System Design: Strong ability to design complex integrations and handle significant throughput/latency challenges.
- Problem Solving: A methodical approach to performance debugging, benchmarking, and resolving bottlenecks in large-scale systems.
- Database Proficiency: Strong experience with Postgres or similar relational databases at scale.
- Education: BS in Computer Science, Applied Sciences, or a related engineering field.
Nice to Have
- Frontend Literacy: Familiarity with React and TypeScript to assist with internal tool visualizations.
- Data Ecosystems: Experience with Databricks, Snowflake, or similar data lakehouse architectures.
- Cybersecurity Domain: Background in threat detection, network security, or fraud prevention.
- Advanced Degrees: MS in Computer Science or Electrical Engineering.
#LI-LR1
AI and our hiring process
Abnormal AI uses AI-assisted tools to help our recruiting team prepare for candidate interviews. These tools analyze resume content and role requirements to suggest interview questions and identify areas for the interviewer to explore. They do not make hiring decisions or screen candidates automatically. Every decision about a candidacy is made by a person.
Abnormal AI is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability, protected veteran status or other characteristics protected by law. For our EEO policy statement please click here. If you would like more information on your EEO rights under the law, please click here.