AI/ML Inference Engineer – Framework Internals

✅ About the job
Your job is to make popular ML frameworks (transformers, vLLM, llama.cpp, NVIDIA Dynamo, etc.) talk to our hardware, routing the compute-heavy forward pass to our hardware while keeping the rest of the framework ecosystem intact.

This means reading framework internals deeply, identifying where to intercept execution, and writing glue code that is correct, maintainable, and performant. You will also track the fast-moving inference ecosystem and flag opportunities relevant to our architecture.

✅ Key Responsibilities
● Intercept and reroute framework model execution to the LPU runtime, with hybrid execution across LPU and GPU where needed.
● Analyze integration gaps across frameworks and determine what to leverage vs. replace
● Track the LLM inference ecosystem and identify what matters for our architecture

✅ Qualifications
● Strong Python for framework-level work; C++ proficiency for bindings and performance-critical components
● Understanding of transformer compute primitives: attention, KV-cache, matrix ops, quantization

✅ Nice to have
● Contributions to open-source LLM inference projects (PRs, not just issues)
● Familiarity with compiler IR, runtime systems, or kernel dispatch mechanisms

✅ What we offer
● Full access to source code across the entire stack
● Full access to paid AI tools — we embrace fast iteration and encourage using them freely
● Small team, high ownership — your ideas can influence decisions at every level

✅ How to Apply
Send your resume. If you have relevant framework contributions, link them — this is the strongest signal we look for.