Software Engineer — RISC-V SIMD Kernels for LLMs

About the Project
The team builds low-level compute kernels for inference/training of well-known LLM models, hand-optimized for a proprietary (custom) chip based on the RISC-V architecture with RISC-V Vector Extension (SIMD intrinsic extension) and a custom VCIX Coprocessors. In terms of functional scope, this is analogous to what PyTorch/Tensorflow provides (matmul/GEMM, convolution, elementwise ops, reduction, attention/softmax, quantization) — but written from scratch for a specific chip rather than routed through an existing backend.

What You Will Actually Do
Write new compute kernels (functionally equivalent to PyTorch operators: matmul/GEMM, convolution, elementwise, reduction, attention/softmax) targeting custom RISC-V hardware.
Implement and tune mixed-precision inference pipelines across multiple numerical formats (FP32, BF16, MXFP8, MXFP4, NVFP4).
Investigate and improve the performance of existing kernels and how they are invoked: optimize cache vs. DRAM memory usage (memory hierarchy, tiling/blocking, prefetching, bandwidth vs. latency trade-offs).
Optimize distributed communication for multi-device and multi-node inference.
Partition and schedule workloads across compute cores to maximize hardware utilization.
Fuse individual kernels into efficient end-to-end pipelines for LLM model execution.
Write C++23 code with heavy use of metaprogramming (templates, constexpr, concepts, CRTP, SFINAE) to generate specialized code for different kernel configurations.
Required Qualifications
Hands-on experience writing low-level compute kernels — GEMM/matmul, convolution, elementwise, reduction, or similar — regardless of whether this was for PyTorch/ATen, XLA, CUDA, ARM NEON/SVE, x86 AVX, or other hardware. The key requirement is actual kernel-writing experience, not just usage of high-level ML frameworks.
Experience with SIMD/SIMT/vector intrinsics on any architecture (RVV, ARM NEON/SVE, x86 SSE/AVX/AVX-512) or with inline assembly, understanding how the compiler maps intrinsics to instructions etc.
Solid understanding of memory hierarchy: cache vs. DRAM, cache lines, latency/bandwidth trade-offs, data locality, tiling/blocking for compute optimization.
Strong interest in and willingness to optimize code for performance and memory efficiency, with the ability to identify bottlenecks and continuously improve execution speed, memory usage, and overall system efficiency.
Strong focus on latency minimization in production inference paths — able to identify and eliminate latency-critical bottlenecks across the full execution pipeline, not just individual kernels.
Comfortable working in a Linux-like environment for building/testing (WSL/Linux), and experience debugging without full GUI debugger tooling (log-based debugging).

Strong Pluses
Direct RISC-V experience (ISA, RVV / custom intrinsic extensions, cross-compilation with a RISC-V toolchain — GCC/LLVM/Clang).
Experience optimizing kernels for LLM inference/training (attention, KV-cache, softmax, layer norm, quantized matrix multiplication).
Experience with proprietary/custom hardware (not reference architectures), including writing code for early-stage chips where tooling and debugging capabilities are limited.
Understanding of collective communication operations, distributed training (Data/Tensor/Pipeline Parallelism).
Understanding of floating-point formats — FP32, BF16, MXFP8, NVFP4/MXFP4 — including their trade-offs in precision, dynamic range, and performance.
Understanding of how numerical format choice affects throughput, memory bandwidth, and training/inference numerical stability.
Strong C++ (17/20/23) with confident use of template programming — real experience in a generic/template-heavy codebase, not just familiarity with the syntax.