AI Inference Engineer
Posted 5
views
Summary
Builds and optimizes large-scale AI model inference pipelines: deploying models efficiently with PyTorch, TensorRT, and vLLM, and improving performance through quantization, kernel fusion, efficient batching, and system-level debugging. Requires strong C/C++ or Python systems skills and 2+ years developing software targeting AI hardware.
- Building and optimizing inference pipelines for large-scale model serving
- Working with frameworks like PyTorch, TensorRT, and vLLM to deploy models efficiently
- Implementing and optimizing ML models using techniques such as quantization, kernel fusion, and efficient batching
- Optimizing and implementing core ML operators
- Investigating and resolving issues through system-level debugging and performance analysis
- Defining and applying practices for testing, deployment, and scaling AI systems
- BSc/MSc in Computer Science, Engineering, Mathematics, or related discipline
- Strong programming skills in C/C++ or Python in Linux environments using common development tools
- Solid knowledge of computer architecture, system software, data structures
- Hands-on experience implementing algorithms in high-level languages
- Exposure to specialized hardware and frameworks such as OpenCL or CUDA
- Experience designing or working with high-performance software systems
- Solid knowledge of ML fundamentals
- Experience in model serving frameworks
- Experience in ML runtimes
- Experience deploying ML workloads across distributed systems
- Experience implementing and optimizing ML operators and kernels with a focus on vectorization and efficient execution
- Experience in hardware-aware optimizations and performance tuning
- 2+ years of experience developing software targeting AI hardware
- Contribution to open-source projects
