Performance Engineer, Inference

Performance Engineer, Inference at Sarvam is a senior role focused on optimizing the production serving path for large distributed LLMs. This involves deep integration and modification of serving runtimes like SGLang or vLLM, managing distributed systems for prefill-decode, and building custom speculators to meet latency and throughput targets.

Skills and focus areas

• performance engineering
• inference optimization
• distributed systems
• serving runtimes
• gpu programming
• model integration
• latency optimization
• throughput optimization
• sglang
• vllm
• nvidia dynamo
• tensorrt-llm