Software Engineer, Model Runtime

Software Engineer, Model Runtime at OpenAI will build the model runtime within the inference engine for executing complex, frontier models at scale on custom silicon. This role involves designing and implementing LLM inference runtimes, building scheduling and memory management systems, and developing distributed execution strategies.

Skills and focus areas

• llm inference
• custom silicon
• scheduling
• continuous batching
• memory management
• kv-cache management
• execution orchestration
• distributed systems
• model partitioning
• communication
• synchronization
• latency optimization