I

inference-serving-architecture

A 0.6-billion-parameter model that memorises an agent's tool catalog, with the data, decoder, serving and evaluation around it · TensorRT and TensorRT-LLM engineering, inference serving architecture, CUDA kernel development, MCP server design

An inference-memory layer for vLLM: the conversation's history in host RAM, the GPU as a bounded working set · KV-cache offload for LLM inference, Intel GPU and SYCL/oneAPI engineering, C++ and native ports with parity proof, GPU inference optimisati

A throughput-first GPU inference pipeline on consumer NVIDIA hardware: broker, saturated runtime, completion, vector store · high-throughput batch inference, token-aware scheduling, GPU inference optimisation, CUDA kernel development