V

vllm

模型路由器：在 Codex Desktop 和 Claude Desktop 使用 Claude、第三方 API 与自定义模型。Local LLM proxy for Codex & Claude Desktop: OpenAI Responses / Chat Completions & Anthropic Messages, Claude Code subscription routing. macOS & Windows.

useful Gentoo overlay Curated ebuilds, AI, tools & science

Benchmark harness and measured results: GGUF vs NVFP4 (llama.cpp and vLLM) and DFlash speculative decoding on a 12 GB Blackwell (sm120) GPU

Make small local LLMs reliable at multi-step agent workflows. Constrained decoding + verify-before-commit: 96.5% end-to-end vs 2.0% unguarded on a 50-step run. Zero dependencies, runs on-prem.

An inference-memory layer for vLLM: the conversation's history in host RAM, the GPU as a bounded working set · KV-cache offload for LLM inference, Intel GPU and SYCL/oneAPI engineering, C++ and native ports with parity proof, GPU inference optimisati

Put a queue, deduplication, a circuit breaker and a dead-letter queue in front of any LLM API (Anthropic, OpenAI, llama.cpp, vLLM), so repeated prompts cost one call and outages fail fast. Rust library, HTTP server and CLI.

A modular experimentation framework for evaluating LLM orchestration strategies over real 5G network capabilities.

Experiments with Subliminal Learning

VRAM to RAM Offloader for AI and vLLM - High-Performance C++23 KV Cache Engine with Multi-Stream GPU Transfers

https://ayinedjimi-consultants.fr

Extreme KV Cache Compression for LLM Inference — C++17/CUDA implementation of TurboQuant (arXiv 2504.19874). 7.5x compression, <2% quality loss.

https://ayinedjimi-consultants.fr

Track AI spending across API providers, MCP tools, subscriptions, and self-hosted GPUs from your terminal. One CLI to see what you're actually paying — Anthropic, OpenAI, OpenRouter, vLLM on your own hardware — with real TCO math. Compare your self-hosted $/MTok vs cloud pricing. Local-first, no SaaS required.