TLDR Tech / Data
GreenBoost — 3-Tier GPU Memory Extension for Linux
GreenBoost is an open-source Linux kernel module and CUDA shim that extends GPU VRAM using system RAM and NVMe, allowing large AI models to run on smaller GPUs without modifying inference code. It works by intercepting CUDA memory allocations and transparently offloading overflow to slower memory tiers, trading some performance for significantly increased capacity.
Read the full story