TensorRT-LLM
NVIDIA outlines five ways to make every AI memory byte work harder
NVIDIA has outlined five co-designed technologies aimed at increasing the amount of useful AI work produced by each byte of memory. The company says the approach enables…
ReplaySSM Caches Inputs Instead of State to Speed Up Hybrid-Model Decoding
NVIDIA and the Dao AI Lab at Princeton University have detailed ReplaySSM, a technique that stores state-space model (SSM) state as a checkpoint plus a buffer of recent…