State Space Models
Apple Study Examines Why Gated State Space Models Struggle to Learn in Context
A paper published by Apple examines how gating affects State Space Models, finding that it can steer them toward memorization in model weights and delay or prevent…
Apple paper proposes weighted state souping for long-context AI
A paper published on Apple’s machine learning research site proposes weighted state souping, a method designed to improve how state space models retrieve information…