Gated DeltaNet
Ai2's StateBench Shows Frontier Models Struggle to Track State Over Long Inputs
Ai2 has introduced StateBench, a benchmark that measures how well language models keep track of entities as they change across long documents. According to Ai2…
Amazon's priming method turns Transformers into hybrid models for under 0.5% of pretraining cost
Researchers at Amazon Web Services have introduced priming, a method that starts from pretrained Transformer weights to build hybrid architectures, which the team says…