Two-Clustering Regime of Token Dynamics in Causal Attention
Summary
This study partially resolves and extends a conjecture on token clustering under causal attention masking, revealing how the dynamics evolve over time.
Why this matters
Relevant to this topic
Source: arXiv cs.AI · arxiv.org