specific method · filed under model architecture
Hierarchical Sparse Indexer
Restricts later indexing stages to a candidate pool selected by an earlier indexer, reducing repeated scoring.
Also called Hierarchical sparse indexing.
- sources
- 2
- model
- 1
- lab adopt it
- 1
- strongest
- core
How sources treat it
One count per evidence span, weakest treatment to strongest.
Documented in
Evidence
5 spans quoted from the sources, strongest treatment first.
a Hierarchical Sparse Indexer restricts later indexers to a candidate pool selected by an earlier indexer, further reducing the number of entries scored per query.
the first layer assigned to Full Mode constructs a candidate pool that later re-indexing layers use as their search domain.
We therefore introduce the Hierarchical Sparse Indexer, which is used only in the decoder of CED to reduce this repeated scoring during decode.
For the Hierarchical Sparse Indexer, we select a maximum of 2,048 blocks with 8 positions, yielding up to 16,384 candidate positions in total.
In the decoder, a Hierarchical Sparse Indexer further restricts later indexing layers to a candidate pool constructed by the first Full Mode layer, bounding deeper indexer cost independently of context length.
Filed alongside
Other methods under model architecture :: token mixer :: sparse attention :: sparse attention indexer.