single mention · data curation
Subtoken-averaged special-token embedding initialization
A new token's embedding is initialized as the mean of its constituent subtoken embeddings.
The sources on this page only mention it in passing. The line above is the extraction's one-line gloss. See tokenization for the methods filed beside it.
Also called subtoken averaging.
Filed under data curation › tokenization