3,299 SVD subword embeddings from BPE-tokenized metal lyrics

A matrix of 3,299 subword vectors (rows) and 20 dimensions (columns). Embeddings derived from truncated SVD on a PPMI-weighted subword co-occurrence matrix built from a BPE-tokenized (GPT-2 `r50k_base`) encoding of the metal lyrics corpus using text2map::weight_ppmi().

Format

A matrix of 3,299 rows and 20 columns

Source

Derived from the metal lyrics corpus in text2map.corpora

Details

Note: rownames are integer BPE token IDs encoded as character strings (e.g. "3847", "14825"), not human-readable words. Decode them on the caller side with rtiktoken::decode_tokens(as.integer(rownames(mat)), "r50k_base"). `rtiktoken` is NOT a dependency of `text2map.pretrained`.

Examples

if (FALSE) { # \dontrun{

## download the model (once per machine)
download_pretrained("vecs_svd20_metal_bpe")

## load the model each session
wv <- load_pretrained("vecs_svd20_metal_bpe")

## check dims
dim(wv) == c(3299, 20)

## decode rownames to text (caller-side)
toks <- rtiktoken::decode_tokens(as.integer(rownames(wv)), "r50k_base")
} # }