Positive Pointwise Mutual Information Weighting

Computes the positive pointwise mutual information (PPMI) weighting of a term co-occurrence matrix (TCM). PPMI is a standard transformation applied prior to building word embeddings via matrix factorization (e.g. SVD): it down-weights chance co-occurrences and clips negative associations to zero, yielding a non-negative matrix amenable to factorization.

Usage

weight_ppmi(tcm, smooth = 0)

Arguments

tcm: A term co-occurrence matrix; either a dense matrix or a sparse Matrix::dgCMatrix. Row and column names are preserved.
smooth: A non-negative numeric scalar added to every cell of tcm (including zeros) before estimating probabilities. Defaults to 0, yielding standard PPMI. Values greater than zero nudge the whole distribution and reduce the influence of rare co-occurrences.

Value

A sparse dgCMatrix of the same dimensions as tcm containing PPMI weights, with row and column names preserved.

Details

The estimator uses marginal probabilities (row and column sums) rather than a diagonal-count proxy, following the standard formulation of Levy & Goldberg (2014). Smoothing, if requested, is applied to the raw counts (including zero cells) before estimating probabilities.

References

Levy, O., & Goldberg, Y. (2014). Neural word embedding as implicit matrix factorization. Advances in Neural Information Processing Systems (NeurIPS).

Author

Dustin Stoltz

Examples

# \donttest{
tcm <- matrix(c(2, 1, 0, 1, 3, 1, 0, 1, 2), nrow = 3,
              dimnames = list(c("a", "b", "c"), c("a", "b", "c")))
weight_ppmi(tcm)
#> 3 x 3 sparse Matrix of class "dgCMatrix"
#>           a         b         c
#> a 0.8938179 .         .        
#> b .         0.2776317 .        
#> c .         .         0.8938179
# }