AAAI Publications, Thirty-First AAAI Conference on Artificial Intelligence

Font Size: 
Distributed Negative Sampling for Word Embeddings
Stergios Stergiou, Zygimantas Straznickas, Rolina Wu, Kostas Tsioutsiouliklis

Last modified: 2017-02-13

Abstract


Word2Vec recently popularized dense vector word representations as fixed-length features for machine learning algorithms and is in widespread use today. In this paper we investigate one of its core components, Negative Sampling, and propose efficient distributed algorithms that allow us to scale to vocabulary sizes of more than 1 billion unique words and corpus sizes of more than 1 trillion words.

Keywords


negative sampling, word embeddings, word2vec

Full Text: PDF