5

I am looking to employ Word2Vec to cluster documents and classify them by topic. However, it seems I need a significant data set [1,2] to achieve such a task.

I have a data set of thousands (not millions) of documents where my total word count is somewhere in the tens of thousands or maybe low hundreds of thousands.

Is there another ML methodology that allows me to attain my goal?

I see how to use TF/IDF to generate words and phrases from a corpus but output as expected is a list of common words and phrases along a flat dimension: enter image description here

What I am looking for is something more along the lines of a high level cluster of vectors in space:enter image description here [source]

Chris
  • 193
  • 1
  • 6
  • Use a pre-trained word embedding then calculate the document embeddings using [this simple algorithm](https://arxiv.org/abs/1704.05358v1). – Emre Jun 02 '17 at 18:09
  • @Emre thank you! Do u have time to submit an answer with a bit more detail for an imbecile like my self? This sounds perfect but confusing :) – Chris Jun 02 '17 at 18:13

2 Answers2

4

You can side step the paucity of training data, and indeed training altogether, by using pre-trained embeddings in numerous languages. After that you can calculate your document embeddings using one of these simple algorithms, which basically amount to running dimensionality reduction on the matrix of stacked word embeddings for each sentence using PCA/SVD:

Note that word embeddings themselves emerge from similar calculations:

Emre
  • 10,481
  • 1
  • 29
  • 39
2

Doc2vec (aka paragraph2vec, aka sentence embeddings) modifies the word2vec algorithm to unsupervised learning of continuous representations for larger blocks of text, such as sentences, paragraphs or entire documents.

It will going to cluster each documents topics in vector space , learn it's semantic meaning. It will perform good with your given size dataset size.check this too Doc2Vec - How to label the paragraphs (gensim)

Abhishek Verma
  • 136
  • 1
  • 8