Learning Word Meta-Embeddings by Using Ensembles of Embedding Sets

Word embeddings -- distributed representations for words -- in deep learning are beneficial for many tasks in Natural Language Processing (NLP). However, different embedding sets vary greatly in quality and characteristics of the captured semantics. Instead of relying on a more advanced algorithm for embedding learning, this paper proposes an ensemble approach of combining different public embedding sets with the aim of learning meta-embeddings. Experiments on word similarity and analogy tasks and on part-of-speech (POS) tagging show better performance of meta-embeddings compared to individual embedding sets. One advantage of meta-embeddings is that they have increased coverage of the vocabulary. We will release our meta-embeddings publicly.