Main Content

Word embedding layer for deep learning networks

A word embedding layer maps word indices to vectors.

Use a word embedding layer in a deep learning long short-term memory (LSTM) network. An LSTM network is a type of recurrent neural network (RNN) that can learn long-term dependencies between time steps of sequence data. A word embedding layer maps a sequence of word indices to embedding vectors and learns the word embedding during training.

This layer requires Deep Learning Toolbox™.

creates a word embedding layer and specifies the embedding dimension and vocabulary
size.`layer`

= wordEmbeddingLayer(`dimension`

,`numWords`

)

sets optional properties
using one or more name-value pairs. Enclose each property name in single quotes.`layer`

= wordEmbeddingLayer(`dimension`

,`numWords`

,`Name,Value`

)

[1] Glorot,
Xavier, and Yoshua Bengio. "Understanding the Difficulty of Training Deep Feedforward Neural
Networks." In *Proceedings of the Thirteenth International Conference on Artificial
Intelligence and Statistics*, 249–356. Sardinia, Italy: AISTATS,
2010.

[2] He, Kaiming,
Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Delving Deep into Rectifiers: Surpassing Human-Level
Performance on ImageNet Classification." In *Proceedings of the 2015 IEEE
International Conference on Computer Vision*, 1026–1034. Washington, DC: IEEE
Computer Vision Society, 2015.

[3] Saxe, Andrew M., James L. McClelland, and Surya Ganguli. "Exact solutions to the nonlinear dynamics of learning in deep linear neural networks." *arXiv preprint arXiv:1312.6120* (2013).

`trainNetwork`

(Deep Learning Toolbox) | `doc2sequence`

| `trainWordEmbedding`

| `wordEncoding`

| `lstmLayer`

(Deep Learning Toolbox) | `sequenceInputLayer`

(Deep Learning Toolbox) | `fastTextWordEmbedding`

| `tokenizedDocument`

| `word2vec`