bert
Description
A Bidirectional Encoder Representations from Transformer (BERT) model is a transformer neural network that can be fine-tuned for natural language processing tasks such as document classification and sentiment analysis. The network uses attention layers to analyze text in context and capture long-range dependencies between words.
[
returns a pretrained BERT-Base model and the corresponding tokenizer.net
,tokenizer
] = bert
Tip
For document classification workflows, use a bertDocumentClassifier
object with the trainBERTDocumentClassifier
function.
[
specifies additional options using one or more name-value arguments.net
,tokenizer
] = bert(Name=Value
)
Examples
Name-Value Arguments
Output Arguments
References
[1] Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. "BERT: Pre-Training of Deep Bidirectional Transformers for Language Understanding" Preprint, submitted May 24, 2019. https://doi.org/10.48550/arXiv.1810.04805.
[2] Srivastava, Nitish, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. "Dropout: A Simple Way to Prevent Neural Networks from Overfitting." The Journal of Machine Learning Research 15, no. 1 (January 1, 2014): 1929–58
[3] Vaswani, A., N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, I. Polosukhin et al. "Attention Is All You Need." Advances in Neural Information Processing Systems 30, 2017. https://proceedings.neurips.cc/paper_files/paper/2017/file/3f5ee243547dee91fbd053c1c4a845aa-Paper.pdf.
Version History
Introduced in R2023b
See Also
bertDocumentClassifier
| trainBERTDocumentClassifier
| dlnetwork
(Deep Learning Toolbox) | bertTokenizer