一個好的自然語言模型，若出現了語言模型的 vocabulary set 未曾收錄的單字，語言模型就會產生 Out-of-Vocabulary (OOV)。 本文介紹 subword algorithms 介於 word-level 和 character-level 解決 OOV 的方法。

職場產業

閱讀書評

學習

[探索] 語言模型基礎：Subword algorithms

Professor Christopher Manning, Stanford University
http://onlinehub.stanford.edu/ 

Professor Christopher Manning
Thomas M. Siebel Professor in Machine Learning, Professor of Linguistics and of Computer Science
Director, Stanford Artificial Intelligence Laboratory (SAIL)
 
To follow along with the course schedule and syllabus, visit: http://web.stanford.edu/class/cs224n/index.html#schedule
 
To get the latest news on Stanford’s upcoming professional programs in Artificial Intelligence, visit: http://learn.stanford.edu/AI.html
 
To view all online courses and programs offered by Stanford, visit: http://online.stanford.edu

SentencePiece