WordPiece
AIWordPiece: A subword tokenisation algorithm, used by BERT, that chooses merges to maximise the likelihood of the training text rather than raw frequency.
WordPiece: A subword tokenisation algorithm, used by BERT, that chooses merges to maximise the likelihood of the training text rather than raw frequency.