New: Intelligent Flashcards spaced-repetition decks for every topic, built to make things actually stick.
When Byte-Pair Encoding builds its vocabulary, how does it decide which pair of symbols to merge next?
It merges the pair that would form the longest possible subword
It merges the adjacent pair of symbols that shows up most often in the training text
It merges whichever pair most raises the likelihood of the training data