Question 3
Consider the following dictionary with the number of word occurrences in a corpus:
wo = { "deeper": 5, "keener": 6, "sweeter": 7,}Note: Append identifier/special symbol </w> to each word at the end.
You will be learning a byte pair encoding, answer the given subquestions in that context:
What is the most frequent byte-pair before the very first merge? Say the most frequent byte pair is (‘a’,‘b’), then enter “ab” (without quotes and white spaces). If there is a tie between two or more candidates, pick the one that occurs first in the original vocabulary.