Large Language Models Quiz 2: 1 December 2024 (September 2024 term)

Consider the following dictionary with the number of word occurrences in a corpus:

Note: Append identifier/special symbol </w> to each word at the end.

You will be learning a byte pair encoding, answer the given subquestions in that context:

How many tokens are there in the initial vocabulary?

Consider the following dictionary with the number of word occurrences in a corpus:

Note: Append identifier/special symbol </w> to each word at the end.

You will be learning a byte pair encoding, answer the given subquestions in that context:

Which of the following pairs has the least frequency before any merge?

Consider the following dictionary with the number of word occurrences in a corpus:

Note: Append identifier/special symbol </w> to each word at the end.

You will be learning a byte pair encoding, answer the given subquestions in that context:

What is the most frequent byte-pair before the very first merge? Say the most frequent byte pair is (‘a’,‘b’), then enter “ab” (without quotes and white spaces). If there is a tie between two or more candidates, pick the one that occurs first in the original vocabulary.