Large Language Models · End Term · 13 Apr 2025 · January 2025 term · Set 1

Question 6: Select correct statements regarding character level token…

Question 6

+2 marksOne or more correct options

Select correct statements regarding character level tokenization:
Note: Assume that the character level vocabulary has English uppercase letters, lowercase letters, digits, special symbols, etc.

Select all that apply.

A
The size of the vocabulary with character-level tokenization will be larger thanthat with word-level tokenization if the corpus consists of all the books published on Harry Potter (in English).
B
The vocabulary will not expand if new English words/sentences are added tothe corpus.
C
No issue of handling unknown tokens as long as the unknown tokens are fromEnglish vocabulary
D
None of these.

Show answer

Correct answers

B
The vocabulary will not expand if new English words/sentences are added tothe corpus.
C
No issue of handling unknown tokens as long as the unknown tokens are fromEnglish vocabulary

See the whole paper Take it as a mock test

Question 6 of 15 in the IIT Madras BS Large Language Models (LLM) End Term paper sat on 13 Apr 2025, in the January 2025 term (IIT M DEGREE FN EXAM QDB2 13 Apr 2025). It carries 2 marks.

This question was also asked in

LLM End Term — 13 Apr 2025, Set 1-2 · question 6

Question 6: Select correct statements regarding character level token…

Question 6

This question was also asked in

More questions from this paper