Question 14
Which of the following methods uses/use the concept: "The farther apart two tokens are, the less they should attend to each other''?
RoPE
APE
ALiBi
RPE
Question 14: Which of the following methods uses/use the concept: "Th…
Show answer
Correct answer
ALiBi
Question 14 of 24 in the IIT Madras BS Large Language Models (LLM) End Term paper sat on 21 Dec 2025, in the September 2025 term (Large Language Models 21 Dec 25). It carries 2 marks.
More questions from this paper
- What happens when you use Batch Normalization with a batch size of 1?
- A language model outputs the following logits for the next token: {cat: 3.2, dog: 2.9, bird: 1.1, fish: 0.5, snake: -0.…
- Figure question
- Figure question
- A research team wants to classify scientific abstracts into multiple topics simultaneously (e.g., ''ML'', ''biology'', …
- A company wants to automatically correct noisy OCR text extracted from scanned documents. The text contains spelling mi…
- Figure question
- During autoregressive inference, key-value (KV) caching is used to avoid recomputing keys and values for previously gen…
- Figure question
- Figure question
- If a model uses Absolute Positional Embeddings (APE), increasing the sequence length from 100 to 1000 requires:
- Figure question
- Figure question
- You trained a model with max sequence length 512. At inference, you need to process sequence length 4096 without retrai…
- Figure question
- Figure question
- Figure question
- Figure question
- Figure question
- Figure question
- A GPT-style causal language model is trained using the next-token prediction objective: During inference, the model mus…
- Under which of the following decoding settings can the model produce different outputs across multiple runs on the same…
- Figure question