Skip to main content

One doc tagged with "decoding"

View all tags

Decoding Strategies

The model outputs a probability distribution over the entire vocabulary at every step — something has to turn that distribution into an actual sequence of chosen tokens. That "something" is decoding, and it's a design decision entirely separate from the model itself: the same weights, decoded two different ways, can produce text that reads as either robotic and repetitive or lively and varied.