mediumLLM basicsSampling
Softmax with Temperature
Turn a model's raw logits into next-token probabilities with temperature scaling, and keep it numerically stable for large logits, as LLM samplers do.
JavaScript or Python
An LLM outputs a raw score (logit) for every token in its vocabulary. Sampling turns them into probabilities with softmax, after dividing every logit by a temperature: below 1 makes the distribution sharper, above 1 makes it flatter.
Given a list of logits and a positive temperature, return the list of probabilities, in the same order, that sum to 1. Logits can be large (such as 1000), so your solution must not overflow. Results are checked to six significant digits.
Examples
Example 1
Input: logits = [1, 1], temperature = 1 Output: [0.5, 0.5] Why: Equal logits give equal probabilities
Example 2
Input: logits = [2, 1, 0], temperature = 1 Output: [0.665241, 0.244728, 0.090031] Why: e^2, e^1, e^0 normalised
Constraints
1 <= logits.length <= 1000-10^4 <= logits[i] <= 10^40.05 <= temperature <= 10
Hints
Hint 1
Divide each logit by the temperature first.
Hint 2
Subtract the largest scaled logit before calling exp: the result is the same, and nothing overflows.
Solution
Show the solution (try it yourself first)
Softmax is exp(z_i) / Σ exp(z_j) with z = logits / temperature. Calling exp(1000) overflows to infinity, so subtract the maximum z first: the shift cancels out in the division, every exponent becomes at most 0, and the largest term is exactly 1.
import math
def softmax_with_temperature(logits, temperature):
scaled = [x / temperature for x in logits]
m = max(scaled)
exps = [math.exp(x - m) for x in scaled]
total = sum(exps)
return [e / total for e in exps]Complexity
Time O(n), space O(n) for the output. This max-subtraction trick is how every deep-learning framework implements softmax, and the same idea gives log-softmax for stable cross-entropy. A good follow-up to discuss: as temperature approaches 0 the output approaches a one-hot vector on the largest logit, which is greedy decoding.
Found a mistake in this problem, its tests or the solution?