learn deep. ship safe.Practice testsFollow ByteMunSubscribe

mediumLLM basicsSampling

Softmax with Temperature

Turn a model's raw logits into next-token probabilities with temperature scaling, and keep it numerically stable for large logits, as LLM samplers do.

JavaScript or Python

An LLM outputs a raw score (logit) for every token in its vocabulary. Sampling turns them into probabilities with softmax, after dividing every logit by a temperature: below 1 makes the distribution sharper, above 1 makes it flatter.

Given a list of logits and a positive temperature, return the list of probabilities, in the same order, that sum to 1. Logits can be large (such as 1000), so your solution must not overflow. Results are checked to six significant digits.

Examples

Example 1

Input: logits = [1, 1], temperature = 1
Output: [0.5, 0.5]
Why: Equal logits give equal probabilities

Example 2

Input: logits = [2, 1, 0], temperature = 1
Output: [0.665241, 0.244728, 0.090031]
Why: e^2, e^1, e^0 normalised

Constraints

  • 1 <= logits.length <= 1000
  • -10^4 <= logits[i] <= 10^4
  • 0.05 <= temperature <= 10

Hints

Hint 1

Divide each logit by the temperature first.

Hint 2

Subtract the largest scaled logit before calling exp: the result is the same, and nothing overflows.

Solution

Show the solution (try it yourself first)

Softmax is exp(z_i) / Σ exp(z_j) with z = logits / temperature. Calling exp(1000) overflows to infinity, so subtract the maximum z first: the shift cancels out in the division, every exponent becomes at most 0, and the largest term is exactly 1.

import math

def softmax_with_temperature(logits, temperature):
    scaled = [x / temperature for x in logits]
    m = max(scaled)
    exps = [math.exp(x - m) for x in scaled]
    total = sum(exps)
    return [e / total for e in exps]

Complexity

Time O(n), space O(n) for the output. This max-subtraction trick is how every deep-learning framework implements softmax, and the same idea gives log-softmax for stable cross-entropy. A good follow-up to discuss: as temperature approaches 0 the output approaches a one-hot vector on the largest logit, which is greedy decoding.

Found a mistake in this problem, its tests or the solution?

Written by Published Tests checked by Deepak
Case 1Input: [1,1], 1Expected: [0.5,0.5]
Case 2Input: [2,1,0], 1Expected: [0.6652409557748218,0.24472847105479764,0.09003057317038046]

Run checks these 2 samples. Submit checks all 7 tests, including hidden ones.

  1. Maximum Subarray SumArrays · Dynamic programming
    medium
  2. Longest Substring Without RepeatsStrings · Sliding window · Hash map
    medium
  3. Group AnagramsStrings · Hash map · Sorting
    medium
  4. Merge Overlapping IntervalsArrays · Sorting · Intervals
    medium