Token Sampling Inspector
Free online sampling parameter lab. Drag temperature, top_k and top_p and watch a candidate distribution get scaled, truncated and cut step by step - with every eliminated token left in place, labelled with the parameter that removed it. Runs entirely in your browser.
Token Sampling Inspector
Free online sampling parameter lab. Drag temperature, top_k and top_p and watch a candidate distribution get scaled, truncated and cut step by step - with every eliminated token left in place, labelled with the parameter that removed it. Runs entirely in your browser.
About the token sampling inspector
How it works
At every step a language model produces a probability over its whole vocabulary. Three deciders then narrow it down. temperature rescales the logits - below 1 sharpens the gap between strong and weak candidates, above 1 closes it. top_k keeps only the k highest and renormalises. top_p accumulates from the top until the running total reaches p, keeping the candidate that crosses the line. The four columns here are those four moments, and eliminated candidates stay in place carrying the name of the parameter that removed them.
Why the order matters
The three deciders do not commute. Scaling first means the truncation sees an already reshaped distribution: raising temperature flattens the curve, which lets top_p reach its threshold later and keep more candidates. Truncating first means the cut is judged on the raw probabilities and temperature only reshapes whatever survived. Different SDKs pick different orders, so this tool exposes the choice instead of hiding one behind a default and implying there is a single standard behaviour.
Where the numbers come from
The starting distributions here are constructed, not captured: three shapes chosen because each one makes a different decider visible - one clear favourite, several close rivals, and a long tail. They are labelled as constructed on screen, because a tool that shows you cause and effect has no business being vague about which half is real. The half that is real is the important half: temperature, top_k and top_p are implemented as they are in an actual sampler, so every transition you watch is the genuine computation. What a constructed distribution cannot give you is the feel of a real vocabulary, which is why loading a small model into the browser is offered as the next step rather than treated as an optional extra.
FAQ
Common questions and answers about this topic.
What is the difference between temperature and top_p, and why is tuning both discouraged?
They act on different things. temperature reshapes the whole distribution without removing anyone - it decides how much better a strong candidate looks than a weak one. top_p removes candidates outright, cutting once the cumulative probability reaches your threshold. The reason tuning both is discouraged is that they interfere: raising temperature flattens the curve, so the same top_p now needs more candidates to reach its threshold and keeps tokens you thought you had excluded. Move one, watch the chain, and you can see the interference directly.
What do top_k and top_p actually mean?
top_k is a fixed headcount: keep the k most likely candidates, discard the rest, renormalise. top_p is a fixed share of probability mass, often called nucleus sampling: walk down from the most likely, adding up probabilities, and stop once the running total reaches p. The candidate that pushes the total past p is kept, since without it the threshold would not be met - which is why a small p can leave you with a single token. top_k always leaves the same number of options; top_p leaves a number that changes with how confident the model is at that step.
Why does the model still give different answers when I did not change any parameter?
Because the last step is a draw, not a decision. The three parameters only decide which candidates are still eligible and with what weights; the token that actually comes out is then sampled from that weighted set. Two runs with identical parameters therefore share the same candidate pool but can still land on different tokens. Press draw again here and watch the chain stay completely still while the result changes - that is the whole of the remaining randomness. Setting temperature to 0 collapses the pool to a single candidate, which is why it behaves deterministically.
My output is too rambling, or too repetitive. Which parameter should I change?
Change one at a time and look at what it does to the chain rather than reaching for a recommended number. Rambling usually means low-probability candidates are still eligible, so tightening the cut with a smaller top_p or a smaller top_k removes them without touching the shape of what remains. Repetitiveness usually means the pool has collapsed onto one or two dominant candidates, so a higher temperature is the lever that widens the gap back. A practical method: hold two of the three fixed, sweep the third from conservative to aggressive, and stop where the surviving set looks right for your task. There is no universally correct value, which is why this tool shows the mechanism instead of recommending settings.
Are these probabilities computed live? Will my own API call behave the same?
The starting distribution is constructed for demonstration and labelled as such, so no, it did not come from a model. Everything downstream of it is computed live in your browser and matches what a real sampler does. So the mechanism transfers to your own calls; the specific numbers do not. Your call uses a different prompt and possibly a different model, tokeniser and decider order, and providers may add steps this page does not model, such as repetition or frequency penalties. If you want a real distribution to experiment on, load a small model into the browser and the same chain will run on its actual output.
Which scenarios are included, and why these?
Three, each chosen so that a different decider becomes the one that matters. With one clear favourite, truncation has almost nothing to remove and temperature is the only knob with a visible effect. With several close rivals, a small move on top_p changes who survives, which is the clearest way to feel why the same question can get different answers. With a long tail, top_k is doing the heavy lifting, cutting away an entire tail rather than a few stragglers. Picking scenarios by shape rather than by topic is deliberate: the shape is what determines which parameter is worth turning, and that is the thing worth learning.