Min-p sampling for LLMs

SECTION 1

The sampling challenge in large language models

Min-p sampling is a text-generation technique published under the MIT license; an Adservio AI researcher is among the authors of the work that formalized it, and the firm was one of the first organizations to integrate it into a production client environment.

At the heart of every large language model lies a single task: predicting the next token from the sequence generated so far. Input text is converted into tokens and then into vector embeddings; at the output layer, those embeddings are converted back into probabilities over the entire vocabulary. The model never truly "chooses" a word, it produces a probability distribution, and an external mechanism decides which token to keep at each decoding step.

That mechanism is called sampling. If it systematically keeps the single most likely token, the output becomes safe but monotonous, predictable, sometimes repetitive. Conversely, picking at random across the entire vocabulary produces incoherent text. Between these two extremes lies a balance that generative models have chased since their earliest days: enough diversity to stay creative, enough discipline to stay coherent.

Min-p sampling offers a structurally different answer from historical approaches: instead of a fixed truncation threshold, it varies that threshold based on the model's confidence at each decoding step. The threshold becomes relative, it depends on how certain the probability distribution is for the current token, rather than on a constant set once and for all.

SECTION 2

Classic sampling techniques and their limits

To understand what Min-p brings to the table, it helps to first look at why established approaches remain insufficient across the diversity of LLM use cases in production.

### Greedy decoding, beam search and temperature

Greedy decoding and beam search are near-deterministic techniques that systematically select the most likely tokens at each step. They favor the highest-confidence choices, which makes them reliable for factual tasks but prevents them from exploring more original phrasing or alternative reasoning paths.

Temperature acts as a risk dial applied to the probability distribution before sampling: a low temperature makes the model cautious, a high temperature encourages it to explore less likely tokens. Used alone, it solves nothing, it amplifies or dampens the problem without ever adapting it to context.

### Top-k and Top-p (nucleus sampling)

Top-k sampling keeps the k most likely tokens at each step, regardless of the shape of the distribution. With a k that's too low, the model stays overly conservative and loses creativity; at high temperature, it becomes noisy and incoherent, because the threshold never adapts to the model's actual confidence level.

Top-p sampling, or nucleus sampling, dynamically selects the smallest set of tokens whose cumulative probability mass exceeds a threshold p fixed in advance. This is a genuine improvement over Top-k, but the threshold p itself stays fixed: at high temperature, the distribution flattens, many tokens end up with similar probabilities, and Top-p can retain a nucleus far larger than necessary, opening the door to repetitive or incoherent text.

### Dynamic thresholding: a first attempt at adaptation

Dynamic-thresholding approaches have tried to adjust the selection threshold based on the model's confidence, but they demand careful, often unstable tuning from one model to the next. The underlying problem remains: at high temperatures (T > 2), the probability distribution flattens to the point where many tokens become statistically equivalent, which can lead to degeneration, repetition, or even gibberish, even with properly tuned Top-p or Top-k.

@cite:comment-evaluer-un-systeme-llm

Assessing the real quality of a sampling strategy, beyond perplexity metrics alone, ties into broader questions about how to objectively measure the performance of an LLM system in production.

SECTION 3

Min-p: a truncation threshold that adapts to the model's confidence

Min-p starts from a simple principle: stop using fixed thresholds. It's a stochastic technique that varies its truncation threshold based on the model's confidence at each decoding step, making that threshold context-sensitive rather than a global parameter set once and for all.

Confidence is measured by the probability of the single most likely token in the distribution, denoted ρ_max. The effective truncation threshold is computed as a fraction of ρ_max, weighted by a user-set parameter p_base, typically between 0.05 and 0.1. Concretely, only tokens whose probability exceeds p_base multiplied by ρ_max are kept in the sampling pool. If ρ_max is high, the model is confident and the threshold becomes mechanically more conservative; if ρ_max is low, the model is torn between several tokens and the threshold drops to let more options through.

This behavior resolves the diversity-versus-quality trade-off that hampered earlier methods, including at high temperature, precisely where Top-k and Top-p degrade the most. By making the threshold relative to the distribution itself rather than to a constant, Min-p stays coherent whether the model is very confident or very uncertain.

SECTION 4

Technical implementation: Min-p as a logits processor

### Integration into the decoding pipeline

Min-p has been implemented as a logits processor, a modular component of the decoding pipeline that adjusts the raw scores produced by the model before the actual sampling step. After temperature scaling is applied, the threshold ρ_max multiplied by p_base is computed for the current step, and every token whose probability falls below that threshold is excluded before the draw. The operation is vectorized and runs in a handful of comparisons per step, with negligible computational overhead relative to the cost of the model's forward pass.

### Adoption across open-source frameworks

The reference implementation has been integrated into the most widely used inference frameworks of 2026: Hugging Face Transformers, vLLM and SGLang all expose a min_p parameter directly in their generation API, alongside temperature, top_k and top_p. Support has also extended to lightweight engines such as llama.cpp and to consumer-facing interfaces built on top of them, like Ollama or LM Studio, which has largely democratized the technique beyond research labs alone. The reference source code remains published as open source, which has facilitated this cross-framework adoption and the reproducibility of published results.

SECTION 5

Real-world use cases in production

Min-p's benefit is most visible everywhere coherence is expected under high temperature, a trade-off that historical methods struggle to hold.

### Diversifying reasoning paths

In brainstorming or problem-solving, generating several distinct reasoning paths increases the odds of landing on a good answer. Available evaluations show that Min-p used at high temperature can outperform greedy decoding on this type of task, achieving a better balance between diversity and accuracy than traditional deterministic methods.

### Training reinforcement-learning agents

Recent reinforcement-learning research relies on Min-p to generate diverse, high-quality training data for exploratory agents. One published configuration pairs a temperature of 1.5 with a p_base parameter of 0.3 for a model in the Llama 3.1 8B Instruct family, illustrating the technique's potential to improve exploration in reinforcement learning without degrading the quality of the generated trajectories.

### Red-teaming and advanced reasoning models

In application-level penetration testing, Min-p helps generate diverse attack samples that reveal vulnerabilities more deterministic methods would miss. A Min-p configuration is also recommended for implementations of reasoning models such as DeepSeek-R1: a p_base value of 0.05 helps discard very unlikely predictions, a setting particularly useful on heavily quantized variants where the output distribution is noisier.

@cite:demystifier-deepseek

SECTION 6

Current limits and research directions: toward Min-z and Top-Nσ

### Sensitivity to heavy-tailed distributions

Min-p is not free of limitations. Its threshold calculation relies on the value of the single most likely token, which makes it sensitive to skewed or heavy-tailed distributions: a handful of extreme logits can distort the threshold and lead to over-truncation or, conversely, under-truncation. Recent extensions such as Top-Nσ and Min-z explore complementary directions, notably normalization by the standard deviation of the distribution and median-centered thresholding rather than maximum-centered, to produce more robust truncation in the presence of heavy tails.

### Combining decoding with chain-of-thought

Work combining uncertainty with chain-of-thought decoding suggests a useful distinction: high-certainty tokens produce the most reliable final answers, while more diverse, lower-probability tokens enrich the intermediate reasoning process. This opens a natural direction for future research, varying the p_base parameter depending on whether the model is drafting a reasoning step or stating its conclusion.

### Extension to other generative domains

Most published validations focus on text generation. Extending Min-p to code generation, multimodal models, or the still largely unexplored very-high-temperature regimes could reveal further gains. Early results in reinforcement learning for trajectory generation are encouraging on this front.

SECTION 7

Security, governance and ethical considerations

A technique that increases the diversity of a model's output calls for particular vigilance. More fluent, more varied text can also make deceptive or harmful content more convincing, which means Min-p should be paired with content filtering and responsible implementation rather than deployed in isolation.

On the security front, raising temperature makes generation more random by construction. The concern that too much randomness could make a model "overshoot" its safety fine-tuning, that is, generate content it was normally trained to avoid, remains a hypothesis worth monitoring; no robust evidence to date shows that Min-p specifically worsens this risk compared with other high-temperature sampling methods.

@cite:il-faut-traiter-les-hallucinations-de-l-ia-comme

Transparency remains the best guarantee of reproducibility: the reference implementation of Min-p is published as open source, with detailed experimental configurations, allowing any team to verify results before deploying them in production. The benefits of entropy- and uncertainty-based methods clearly outweigh the risks identified so far, and safety and alignment research has every interest in continuing to exploit these signals: they directly benefit robustness, truthfulness, and hallucination reduction.

Disclaimer: the claims and opinions expressed in this article are those of the author(s) and do not necessarily reflect the views of Adservio.

FAQ

Frequently asked questions