TL;DR
A search-based decoding method that generates high-quality text by maximizing the difference in probability distributions between a strong expert model and a weaker amateur model.
Contrastive decoding operates entirely at inference time by contrasting the token logits of two different models to refine the output distribution. The method leverages the observation that smaller models tend to exhibit predictable errors like repetition and semantic drift, which are also present in larger models but to a lesser degree. By subtracting the amateur model's probabilities from the expert's, the system downweights generic, off-topic tokens and upweights more creative, factual, and reasoning-dense options.
Why this matters for your business
It provides a training-free way to dramatically improve text coherence, reduce hallucinations, and enhance reasoning performance on downstream tasks using existing models.
