The More Popular, The Harder to Forget: Adaptive Popularity for LLM Unlearning

The Problem: Popular Facts Are Harder to Unlearn

Large language models memorize facts during training, and some facts stick harder than others. Popular facts—those appearing frequently in training data or widely known—embed themselves more deeply in the model's weights. When researchers try to make a model "forget" sensitive or outdated information, existing unlearning methods treat all facts equally, applying the same pressure to remove a celebrity's birthdate as a obscure historical detail. The result: popular facts leak back out, especially when users rephrase their queries.

How AdaPop Works

The authors' key insight is that unlearning should be adaptive—harder facts need harder pushing. AdaPop combines three ingredients:

Popularity-weighted gradients. The method estimates how "popular" each fact is using an external signal: either Wikidata sitelinks (a measure of a fact's notability across Wikipedia languages) or an LLM-as-Judge that rates importance. This popularity score becomes an exponent that amplifies the gradient updates for hard-to-forget facts. A common fact gets stronger pressure to be removed than a rare one.

Local token confidence. Rather than treating an entire fact uniformly, the method looks at the model's confidence on individual tokens within that fact. Tokens the model is very confident about (high probability) get higher-weighted updates, focusing effort where the model has strongest memorization.

Automated balance control. Unlearning creates a tension: push too hard to forget and the model breaks on things it should remember (the "retain set"). Push too soft and the forgotten facts leak back. Instead of manual tuning, AdaPop uses a dual-ascent controller that automatically adjusts the retain penalty each epoch, gradually finding the sweet spot.

Why It Matters

Across three model families and two benchmarks, AdaPop dramatically reduces information leakage. Under paraphrased queries (where users rephrase the same question), forgotten content leaks ~5 times less than competing methods. Even under adversarial reformulations—deliberately tricky rewording to trick the model—leakage is ~1.6 times lower.

The authors validate their approach with a telling internal metric: hidden states (the model's internal representations) for forgotten facts move much further away from their pre-unlearning values under AdaPop than under other methods. Simultaneously, representations for facts the model should retain stay close to the original, showing the method achieves genuine selective forgetting rather than degrading overall capability.

This matters because unlearning is increasingly critical: removing harmful stereotypes, respecting privacy, correcting outdated information, or complying with regulations like the right to be forgotten. As LLMs become production systems, the ability to selectively remove memorized content—especially the sticky, popular stuff—is no longer a curiosity but a requirement.