A Ranking That Never Moves Is Either Finished or Abandoned

The date on a ranking is part of its claim

Most 'best of' pages on the internet are photographs. Someone assembled a set of products on a particular afternoon, ordered them according to a judgement they held that week, and published. The page then sits there while the category moves underneath it: products get reformulated, discontinued, undercut, superseded. Nothing on the page changes, so nothing signals that its claim has expired.

This is the failure mode of frozen rankings, and it is worse than being wrong on the day, because a stale list looks exactly like a current one. The reader has no way to distinguish a ranking that was carefully maintained from one that has not been touched since a previous formulation of its top entry was on sale.

Our lists are recomputed instead, which trades one problem for another: positions move, and a reader who bookmarked a page can return to find the order different. That churn is worth explaining rather than hiding, because each of the four things that cause it tells you something about the category. And it means a list's timestamp is not decoration — it is a claim about when the underlying assessment was last true.

A list is a filter, not a shortlist

The first structural thing to know is that our lists are not hand-assembled sets of favourites. Each one is defined by criteria — a category paired with a skin type, a condition, a price band or an ingredient — and every product in the catalogue meeting those criteria is a candidate for that list. ↗The serum ranking for oily skin is not a curated ten; it is the whole qualifying population, ordered.

That design has an immediate consequence. When the catalogue grows, lists change without anyone editing them. A product added in an unrelated batch that happens to satisfy an existing filter enters that pool automatically and takes whatever position its assessment earns. Nobody decided to promote it over an incumbent. It simply qualified, and the ordering is computed.

The alternative — a fixed shortlist maintained by hand — has an obvious flaw once a catalogue passes a certain size: it can only contain products someone remembered to consider. A filter has no memory and no favourites, and the population it selects is auditable. You can read the criteria and check that the products present are the products that belong.

The order within the pool comes from the assessment engine, and it is worth being specific about the shape rather than the numbers. Six factors carry fixed weights that sum to one: formula quality at 25%, safety and tolerability at 20%, real-world results at 20%, value at 15%, usability and feel at 10%, and brand trust at 10%. Those group into three chapters — the formula at 45% of the total, the experience at 30%, the deal at 25%. The weights are identical for every product in every category, which is what makes cross-product comparison mean anything. ↗A curated routine can express a point of view about which products belong together; a ranking should not express a private one about how to weigh them.

The cause nobody reports: the product changed

Of the four things that move a ranking, this is the one readers almost never hear about, and it is the most consequential.

Products get reformulated. Suppliers change, a preservative is dropped for regulatory reasons, a fragrance is reworked, a concentration is quietly adjusted, a supply chain substitutes an emulsifier. The name on the front stays the same. The packaging usually stays the same. The accumulated reviews — thousands of them, describing a formula that no longer exists — stay attached to the listing. From the outside almost nothing announces that the product in the box is not the product that earned its reputation.

The mechanism we use for this is deliberately blunt: every product carries a fingerprint that is a sha256 hash of its normalised ingredient list. Change the INCI and the hash changes; when the hash changes the product is reassessed as what it now is, and the change is recorded with a cause. That is also why normalisation is a serious piece of engineering rather than a formatting nicety — an inconsistently parsed comma can make an unchanged formula look reformulated, which is a false alarm that costs real trust.

The reader-facing consequence is that a long-standing recommendation can move down a list without anyone changing their mind about anything. When a staple like ↗CeraVe Moisturizing Cream, ↗Paula's Choice 2% BHA Liquid Exfoliant or ↗COSRX Advanced Snail 96 Mucin Power Essence shifts position, the first question worth asking is not whether the ranking got worse — it is whether the ingredient list did something. The Compare Products tool exists partly for this: putting two formulas side by side row by row is how you see a substitution that the marketing copy will never mention.

When the evidence moves, everything containing that ingredient moves

The third cause is upstream of any individual product. Scores are built partly on graded evidence for specific ingredient-and-claim pairs, and that grading is not frozen either. A systematic review lands. A large trial fails to replicate an earlier result. A regulator changes a permitted concentration. When the assessment of an ingredient's evidence for a claim changes, every product leaning on that ingredient for that claim moves together.

This produces the most disorienting kind of churn, because several products can shift in the same direction at once with nothing having changed about any of them. It also produces the most defensible kind, since the alternative is a ranking that keeps recommending something after the reason for recommending it has weakened.

The fourth cause is the engine itself. Methodology gets revised — a leaf that was too crude, a weight that was miscalibrated, a proxy replaced with something better measured — and a methodology release recomputes the whole catalogue in a single pass. That is not optional tidiness; a mixed state, where some products carry old scores and some carry new ones, would make every comparison between them meaningless. So lists record both the engine version and the grade-boundary version they were computed under, which is what lets a change in position be attributed rather than merely noticed.

Where judgement is still allowed, and where it isn't

None of this means a ranking is a machine with no humans in it. It means the human judgement is placed somewhere accountable.

Editorial judgement sets the criteria: what a list is for, which filters define its population, what the buying guide says a reader should weigh. A list also carries pins — an explicit mechanism for holding a product at a position for a stated reason. What matters is that pinning is visible and separate from the computed order, rather than a thumb on the scale disguised as a result.

What judgement is not allowed to do is move a score. Caps and gates that limit a product's score can only be entered through published criteria with a public record as their source, never editorially, and a gate that is disputed keeps its effect in force while the challenge is displayed. Once a product has a score, its position is arithmetic.

There is also a deliberate deadband, because a system that recomputes continuously will otherwise generate constant meaningless motion. A change is only recorded when it clears a threshold — two points, or one grade step — and every recorded change must carry a machine-readable reason naming the cause: an evidence-table change, a gate, a reformulation, a collection pass, or a methodology release. No score moves without an explanation on the record. That is the difference between a list that churns and a list that updates.

How to read a ranking that moves

Three practical habits follow from all of this.

First, read the date. A ranking's timestamp tells you when its claim was last computed, and on a recomputed list that is a meaningful piece of information rather than a formality. A list that has not moved in a year is either sitting in a genuinely static category or is not being maintained, and those look identical from the outside unless the page tells you which.

Second, treat position changes near the middle as noise and changes at the extremes as signal. Products clustered in the middle of a large pool are separated by small margins, and a two-point move reshuffles several of them without meaning much. A product leaving the top of ↗a moisturiser ranking altogether, or arriving there from nowhere, is the kind of movement worth reading the reason for.

Third, and most usefully: your own shortlist is not a ranking. A list orders a population against fixed weights that are identical for everyone, which is exactly what makes it comparable and exactly what makes it impersonal. It cannot know that you react to fragrance, that texture decides whether you will actually use something, or that you have three conditions competing for the same routine step. My Skin holds that context, and the sensible way to use a ranking is as the filtered candidate pool it is — then narrow it against the things about you that no weighted average is entitled to guess.