Better Rankings, Fewer Comparisons: Bandits for Ordering

This article was revised in October 2026 to describe the algorithms compere actually implements and to remove performance figures that had not been published.

The Ranking Problem

You have 1,000 items. You need to rank them from best to worst. The only way to assess quality is to show two items to a human evaluator and ask which one is better. Each comparison costs time and money. How many comparisons do you need?

A naive approach — compare every pair — requires nearly 500,000 comparisons. That is obviously impractical. A sorting algorithm like merge sort would need about 10,000 comparisons, which is better but still expensive if each comparison costs a dollar or takes five minutes of expert time. And that estimate assumes human judgements are perfectly consistent, which they are not: the same evaluator shown the same pair twice will sometimes answer differently, and two evaluators will often disagree.

This problem appears everywhere. Preference data for training reward models, A/B tests of designs or copy, leaderboards for games and products, annotation campaigns: in all of these the fundamental challenge is the same. Produce a useful ranking with as few comparisons as possible, given that comparisons are noisy.

This is the problem compere addresses.

Why Sorting Algorithms Fall Short

Traditional sorting algorithms assume two things that do not hold in practice. First, they assume comparisons are deterministic — that asking “is A better than B?” always yields the same answer. Human judgements violate this routinely, and a single inconsistent answer can send a comparison sort badly astray. Second, they treat every comparison as equally worth making, when in practice some pairs tell you far more than others: comparing a newcomer nobody has seen tells you more than comparing two items you have already compared twenty times.

These two observations point towards a different framework. We want an approach that tolerates noisy answers, that spends comparisons where they are most informative, and that produces a usable ranking at any point rather than only at the end.

Multi-armed bandits provide exactly this framework.

The Multi-Armed Bandit Connection

The multi-armed bandit (MAB) problem is a classic framework in sequential decision-making. You have several slot machines (arms), each with an unknown payout distribution. At each step you pull one arm and observe the reward. Your goal is to maximise total reward, which requires balancing exploration (trying arms to learn their payouts) with exploitation (pulling the arm you currently believe is best).

The connection to ranking becomes clear when you reframe the problem. Instead of slot machines, you have items. Instead of pulling an arm, you choose items to put into the next comparison. Instead of a payout, you observe which item wins. Exploration means giving under-sampled items a chance to show what they are; exploitation means concentrating comparisons on items that are winning.

This reframing turns ranking from a sorting problem into a sequential experimental design problem, which is what bandit algorithms are built for.

How compere Works

compere is deliberately simple. It uses exactly two well-known algorithms, one to decide what to ask and one to record what was learned.

Choosing the next pair: UCB1. For each entity, compere computes an Upper Confidence Bound score:

UCB(i) = win_rate(i) + c * sqrt(ln(N) / n_i)

Here win_rate(i) is the entity’s share of comparisons won, n_i is how many comparisons it has been in, N is the total number of comparisons, and c is an exploration constant (1.414 by default). The first term rewards entities that win; the second rewards entities that have not been compared much. Entities that have never been compared receive a large fixed priority (UCB_UNEXPLORED_WEIGHT, 1000 by default), so newcomers are brought into play quickly. The two highest-scoring entities are paired for the next comparison.

The practical effect follows from the formula. Early on, the exploration term dominates and comparisons spread across the population. As counts grow, the exploration bonus shrinks and comparisons concentrate on entities with high win rates — the contenders for the top of the table — while entities that lose consistently are asked about less often.

Recording the outcome: Elo. Once a comparison is submitted, both entities’ ratings are updated with the standard Elo rule:

expected_a   = 1 / (1 + 10^((rating_b - rating_a) / 400))
new_rating_a = rating_a + K * (actual_a - expected_a)

New entities start at 1500 and the default K is 32. An upset — a low-rated entity beating a high-rated one — moves both ratings more than an expected result does. The leaderboard is simply the entities sorted by rating.

An alternative pairing strategy. Alongside UCB1, compere exposes a similarity-based selector that pairs entities with similar ratings. That is useful when you want to resolve close calls rather than explore.

Using It

compere runs as a FastAPI service with interactive documentation at /docs, or as a Python library over SQLAlchemy. The service flow is the whole idea in four calls:

# Start the API server
compere --port 8090

# Create entities to rank
curl -X POST localhost:8090/entities/ -H "Content-Type: application/json" \
  -d '{"name": "Option A", "description": "First choice"}'

# Ask which pair to compare next (UCB1)
curl localhost:8090/mab/next_comparison

# Record the judge's choice, then read the leaderboard
curl -X POST localhost:8090/comparisons/ -H "Content-Type: application/json" \
  -d '{"entity1_id": 1, "entity2_id": 2, "selected_entity_id": 1}'
curl localhost:8090/ratings

The judge can be a person behind an annotation interface, an automated metric, or an LLM acting as evaluator. compere does not care where the answer comes from; it decides what to ask and keeps score.

Handling Noisy Judgements

Human comparisons are noisy, and it is worth being exact about how compere copes and how it does not.

Elo absorbs noise by design. A single surprising result moves ratings by a bounded amount, at most K points, and a run of consistent results outweighs an occasional reversal. Two evaluators who disagree on a pair simply contribute opposing updates. Nothing in the system breaks when the same pair gets different answers on different days.

What compere does not do is model the noise explicitly. It does not estimate per-judge reliability, it does not fit a Bradley-Terry or Thurstone model by maximum likelihood, and it does not report confidence intervals over positions. Elo also carries assumptions of its own: it presumes preferences are broadly transitive and follow a particular logistic noise model, so a population with strong cyclic preferences (A beats B, B beats C, C beats A) will produce ratings that summarise the cycle rather than represent it.

If you need calibrated uncertainty, or principled pooling across judges of different reliability, a Bayesian Bradley-Terry fit or TrueSkill is the right tool. compere is for the common case where you want a good leaderboard cheaply and can tolerate a point estimate.

How Many Comparisons Does It Save?

We have not published a measurement, and we would rather say so than quote one. The saving from adaptive pair selection depends on things no general figure can capture: how many items there are, how spread out their quality is, how noisy the judges are, and whether you care about the whole order or only the top.

The test is cheap to run on your own data. Take a sample of items, collect comparisons once with /mab/next_comparison choosing the pairs and once with uniformly random pairs, and plot how quickly each leaderboard stabilises — or, if you have a trusted reference order, how quickly each converges on it. That number, measured on your items with your judges, is the one that should drive the decision.

Applications

Preference data for reward models. Collecting human preferences for RLHF is pairwise by nature. Adaptive pair selection spends annotator time on comparisons that move the ratings rather than on pairs whose outcome is already obvious.

A/B testing. Designs, copy or features can be entered as entities and compared by users or reviewers, with the leaderboard updating as results arrive.

Leaderboards. Games, products and competitions where new entrants arrive over time benefit from UCB1’s priority for unexplored entities, which brings a newcomer into contention quickly.

Annotation tools. compere can sit behind a labelling interface that shows two items and records a choice, and a demo Vue.js interface ships in the repository.

Practical Considerations

compere is designed to be used by people who are not experts in bandit algorithms. The tuning surface is small and explicit: the exploration constant, the unexplored-entity weight, the Elo K-factor and the initial rating, all set through environment variables. Persistence is SQLite by default and PostgreSQL through DATABASE_URL; authentication and rate limiting are optional and off by default.

One property to understand before relying on it: Elo with a fixed K-factor weights recent results and lets ratings drift as outcomes change. That is helpful when the items themselves change over time, and unhelpful when you want a stable final answer from a fixed set of items — in which case it is worth looking at how much ratings are still moving before you stop collecting.

Why This Matters

Ranking is one of those problems that seems simple until you try to do it well at scale. The combination of noise and cost makes it a poor fit for classical sorting and a natural fit for sequential decision-making.

compere’s contribution is not a new algorithm. It is a small, inspectable combination of two old ones — UCB1 to decide what to ask, Elo to keep score — packaged so that a team can put it behind an annotation interface in an afternoon. Whether it saves enough comparisons to matter for your problem is an empirical question, and the experiment to answer it is cheap.