How Does It Work?

Overview: Rank Aggregation

An instance of a rank aggregation problem consists of a set of items and a set of rankings. A solution to a rank aggregation problem is a single aggregated ranking which is as "consistent" as possible with each of the original rankings.

An Example

On CityRank.ch our algorithm takes as input set of cities, like New York, Geneva, Chicago, and London and a set of indicators:

Test A	Test B	Test C	Test D
New York	Chicago	Chicago	Geneva
Geneva	London	New York	Chicago
Chicago	New York	Geneva
London

With this data as input, the algorithm produces an overall, aggregated ranking:

City

Chicago 100

New York 65

London 52.7

Geneva 0

Voting Algorithms

Rank aggregation is a version of voting in which the set of items to be ranked are the candidates and each voter expresses his or her preferences by creating a ranking. The objective is to pick either a single winner (in the case of a single open position) or a set of winners, usually ranked. This set of winners can be taken to be the aggregated ranking. There is a vast literature on voting algorithms known as social choice theory.

Scaled Data

Test A	Test B	Test C	Test D
New York 100	Chicago 100	Chicago 100	Geneva 100
Geneva 50	London 50	New York 50	Chicago 50
Chicago 37.5	New York -62.5	Geneva -125
London 25

Pairwise Input Data

	Chicago	Geneva	London	New York
Chicago	0	-54.2	-31.3	-50
Geneva	54.2	0	-25	112.5
London	31.3	25	0	-18.8
New York	50	-112.5	18.8	0

Pairwise Output Data

	Chicago	Geneva	London	New York
Chicago	0	-75.1	-35.5	-26.2
Geneva	75.1	0	39.6	48.8
London	35.5	-39.6	0	9.3
New York	26.2	-48.8	-9.3	0

Notes

Match cell colors to column or row title to determine preferences, i.e. if row A, column B is blue then city A (colored blue) is preferred. If row A, column B is red then city B (colored red) is preferred.

More information: the value in row A, column B (where A and B are cities) is equal to the arithmetic mean of the score differences of A and B (where the score difference is the value of B minus the value of A for each ranking in which both appear). A positive value in row A, column B indicates that when all of the rankings are averaged together, city B is preferred to city A.A negative value indicates the opposite.

To obtain the final, aggregated ranking, the values which gave the pairwise output data are rescaled linearly so that city ranked #1 has a score of 100 and the city ranked last has a score of 0. (A sensible alternative would be to use the scaling from earlier, where city #50 is ranked 50.)