Top Condorcet.org

Ratings

I have attempted to justify that Ranked Pairs provides the best interpretation of a ranked ballot. That is, given that voters provide ranking of candidates, Ranked Pairs chooses the candidate who is most likely the best choice.

However, it can also be argued that plurality is the best method if only first preferences are known. That is, if you only know the first preference of voters, you can't do better than pick the candidate who receives the most votes. It is because plurality seems defective that I conclude that first choices are not enough to make a reasonable decision, and that with more information, a better method can be found.

But what if we had even more information than rankings? It is possible to construct a method where voters rates each candidate, for example from 0 to 100. This provides not only the order of preference, but the relative level of preference.

The typical way to use this information is to simply find the average rating for each candidate. The candidate with the highest average is then declared the winner. This method is not majoritarian, in that it is possible for a candidate to lose even if it is ranked first on a majority of ballots. But we shouldn't assume that all methods must be be based on majoritarianism. Instead, this method is based on the principle of utility.

The idea is that each person rates each candidate based on how much they like the outcome of that candidate winning. Then, the winner is the candidate who provides the greatest possible average satisfaction with the result. If the ratings of voters tend to reflect value for society, then the candidate with the highest average rating could be considered the best guess for most beneficial for society, or greatest utility.

The first serious problem with this has to do with strategy. It turns out that you are better off rating each candidate at the extremes. For example, with a scale of 0 to 100, you should rate each candidate as 0 or 100. In effect, this gives the same result as an approval election.

This presents a very serious practical problem with using ratings, or attempting to measure perceived utility. However, I think that the problem goes deeper. I would argue that Average Ratings wouldn't be a desirable method, even if it average ratings could be accurately read.

There are places where Average Ratings is appropriate. For example, in an essay contest, it makes sense to have a panel of judges who rate the essays. Each judge is assumed to rate the essays accurately, as opposed to rating them with the goal of maximizing his or her pleasure with the outcome. As well, the judges are assumed to be qualified, and have similar views and criteria for the rating.

If a group of people are trying to decide what to have for dinner, it may make sense to maximize the total utility of the outcome. One way this could be done is with average ratings. This brings up an important issue, however. Let's say that each person rates each meal from 0 to 100 based on their tastes. However, one person is allergic to one of the meals. To him, the difference in utility between that meal and the others is greater than the difference in utility for the other peoples' tastes.

So, if we want true utility, a 0 to 100 scale is inappropriate. It should really be an open-ended scale. This brings up some additional problems. Even if the ratings are sincere, some people might exaggerate the importance of the difference between outcomes.

Strength of Opinion and Reliability

If a voter prefers candidate A to candidate B, this is all that concerns Ranked Pairs. Ranked Pairs is not interested in how much the voter prefers candidate A to B. The idea is that the more people show the preference, the more likely it is to be true, the more show the opposite, the more likely false. In average ratings the principle is that a more accurate result can be obtained by asking people to assess how strongly they believe the statement, and using this fact to judge how likely they are to be true.

In fact, this is a rather problematic notion. There is something rather peculiar about asking people to self-assess the likelihood of their beliefs being true. After all, people who make bad decisions about how to rank the alternatives are also likely to make poor assessments of how likely their beliefs are to be true.

Consider that a person who tends to see things in extremes, and therefore rates candidates toward the top or bottom of the scale may influence the race between certain candidates many times more powerfully than people who are more measured in their opinions.

Top Condorcet.org