Researchers find appointed justices outperform elected counterparts

State supreme court justices who don’t face voters are generally more effective than their elected counterparts, according to research led by Princeton University political scientists.

The research combines data about almost 6,000 state supreme court rulings nationwide between 1995 and 1998 with a new theoretical model to reach the conclusions that appointed justices generally bring a higher quality of information to the decision-making process, are more likely to change their preconceived opinions about a case, and are less likely to make errors than elected justices.

“Judges may be appointed to state supreme courts, elected in competitive elections or face retention elections. We wanted to see whether these selection methods can be associated with differences in the attributes of the judges themselves and with differences in the ways these judges interact with each other in the court,” said Matias Iaryczower, an assistant professor of politics at Princeton, who conducted the research with Princeton graduate student Garrett Lewis and Matthew Shum, a professor of economics at the California Institute of Technology.

According to the researchers, the information quality for justices who don’t face voters is on average 33 percent larger than that of justices who face retention elections at some point after being appointed and 39 percent larger than that of justices who are elected. That means justices who don’t face voters in general demonstrate a greater ability to analyze information about a case to reach a correct decision under the law.

Error rates are low overall, Iaryczower said, but differences based on how justices are selected are meaningful. Justices appointed for life and appointed justices with political reappointment on average have a lower probability of reaching an incorrect decision (0.1 percent) than both justices who face retention elections (0.5 percent) and justices who are elected (0.3 percent).

The work is detailed in an article published in January in the Journal of Public Economics.

“A longstanding question in economics and political sciences involves whether public officials should be elected or appointed. A theoretical literature has argued that elections may serve to discipline public officials but may also provide incentives for officials to inappropriately pander to shifts in public opinion,” said Brian Knight,  a professor of economics at Brown University and co-editor of the paper. “The research by Iaryczower, Lewis and Shum provides one of the first efforts to quantify these advantages and disadvantages of elections.”

In search of correct decisions

The researchers, who are interested in the differences between elected and appointed government officials, put their focus on justices of state supreme courts because these courts of last resort are similar institutions in each of the 50 states. But their members are selected in different ways — through appointment, election or retention elections. In addition, the courts are working to the same goal: determining the correct decisions under the law.

Data used in the research came primarily from the State Court Data Project, which provides a detailed compilation of data from state supreme court cases in all 50 states from 1995 through 1998. The data includes the particulars of each case, including how each justice ruled, and additional data on each of the 520 justices who served on one of the courts during that time period. The researchers focused on 5,958 criminal cases the courts ruled on in the period.

Information on each case and justice was coded by the researchers, who then applied a model that looked at how bias and information quality interact to shape each justice’s decisions.

“We can think of each judge as endowed with two key components for decision-making, which can vary depending on the characteristics of the case and the individual justice,” Iaryczower said. “The first is a bias parameter, representing the justice’s individual preferences (coming from ideology, a legal position, personal experiences, etc). The second is a parameter measuring the quality of the justice’s information: her ability to go from the facts of the case to a correct decision under the law.”

Iaryczower offered an example from another context to explain the ideas of bias and quality of information as they are applied in the research: “Consider two senior faculty members from different disciplines deciding a junior appointment. Both professors would support hiring a candidate who excels at research. Each faculty member, however, might be more inclined to hire within the faculty member’s own field. She might be ‘biased’ toward thinking that her own discipline is more relevant, or useful, for advancing science.

“On the other hand, both professors might differ in their ability to evaluate the candidates’ research potential based on their own understanding of the candidates’ writings and achievements. This is the ‘quality of their information’.”

In a similar way, the model created by the researchers attempts to measure the interaction of bias and information quality in decision-making by justices.

Estimating error rates

The researchers also used the model to estimate how often justices made errors in their rulings, meaning they reach an incorrect decision under the law.

“We cannot know on a case-by-case basis what is the right answer, but we can attach a probability to each decision being correct under the law, given the votes of all justices and the characteristics of the case,” Iaryczower said. “Once we have recovered the bias and quality of information of all justices in the court, we can compute theoretically the probability of making a mistake.”

The researchers also estimated the probability that judges vote differently than they would have in the absence of case-specific information. Their measure is the probability that a justice would decide a case differently than what she would decide it without case-specific information. This is captured in a “FLEX” score. The average FLEX score — on a scale from zero to one — was 0.37  for elected justices and 0.60 for justices appointed for life, reflecting that appointed justices are more willing to change their preconceived opinions about a case.

The research on state supreme courts is part of a set of projects Iaryczower and his collaborators are pursuing in a similar vein. They are also looking at the workings of deliberations in appeals courts and the impact of campaign contributions to decision-making in courts.

The research described in the Journal of Public Economics article was supported by grants from the National Science Foundation.