Chamber Affiliate Recycles Flawed Methodology to Defend Forced Arbitration for Consumers
Just as wrong the second time around.
By Karla Gilbride
Cartwright-Baron Senior Attorney
In 2019, Congress held hearings on the FAIR Act, a bill that would make it illegal to force employment, consumer, antitrust and civil rights disputes into arbitration. The Institute for Legal Reform, an affiliate of the nation’s chief big business lobby – the U.S. Chamber of Commerce – tried to influence that legislative debate with a report claiming that arbitration is actually better for workers than going to court. That report was widely criticized for omitting key data and making misleading comparisons. And it didn’t stop the House from passing the FAIR Act.
Now the Chamber is at it again. In November 2020 the Institute for Legal Reform published a second report, this time comparing the courts and arbitration as forums for resolving consumer disputes. Not surprisingly, the conclusions were the same: consumers, like workers, supposedly do better in arbitration!
But the sequel is plagued with the same methodological problems as the first report, and Congress and other decisionmakers should give it just as little weight.
For one thing, like the first report, the consumer study only compares arbitration results to results obtained in federal courts. But the vast majority of civil cases are filed in state courts, because the Constitution and Congress impose limits on the sorts of cases that can be brought in federal court. Though the exact proportion varies from year to year, there are around fifty times more civil cases filed in state courts in a given year than in federal courts.
Not only are there more state court cases to begin with, but significantly more of them lead to victories for plaintiffs—many of whom are consumers. The Bureau of Justice Statistics, an office within the Department of Justice, found that nearly 60% of state-court trials led to verdicts for the plaintiff. During a similar time period in the early 2000s, only 35% of federal court cases resolved in plaintiffs’ favor. Thus, leaving state court cases out of the analysis, as the Chamber report does, is like comparing the population of the EU to the population of Canada and then positing that the results say something meaningful about living in Europe as opposed to North America.
And this wasn’t enough skewing of the sample for the report’s authors, who then decided—continuing the analogy—to remove all the cities in Canada from their population comparison. What they actually did was to remove all class actions from their federal court data set, something one would only learn by reading nearly to the bottom of the report’s Methodology section, on page 13 of 14. This is a rather glaring omission, considering a 2015 study from the Consumer Financial Protection Bureau, which found that just a subset of consumer class actions yielded $2.2 billion in cash payments to 34 million people, not including attorneys’ fees. And the monetary recoveries in class actions don’t account for some of their most important results: securing injunctive relief to force corporations to put an end to harmful practices, and deterring other companies from similar wrongdoing.
With all state court cases and class actions eliminated from the study, one might expect the number of arbitrations to dwarf the number of court cases remaining in the sample. One might especially expect this after reviewing research from Consumer Reports, which found that over two-thirds of its most popular reviewed products came with mandatory arbitration as a term of purchase. For consumers of these and many other products and services, bringing a dispute in court is not even an option.
But despite all of the data manipulations described above, court cases still outnumber arbitrations in the Chamber’s consumer sample by a factor of more than three to one. The report obscures this fact by stating many of its top-line conclusions in the form of percentages rather than whole numbers, noting that 57% of consumer disputes in arbitration settle compared to 85% of federal court disputes. Even expressed in percentages, federal court still comes out ahead, given that each settlement yields some amount of relief for consumers. But when expressed in actual numbers, the advantage of court over arbitration becomes much more pronounced: 14,024 arbitrations ended with the consumer claimants receiving some relief via settlement, compared to 65,038 federal cases that ended in settlement—or about 4.6 times more.
When it comes to the disputes that terminated with a decision by judge, jury or arbitrator, the Chamber’s penchant for using percentages and not numbers cuts in the other direction, making the difference in outcomes seem larger than it is. The report’s very first “key finding” touts a 44% win rate for consumers in arbitration compared to a 30% win rate in court, but buried much further down, in the fine print as it were, we find that 1,823 arbitrations and 1,417 federal court cases during the study period resulted in awards to consumers, a difference of just over 400 cases. Now imagine what those numbers would have looked like if the Chamber hadn’t excluded all of the federal class actions and consumer cases resolved in state court during the same six-year time frame!
None of this is to say that court is always superior to arbitration for every consumer in every instance. There may be disputes where arbitration really is the faster and better alternative, so long as it is freely chosen. If the Chamber wanted to commission a study to seriously examine the pros and cons of the two forums, so that consumers could make more informed decisions about when to pursue each one, that would have been a valuable contribution to the empirical literature. But that’s not what this report is.
Instead, by artificially shrinking the universe of court cases, ignoring the relief afforded to millions of consumers in class actions, and using percentages to exaggerate the perceived numerical advantage of arbitration, this document found a way to make the data fit the pre-ordained conclusion its sponsors wanted it to reach. The next time the FAIR Act comes up for discussion in Congress, the findings of this report should not be given any airtime. Just like its predecessor, the report’s glaring methodological flaws make it impossible to take seriously.
Image by NCinDC via Flickr.
