Why outcome probabilities > lawyer win rates
There are 16,198 barristers admitted to the bar in England. There are 10,933 lawyers admitted to the bar of the United States Court of Appeals for the First Circuit. How will you ever find the right one for your case?
Conventional methods such as asking friends, consulting legal directories or going to the law office next door are bad choices. Qualitative descriptions do not guarantee performance. You wouldn’t pick an investment fund based on hearsay — you pick it based on results.
Legal analytics offers a better metric: win rates. That’s a good start. A lawyer’s win rate is simply the amount of cases they win divided by the total amount of cases they argue. Win rates can be tailored to particular scenarios — for example, a lawyer’s win rate before a specific judge. Win rates give you a much better idea about a lawyer’s performance than what people think about them.
But there’s a better metric: outcome probability. This is the probability that a lawyer will achieve the desired result. This metric is not the same thing as a win rate. If a lawyer wins 100% of their cases, the outcome probability for a ‘win’ is not 100%. If a lawyer wins 10% of their cases, the probability for a ‘lose’ is not 90%.
Outcome probabilities are the result of a complicated assessment of every known feature of the case. What is the lower court? Who is opposing counsel? Who are the judges on the panel? What law schools did all these people go to? What is the subject of the case? How much money is involved? Win rates miss all of this crucial information because they only consider how many times a given lawyer wins.
It’s easy to show why outcome probability is a better metric than win rate: it responds to changes in the case. If your opposing counsel changes, the outcome probability changes. If the judge(s) change, the outcome probability changes. If the amount claimed changes, the outcome probability changes. Meanwhile, the win rate stays the same.
Outcome probabilities are much harder to calculate than win rates. It takes sophisticated AI to correlate all of the moving pieces involved in litigation and to evaluate their respective importance to the outcome. This difficulty, however, also opens up the possibility to truly understand which factors are driving the outcome. A win rate doesn’t tell you anything about what’s going on in the judge’s head.
At CourtQuant, we give our clients access to both win rates and outcome probabilities. The former give you a good idea of a lawyer’s general ability to win cases. But it’s the latter you want to know to pick the right lawyer and to craft the right strategy for a case that really matters.
Ludwig is the CTO at CourtQuant, a London-based AI startup that predicts litigation outcomes, selects the best lawyers, judges and courts for individual cases and offers custom solutions for insurance and claims management.
