DATA SCIENCE INITIATIVE

Although U.S. court documents are publicly available online, they sit behind expensive paywalls inside a difficult-to-navigate database.

A Northwestern University-led team says these barriers prevent the transparency needed to establish a fair and equal justice system. Making all court records open and available will allow researchers to systematically study and evaluate the U.S. justice system, yielding information with potential to direct policy.

“In principle, litigation is supposed to be open to the public,” said Northwestern data scientist Luís A. Nunes Amaral. “In reality, the lack of access to court records seemingly undercuts any claim that the courts are truly ‘open.’”

The new insights will be published on Friday, July 10 in the journal Science. Amaral is the corresponding author of the paper. His co-authors include computer and data scientists, legal scholars, journalists and policy experts.

Northwestern artificial intelligence (A.I) researcher Kristian Hammond and the C3 Lab are developing an A.I. platform that provides users with access to the information and insights hidden inside federal court records, regardless of their data and analytics skills.

“The problem with court data is the same problem as with a lot of datasets,” Hammond said. “The data cost money, and the technical skills to use them cost money. That means very few people have access — not just to the data — but to the information that we all need that’s hidden inside of it.”

We really can ask the broadest questions. The ultimate goal is to ask if the court system is acting fairly.

With this tool, the researchers can link courtroom data to other public data to explore questions such as: How do different judges affect the outcomes of similar cases? Does it make a difference to be defended by a big law firm compared to a smaller one? And how many cases settle?

“We really can ask the broadest questions,” Amaral said. “The ultimate goal is to ask if the court system is acting fairly.”

Amaral is the Erastus Otis Haven Professor of Chemical and Biological Engineering in Northwestern’s McCormick School of Engineering and the director of the Northwestern Institute on Complex Systems. Hammond is the Bill and Cathy Osborn Professor of Computer Science at McCormick and the director of Northwestern’s Master of Science in Artificial Intelligence program.

Northwestern co-authors include data scientist Adam Pah from the Kellogg School of Management; legal scholars David Schwartz, Sarath Sanga, Zachary Clopton and Peter DiCola from the Northwestern Pritzker School of Law and journalism researcher Rachel Davis Mersey from the Medill School of Journalism, Media, Integrated Marketing Communications.

Evaluating access to justice

To help quantify and evaluate citizens’ access to justice, the researchers examined judicial waiver decisions. Anyone who files a lawsuit in a federal court must pay a $400 filing fee, which is unaffordable for many Americans. To waive these fees, litigants can file an application. Because there is no uniform standard to reviewing these requests, the Northwestern team found judges’ decisions varied widely. In one federal district alone, judges approved waivers anywhere from less than 20% to more than 80% of the time.

“If all judges reviewed fee waiver applications under the same standard, then grant rates should not systematically differ within districts,” the authors wrote. “We find, however, that they do.

Being able to systematically analyze these records allows us to identify issues with equity.

“Being able to systematically analyze these records allows us to identify issues with equity,” said Pah, a clinical assistant professor of management and organizations at Kellogg. “While we don’t know what that rate should be, we do know that it should be consistent to be fair.”

The research team believes these types of variations can be fixed if the public can access and analyze court records, in order to give the justice system quantitative feedback. To do this, the researchers recommend a three-pronged approach:

1. Make court records free to dismantle the barrier to access;
2. Link courtroom data to external data — such as information on judges, litigants and lawyers — to build a collaborative knowledge network; and
3. Empower the public by providing access to the information that flows from the analysis of the federal court data.

Transforming study, journalistic coverage

To help with this approach, the researchers are developing SCALES-OKN (Systematic Content Analysis of Litigation Events Open Knowledge Network), an A.I.-powered platform that makes the federal courtroom data and insights available to the public. The team believes the tool has potential to transform the ways academics, scientists and researchers approach legal study, as well as how journalists cover the justice system.

“The information made available by SCALES will be relevant to scholars in a wide range of disciplines, including law, economics, political science, linguistics and others,” said Schwartz, professor of law and the associate dean of research and intellectual life at Northwestern Law.

“Our ability to understand and improve the law — everything from employment discrimination to intellectual property to securities regulation — depends critically on our ability to access legal data,” said Sanga, an associate professor at Northwestern Law. “By opening up court records, SCALES will finally enable researchers to systematically examine the court system and the practice of law. Social scientists will use this resource in much the same way that they use the U.S. Census. It will provide both a detailed and big picture view of the process by which litigants navigate the justice system, as well as the process by which judges administer justice.”

“SCALES will transform the way journalists are able to cover the American justice system,” said Mersey, associate dean of research at Medill. “The interface will allow reporters, both with and without data analytics skills, to quickly and easily access judicial information and court records to cover uses of social justice, equity and due process. At a time when media organizations have trimmed newsroom staffs and decreased the amount of money that can be spent gathering information, SCALES will prove to be a powerful partner in ensuring the justice systems operates in an open and accessible way.”

The paper, “How to build a more open justice system,” was supported by a gift from John and Leslie McQuown and by the National Science Foundation (award number 1937123).