After setbacks, Spokane County abandons custom criminal justice algorithm in favor of simpler tool
Spokane County is adopting a new computer algorithm designed to help judges decide which defendants should remain in jail, abandoning a more costly, custom-developed program that was hampered by technical and logistical problems.
The Spokane Assessment for Evaluation of Risk, known as the SAFER tool, was touted as a cornerstone of local efforts to reduce jail overcrowding and eliminate racial disparities in the justice system.
Developed with help from a criminologist at Washington State University, the tool examined some 30 factors – such as criminal records, drug addiction and employment history – to produce a score indicating the likelihood a defendant would miss a court date or commit a new crime if released before trial.
But because of staff turnover, software glitches and the challenge of syncing the tool with state court data, the program never worked as intended. After nearly three years of testing and tinkering, officials have scrapped the SAFER tool in a favor of a simpler, off-the-shelf program called the Public Safety Assessment, or PSA, which will require less raw information.
Maggie Yates, who recently stepped in as Spokane County’s criminal justice administrator, said officials agreed they could not spend more time and resources developing the SAFER tool. The PSA is more “streamlined,” she said, and will finally enable the county to standardize how it evaluates defendants.
Although the jail population has not budged despite several years of grant-funded reform efforts, Yates said the new assessment tool could help the county stop incarcerating low-risk defendants only because they can’t afford to post bail.
What went wrong
In early 2016, shortly before Spokane County was awarded a $1.75 million grant from the MacArthur Foundation, the county and the city of Spokane jointly awarded a $70,000 contract to WSU Spokane criminologist Zach Hamilton.
An expert on predictive risk algorithms, Hamilton was tasked with creating the SAFER tool from scratch. It was supposed to be a highly sophisticated piece of software, capable of processing huge amounts of raw data and carefully tailored for defendants in Spokane County. In those ways, it’s very different from the PSA, Hamilton said.
“We created multiple models, we looked at males and females separately, selected a bunch of items that were collected by an interview, and utilized a lot of localized data that was collected since 2006,” he said. “And by using all of that data, we were hoping to make a tool that was more localized and, in a sense, more accurate for the Spokane population.”
But the same breadth of data that would have helped fine-tune the SAFER algorithm also made it cumbersome for the pretrial services team tasked with implementing it.
The plan had been to automate the program by connecting to a database of criminal records maintained by Washington’s Administrative Office of the Courts. A similar system is in place for juvenile offender data, but Hamilton said the courts office was reluctant to share adult offender data with individual counties.
That meant pretrial services staff had to go through a tedious, time-consuming process of interviewing defendants, researching criminal records and manually entering much of that information into the SAFER program.
Hamilton developed a scaled-down version of the tool, SAFER Lite, which was in use for several months last spring. But the pretrial services office had some vacant positions and newer staff members, and judges were reporting the risk scores weren’t as helpful as the evaluations the office had previously provided.
“The brick wall that we ran into was trying to get that last integration piece,” Hamilton said. “We had created the tool. We had put it into software. … But we never got to the point where we could fully integrate the tool utilizing the AOC’s software system.”
Hamilton said the state office did eventually approve the data transfers the county had been requesting, but that meant there could still be months of programming ahead, and officials decided to quit the project.
Yates said roughly $600,000 from the MacArthur grant was used to hire additional pretrial services staff to implement the SAFER tool and perform other new duties. Another $150,000 was budgeted for SAFER but only some of that was spent, she said.
Replacement tool
Developed by the Laura and John Arnold Foundation and available for free to any jurisdiction that wants to use it, the Public Safety Assessment is expected to be implemented in Spokane County in the latter half of 2019. It’s currently in use statewide in Kentucky, Arizona, New Jersey and Utah, and in counties in at least a dozen other states.
According to an explainer on the Arnold Foundation’s website, the PSA “examines nine factors related to a person’s age, current charge and criminal history and produces two risk scores: one that predicts risk of failure to appear for future court appearances, and a second that predicts risk of committing a new crime if released before trial.”
“The scores are based on a one-to-six scale, with higher scores indicating a higher level of risk. The PSA also flags defendants who indicate an elevated risk of committing a new violent crime if they are released before trial. The PSA results can help inform a judge’s decision-making; however, the final release decision always rests with the judge.”
Risk assessment algorithms have come under scrutiny amid evidence they can perpetuate racial biases already prevalent in the criminal justice system. But Yates said she’s optimistic about the PSA. She pointed to Yakima County, which began using the algorithm in early 2016 as part of a series of pretrial justice reforms.
“We virtually eliminated any racial disparity (in the pretrial jail population) after implementing the program, which came as a complete surprise to us,” said Yakima County Superior Court Judge Richard Bartheld.
Bartheld cited a 2017 analysis which found that Yakima County’s judges had granted pretrial release to about 64 percent of white defendants but only 49 percent of Hispanic defendants.
That margin shrunk after the PSA was implemented: Bartheld said the percentage of white defendants released before trial rose to 73 percent, while the share of Hispanic defendants rose to 75 percent. Hispanic people make up about half of Yakima County’s population.
Another sign of success: According to Bartheld, of all the defendants released before trial, the share who were released on their own recognizance rose from 36 percent to 85 percent after implementation of the PSA. That means significantly fewer people had to post bail and were allowed to walk free on the promise they would come back to court.
“We’re trying to get away from this mindset that if someone is just accused of a crime then we need to lock them up and throw away the key,” Bartheld said.