When he first approached the Harvard Legal Aid Bureau, an organization staffed by law students, Greiner made his pitch this way: Given that the number of people who come to HLAB looking for help far exceeds the number they can offer representation, why not randomize the sorting process, create a control group, and try to learn something about the organization’s impact? Initially, Greiner proposed running the study using either domestic abuse cases or eviction cases.

The answer from HLAB’s leaders was a firm no. “We pretty quickly decided we weren’t willing to do that,” said faculty director David Grossman, explaining that the student volunteers simply had too much confidence that their sorting methods were working to jettison them in favor of a random number generator. These methods, which took into account not only the likelihood that HLAB could make a difference in a particular case, but also the urgency of the case and the number of people affected by its outcome, were based on years of experience and institutional knowledge. Given the high stakes in eviction and domestic abuse cases, Grossman said, “randomizing away” clients HLAB would take under normal circumstances felt deeply immoral.

HLAB’s reluctance pointed to one of the main hurdles Greiner is facing as he looks for more legal services providers to work with on experiments: For an organization to do what he’s asking involves an admission that what they do normally might be ineffective. “Given our experience, we feel like we can pick the cases where we can make a difference,” said Greater Boston Legal Services executive director Jacquelynne Bowman. “But if it’s randomized, then we may miss out on providing representation to someone who truly needs it.”

Convincing professionals to doubt their methods is never easy, said Greiner, but it might be especially hard in the world of legal services. “Is it surprising that not every single legal aid attorney possesses the kind of self-doubt that might cause her to desire to undergo a rigorous evaluation of her...activities?” he asked. Nevertheless, Greiner believes this kind of self-doubt is important, precisely because people in the trenches can’t always look objectively at what they’re doing. “I don’t deny that...experiential observation is worthwhile,” he said. “But my job is to question and test it, because it turns out that human beings sometimes stink at figuring out what works and what doesn’t from day-to-day observations.”

***

The Harvard Legal Aid Bureau ultimately agreed to work with Greiner when he proposed running a study on people seeking unemployment benefits. This was an area of their practice they were already unsure of, said Tim Visser, the group’s president, with low enough stakes that the moral issues were not prohibitive. According to Visser, the students saw potential in Greiner’s study. In theory, it promised to reveal patterns that would tell them what to look for in clients.

Greiner’s surprising results, which were later published in the Yale Law Journal, indicated that people who were offered legal help from HLAB were no more likely to prevail in their cases—and worse, had to wait longer for their cases to be resolved—than those who were not. “That right there was a revelation,” said Greiner. More than 90 percent of people offered HLAB’s help received it; among those turned away, at least 39 percent found representation elsewhere. The fact that both groups scored a win rate of around 70 percent meant that getting help from HLAB was not increasing people’s chances as much as had been assumed.