The New Legal Empiricism & Its Application to Access-to-Justice

doi:10.1162/DAED_a_00536

The New Legal Empiricism &

Its Application to Access-to-Justice Inquiries

D. James Greiner

Abstract: The United States legal profession routinely deals with evidence in and out of courtrooms, but

the profession is not evidence-based in a scientiﬁc sense. Lawyers, judges, and court administrators make

decisions determining the lives of individuals and families by relying on gut intuition and instinct, not

on rigorous evidence. Achieving access to justice requires employing a new legal empiricism. It starts with

sharply deﬁned research questions that are truly empirical. Disinterested investigators deploy established

techniques chosen to ﬁt the nature of those research questions, following established rules of research ethics

and research integrity. New legal empiricists will follow the evidence where it leads, even when that is to

unpopular conclusions challenging conventional legal thinking and practice.

The U.S. legal profession routinely deals with ev-

idence in and out of courtrooms, but the profes-

sion is not evidence-based in the scientiﬁc sense.

Lawyers, judges, and court administrators, as they

work in the U.S. justice system, make decisions

that determine the lives of individuals and fami-

lies. But they overwhelmingly rely on gut intuition

and instinct, not on rigorous evidence. The choice

to eschew evidence matters. The profession is ask-

ing the wrong questions about topics fundamental

to our system of government, and “if they can get

you asking the wrong questions, they don’t have to

worry about answers.”

Here are two examples:

Judges decide which arrestees to incarcerate ver-

sus who to release pending disposition of criminal

cases in so-called bail hearings. They frequently fo-

cus on information available only from an interview

of the arrestee, such as how long they have lived in

the community, their employment situation, and

the presence of any family in the area. But arrest-

ee interviews raise concerns of self-incrimination,

d. james greiner is the Hon-

orable S. William Green Profes-

sor of Public Law and Faculty Di-

rector of the Access to Justice Lab

(

a2j) at Harvard Law School. He

has published in such journals as

Harvard Law Review, The Yale Law

Journal, and Quarterly Journal of

Political Science.

148 (1) Winter 2019 65

D. James

Greiner

and are often unavailable because arrest-

ees cannot be or do not choose to be in-

terviewed. Scientiﬁc evidence suggests

that if complete or nearly complete in-

formation on an arrestee’s past criminal

history (if any) is available–information

from existing administrative records–

an interview with the arrestee is unnec-

essary and provides little additional rele-

vant information.

In some jurisdictions, most low-income

survivors of domestic violence petition-

ing for civil protection orders will not re-

ceive full lawyer representation. Over-

subscribed legal-services providers often

decide to provide a few survivors/peti-

tioners full representation and to provide

something less (such as self-help materi-

als and/or an explanatory telephone call)

to most. But in making these triage deci-

sions, lawyers often ask themselves which

cases they can win (meaning obtain a civ-

il protection order), not in which cases

their representation is likely to make a

difference.

Winning a case and making

a difference in a case are two different

things. There is little evidence about how

to identify cases in which full representa-

tion makes a dif ference.

These examples illustrate that the ques-

tions the legal profession chooses to ask

about services it provides to poor and

low-income people have substantial con-

sequences about who spends time incar-

cerated and who obtains legal protec-

tion from abusers. In other words, these

choices have immediate consequences

for real people.

The new legal empiricism, which exists

in pockets in the academy but only rare-

ly outside of it, could transform the U.S.

legal profession into an evidence-based

ﬁeld. As the examples above indicate, ra-

tionalizing those areas of law might re-

duce crime and incarceration and permit

more effective triage decisions by legal-

services providers. In other words, fewer

people could go to jail, and those who do

go might spend less time there, with no

increase in crime rates or threat to the ad-

ministration of justice. More survivors

of domestic violence could have judicial

protection orders. None of this would re-

quire additional resources.

As of the 1930s, U.S. medicine was not

yet a science either. Physicians relied al-

most exclusively on gut instinct and intu-

ition, informed by dubious claims from

drug companies, to decide which drugs

to provide patients. Over the next forty

years, medicine transformed itself into a

ﬁeld in which physicians made decisions

about treatment based on evidence from

randomized controlled trials (

rcts).

The

U.S. population lives longer and health-

ier lives in part because of this transfor-

mation. The emergence of the new le-

gal empiricism, one hopes, indicates that

eighty-plus years later, law may begin to

follow medicine’s example.

What is the new legal empiricism? The

“new” here modiﬁes “legal,” not “empir-

icism.” New legal empiricism is simply

strong empiricism, as developed and im-

plemented over the past decades in ﬁelds

outside of law, now ﬁnally applied to law.

It is “new” in the sense that law has fol-

lowed neither medicine nor other social

sciences (sociology, economics, political

science, psychology) in demanding that

strong empiricism become the standard

for investigations that researchers con-

duct and the basis for decisions that those

in the ﬁeld make.

There are many kinds of empiricism,

and empirical projects can further a va-

riety of goals. Historians engage in em-

piricism, frequently based on archival

records and oral interviews with the el-

derly, and have their own standards of in-

ference. Literary scholars sometimes in-

vestigate the lives of the authors whose

work they interpret. The new legal em-

66 Dædalus, the Journal of the American Academy of Arts & Sciences

The New Legal

Empiricism &

Its Application

to Access-to -

Justice Inquiries

piricism involves investigations into how

the current legal system works, and how

to change the world for the better, how-

ever “better” is deﬁned.

The ﬁrst step in strong empiricism is

sharply deﬁning the questions to address.

The legal profession frequently fails here,

and when it does, nothing that follows

matters. Relevant questions emerge from

conversation among empiricists, prac-

titioners in a ﬁeld, and their customers

or clients. In the “bail” release example

above, a judge or a legislator might con-

sult an empiricist for help in easing jail

overcrowding or ameliorating racial dis-

parities in incarceration. The empiricist

might examine jail rosters to discover

that a substantial portion of jail residents

are arrestees awaiting trial on the charges

leveled against them, examine whether

some racial groups are over- or under-

represented, and ask how decisions are

made about whom to incarcerate before

trial.

Those questions might lead to the fol-

lowing information: In practice, after a

law enforcement ofﬁcer arrests some-

one, the arrestee is taken to a jail and held

there while a prosecutor ﬁles charges

against him. In most U.S. jurisdictions,

the arrestee is brought before a judge (or

the equivalent) who decides whether to

release the arrestee as he awaits trial on

those charges, and if so, what conditions

the arrestee will have to meet to secure re-

lease. As a matter of broad policy, judges

could release all arrestees, but doing so

carries risks that some arrestees will fail

to return for hearings or will commit new

crimes while on release, either of which

undermines the administration of the

case and the public’s conﬁdence in the ju-

dicial system. Judges could release no ar-

restees, meaning incarcerate them all, but

that carries ﬁscal and human costs. Judg-

es in all U.S. jurisdictions, following dic-

tates of statutes and court decisions that

vary in the degree of discretion delegated

to judges and in the factors judges can or

must consider, release some arrestees and

not others, imposing conditions on all ar-

restees released. Bail is one well-known

condition: the arrestee must arrange for

the deposit of a certain amount of money,

set by a judge, into a court account, with

the money returned at the end of the case

if the arrestee meets the conditions of re-

lease, but subject to forfeit to the court if

the arrestee misbehaves.

My research suggests that judges mak-

ing predisposition release decisions are

attempting to minimize three rates: 1)

the rate at which arrestees fail to appear at

court hearings in their criminal cases; 2)

the rate at which arrestees commit crimes

(especially violent crimes) between ar-

rest and case disposition; and 3) the rate

at which arrestees are incarcerated. Judg-

es and legislators are also often concerned

about potentially unlawful racial dispari-

ties in the population of arrestees incar-

cerated at the bail stage and about wheth-

er criminal records or other risk factors

justify racial disparities observed.

With that in mind, the empiricist and

the judge or legislator might agree that the

broad question is how to rank groups of

arrestees according to the objective, un-

biased risk that they will commit crimes

or fail to appear if released. High-risk ar-

restees could be incarcerated, the remain-

der released. The empiricist might break

this broader question into two: What ob-

servable factors classify arrestees accord-

ing to their risk of committing crimes or

failing to appear, without introducing ra-

cial bias into the bail decision? And does

providing some kind of compilation or

summary of those factors, for example,

in the form of a score that classiﬁes risk,

result in reductions of some or all of the

rates listed above?

To an empiricist, these last questions

are of different types. The ﬁrst, about

148 (1) Winter 2019 67

D. James

Greiner

factors that classify (or predict) arrestee

risk of misbehavior, gets at the way the

world is. It is descriptive. The second,

about whether providing information on

risk to judges improves criminal justice

outcomes, is causal. The causal question

gets at whether doing something (pro-

viding risk information to judges) ver-

sus doing something else (not providing

risk information to judges) alters a set of

outcomes (the three rates above) in a de-

sirable way.

Once the empirical questions to be ex-

plored have been identiﬁed, the empiri-

cist must determine how to answer them.

There will always be several options: qual-

itative techniques, such as structured inter-

views of potentially knowledgeable peo-

ple; focus groups, in which an empiricist

assembles groups of potentially knowl-

edgeable people, provides them with

open-ended questions, and elicits infor-

mation from the resulting discussion;

structured observation of relevant events; or

reviewing relevant documents to look for pat-

terns. Still other options include: quantita-

tive techniques, such as surveys of random-

ly selected individuals or cases or judges;

predictive models, which explore whether

and how well precursor variables predict

the value of ultimate variables; and ran-

domized controlled trials, in which the em-

piricist randomly assigns cases, people,

judges, or units of some kind to one condi-

tion or another and then measures which

condition produces a more desirable set

of outcomes. A practitioner of the new le-

gal empiricism, like a practitioner of rig-

orous empiricism outside the legal con-

text, chooses the method appropriate for

the questions to be addressed.

To illustrate this second step, return

to the two smaller empirical questions

identiﬁed above: What factors accurate-

ly predict arrestees’ risk of misbehavior

if released on bail? And does providing

information to judges about those factors

result in better reduction of the rates of

failure to appear at hearings, new crimes

committed between arrest and case dis-

position, and incarceration?

The empiricist will choose qualitative or

quantitative techniques to address these

questions. Which ones? In many situa-

tions, qualitative techniques are either su-

perior to other options or an integral part

of an overall research plan. For these two

questions, however, I would look primar-

ily to quantitative techniques. One reason

is the objectivity of the information like-

ly to be obtained by interviewing partici-

pants in the bail hearings. The goal of the

empirical project is to improve the judg-

es’ decision-making in these hearings, so

there is reason to be cautious about rely-

ing exclusively on talking to people and

observing the settings that are the target

of improvement efforts.

Instead, researchers exploring this

question have compiled data potentially

available to judges making arrestee re-

lease decisions, such as information about

charges, arrestee criminal history, their

history of appearing or not at past hear-

ings, their ties to the community, their

race or ethnicity, and other factors. Re-

searchers have connected this informa-

tion to information about key outcomes,

such as judges’ release decisions, arrest-

ees’ failure to appear, and arrestees’ com-

mission of new crimes. Applying statisti-

cal techniques to these data, researchers

have created scoring systems or algo-

rithms relating observable information

about arrestees to release decisions, fail-

ures to appear, and new crimes.

The scor-

ing systems or algorithms are known as

“risk assessment” instruments or scores.

They aim to use variables that are observ-

able at release hearings and are racially

unbiased, along with a set of weights de-

rived from the data analysis exercise just

described, to classify arrestees according

68 Dædalus, the Journal of the American Academy of Arts & Sciences

The New Legal

Empiricism &

Its Application

to Access-to -

Justice Inquiries

to the risk that they will fail to appear or

commit new crimes if released. This is

the new legal empiricism at work.

The new legal empiricism is about more

than selecting and implementing the right

research techniques to obtain and ana-

lyze the right data; it is also about creat-

ing research norms that protect the credi-

bility of the research and researcher.

New legal empiricism practitioners

must follow general norms of social sci-

ence research that have emerged to safe-

guard research integrity and the appear-

ance of research integrity. For example,

empiricists should not engage in investi-

gations in which they have a ﬁnancial or

other interest that might affect their im-

partiality. Independent researchers ordi-

narily produce more credible results. Re-

searchers must also follow the evidence.

If the strong empirical techniques sug-

gest or lead to truths that are unpopular

among certain constituencies, or even

normatively creepy, the empiricist must

report those results.

New legal empiricism practitioners

should stay cognizant of the limits of re-

search techniques, and remember to in-

clude a discussion of those limits in the

publications they produce. Where possi-

ble, empiricists specify their methods and

research goals before they start doing any

research, for example, by “registering” a

proposed study on one of several websites

that exist for this purpose. Where possi-

ble, consistent with conﬁdentiality- and

use-agreement limits frequently ground-

ed in concerns of ethics and privacy, em-

piricists make their data and statistical

code publicly available to allow replica-

tion of results.

New legal empiricism practitioners

should also separate, to the extent possi-

ble, the facts they investigate and gener-

ate from value judgments required in any

policy decision. Returning to the example

of risk assessment instruments, recall

that the aim was to minimize simultane-

ously the rates at which arrestees were in-

carcerated, failed to appear at court hear-

ings, and committed new (and especially

violent) crimes. Minimizing these rates

might involve tradeoffs: more incarcer-

ation might mean few failures to appear

and less crime. How to weigh these rates

against one another in setting policy in

this area is a value judgment, not an em-

pirical question. Though 100 percent

separation between empiricism and val-

ues is neither possible nor desirable, a

fair amount of distance between them is

achievable and essential to the empiri-

cist’s credibility.

Returning to the running example of

risk assessment instruments for bail hear-

ings, assume that the initial step of con-

structing a risk assessment instrument is

complete, so that an empiricist has iden-

tiﬁed a set of variables observable after

arrest and not too closely associated with

race that, weighted in a speciﬁed way, ap-

pears to classify arrestees according to

risk of misbehavior. Assume that the re-

searcher followed best practices with re-

spect to research integrity.

The next question is whether sharing

the risk assessment scores with judges

will change their behavior, by facilitat-

ing release decisions that minimize ra-

cial imbalances along with rates of in-

carceration, failure to appear, and/or

new criminal activity. This question is

one of program effectiveness, suggest-

ing that the backbone of research should

be one or more

rcts, because answer-

ing the research question requires being

able to compare judicial decisions (and

results of those decisions) when the risk

assessment instrument is available, and

when it is not, to see which produces

better outcomes. Only

rcts, with their

random allocation of judicial decisions

to a risk-assessment condition versus a

148 (1) Winter 2019 69

D. James

Greiner

no-risk-assessment condition, can assure

(up to statistical uncertainty) that differ-

ences observed in the outcomes are due

to the difference in conditions and not

some alternative factor.

In this example, without an

rct, a

judge might choose to use a risk assess-

ment only in cases the judge considers

close or difﬁcult. If so, and if defendants

misbehave at higher rates in the cases

the judge considers close or difﬁcult, a

comparison of misbehavior rates in cas-

es with risk assessment scores to misbe-

havior rates in cases without such scores

will show more misbehavior when risk

assessment scores are considered, mak-

ing it look like the risk assessment leads

to worse outcomes. An

rct would pre-

vent this kind of misrepresentation.

To answer a research question such as

this,

rcts, though necessary, are not al-

ways sufﬁcient.

rcts will disclose wheth-

er the availability of the risk assessment

improves criminal justice outcomes but

not why it does or does not. To ﬁnd out

why, a practitioner of the new legal em-

piricism should attempt to supplement

the

rcts with other quantitative tech-

niques, such as comparisons of rates of

release, failures to appear, new criminal

activity, and racial statistics before and

after the risk assessment was adopted. A

new legal empiricism practitioner should

also deploy court observations, inter-

views, and other qualitative techniques.

These techniques will generate informa-

tion about how risk assessment scores

work with judges’ decision-making, and

thus why scores do or do not help. That

information, in turn, will allow a re-

searcher to theorize about when scores

might work in other court systems.

Now imagine that several researchers

were at work on this question and they all

found the same thing: When they com-

pared release patterns, failures to appear,

new criminal activity, and racial balances

before and after the implementation of

the risk assessment program, they dis-

covered improvements in some or all of

these measures roughly coinciding with

the implementation of a risk assessment

score program. At the same time, howev-

er,

rcts showed no effect of the use of the

scores on outcomes. Interviews with judg-

es, prosecutors, defense attorneys, local

government ofﬁcials, and court admin-

istrators showed that communities ordi-

narily implement risk assessment scores

as part of an overall package of criminal

justice reforms. Such reform packages in-

clude reduction of time spent in jail before

bail hearings, faster processing of infor-

mation from law enforcement to prosecu-

tors, and/or creation of programs provid-

ing differing levels of monitoring (ankle

bracelets, drug testing, automated call-

in services) that facilitate predisposition

release. In the face of 1) a favorable be-

fore-after comparison, but 2) nothing sta-

tistically signiﬁcant on the

rct compari-

son that evaluated the risk assessment in-

strument exclusively, a practitioner of the

new legal empiricism might infer that the

elements of the reform packages adopted

contemporaneously with the risk assess-

ment are probably responsible for the fa-

vorable changes, not the risk assessment

itself.

The new legal empiricism means begin-

ning with a speciﬁc set of questions. The

questions to be investigated are not value

judgments masquerading as factual inqui-

ries; they are empirical. The investigation

proceeds with an impartial investigator’s

deployment of established techniques

chosen to ﬁt the nature of the research

questions. The investigator implements

these techniques in a manner that pro-

tects the integrity of the investigation

and her own neutrality. Helpful practic-

es include prespeciﬁcation, transparen-

cy, making data and coding available for

70 Dædalus, the Journal of the American Academy of Arts & Sciences

The New Legal

Empiricism &

Its Application

to Access-to -

Justice Inquiries

replication, and deﬁning variables clearly.

The investigator follows the evidence

where it leads, including to unpopular

conclusions, and she is careful to explain

the limits of the techniques she deploys.

All of this is new only to law.

The new legal empiricism has a great

deal to offer to the ﬁeld of access to jus-

tice. The ﬁeld misses out when research-

ers and their partners in the U.S. legal

profession choose not to render the re-

sults of their research credible.

Consider a question at the heart of ac-

cess to justice: how much difference, if

any, do different levels of legal assistance

make? This broad question can be ad-

dressed by identifying a particular level

of service–for example, offering self-help

materials to individuals faced with a cer-

tain kind of dispute–and identifying an

alternative level of service–for example,

offering attorney-client relationships to

individuals faced with that same kind of

dispute. Having identiﬁed differing ser-

vice levels, a researcher attempts to ascer-

tain how much each level costs as well as

the outcome variables the services are de-

signed to affect. Examples of typical out-

come variables include obtaining a favor-

able adjudicatory result, addressing the

underlying socioeconomic issue that led

to the adjudication, assuring that each lit-

igant feels that she is treated with digni-

ty and respect during the adjudication,

and reminding government ofﬁcials (the

judge, the court staff) that human be-

ings are involved in each of the cases they

adjudicate.

In the past, many researchers would

have proceeded by identifying a set of

cases in which a litigant experienced one

service level (say, self-help materials)

and a set of cases in which a litigant ex-

perienced a different service level (say,

full attorney representation), comparing

the outcomes of litigants in each set, and

then attributing any observed disparities

in those outcomes to the difference in ser-

vice levels. Sometimes, such researchers

could measure observable background

variables, such as race or gender or some

measure of case complexity, and attempt

to use statistical models to “control for”

those background variables.

By contrast, a researcher working in

the new legal empiricism proceeds by de-

ploying an

rct, supplemented (ideally)

by qualitative techniques, to understand

how the adjudicatory system at issue

functions. Only an

rct can assure, up to

statistical uncertainty, that any differenc-

es observed on the outcome variables ex-

perienced by litigants with one service

level (self-help materials) versus those

experienced in another service level (full

representation) are due really to the dif-

ference in service level offered as op-

posed to, say, differences in the individu-

als’ unobservable characteristics, such as

motivation level, articulateness, or case

characteristics.

This difference in methodology goes

to the heart of what the new legal empir-

icism is and what it can offer. Practicing

lawyers say that litigants who self-select

into receiving self-help materials and lit-

igants who obtain full representation do

not have the same distribution of motiva-

tion, or articulateness, or case character-

istics. Speciﬁcally, people who success-

fully search for a legal-services provider,

ﬁnd their way through its intake system,

and persuade it to provide full represen-

tation–all sufﬁciently early in a matter

for the attorney to provide real services–

are likely more motivated, more artic-

ulate, and have cases that have different

characteristics than those who do not. A

new legal empiricism researcher’s use of

rct-backboned study provides cred-

ible measurements of how much differ-

ence the disparate service levels make,

unconfounded by the effects of the dispa-

rate background variables. The new legal

148 (1) Winter 2019 71

D. James

Greiner

empiricism offers credible evidence of

the effectiveness, and cost effectiveness,

of disparate legal-service levels. Other re-

search offers little or nothing credible.

The difference in choice of method is

real. A few years ago, statistician Cassan-

dra Wolos Pattanayak and I compiled over

one hundred studies comparing the effec-

tiveness, and cost effectiveness, of legal-

services providers’ disparate levels of ser-

vice.

This quantity of studies addressing

the same research question demonstrates

both the question’s importance and the

vastness of the effort dedicated to answer-

ing it. But of the more than one hundred

studies, only about seven (depending on

how one counts) deployed

rct-backbone

methodology. These seven studies reach

seemingly contradictory conclusions, with

some suggesting the politically unpopu-

lar conclusion that for some legal settings,

and some sets of clients, higher (and more

expensive) levels of legal services make

little difference vis-à-vis lower, less ex-

pensive levels. The other studies are, as

detailed above, not credible. The poten-

tial of, and need for, the new legal empiri-

cism is evident.

Nonetheless, legal-services providers

continue to produce (and, apparently,

rely on) studies that do not deserve cre-

dence. Many of these are implemented by

or commissioned by the programs them-

selves. They almost invariably reach lau-

datory conclusions. By way of example,

a recent study of telephone advice pro-

grams in one state concluded that the

evaluated programs “are achieving the

primary goal of telephone-based legal as-

sistance, which is to make legal assistance

accessible to every eligible person . . .

without sacriﬁcing service quality and ef-

fectiveness in the process.” The research-

ers arrived at this conclusion by speaking

to former clients they could reach and re-

quiring the evaluated programs to con-

duct a “self-assessment.”

The evaluators

spoke only to individuals who received

telephone services (and who they could

reach), so there was no comparison

group of individuals who did not receive

telephone services. There could be no

comparison necessary to conduct any kind

of evaluation, much less the randomized

evaluation needed for credibility. Here

is the essence of self-evaluation: I am the

greatest law professor on Earth. Just ask me.

This is not the new legal empiricism.

To take another example from access to

justice, consider the following narrative:

Low-income individuals frequently en-

counter civil legal problems. Most would

like to obtain attorneys to help them re-

solve those problems. The primary rea-

son they do not consult or retain attor-

neys is that they cannot afford lawyers’

fees. To further access to justice, then,

governments, philanthropists, and others

should pump money into existing legal-

services programs to ﬁll the “justice gap.”

The narrative above has two basic

points: low-income individuals frequent-

ly encounter civil legal problems, and

most would like to obtain attorneys to

help resolve those problems but do not

do so because the cost is too high. Is the

ﬁrst point true? Is the second?

For many in the U.S. legal profession,

the answers to these two questions are

too obvious to require research. As sug-

gested by the phrase “justice gap,” it sim-

ply must be the case that low-income in-

dividuals and families desire but cannot

get legal services, speciﬁcally traditional

attorney-client relationships. But a hypo-

thetical new legal empiricism researcher

would not accept the idea that truths this

important are too obvious to investigate.

An editor of this volume, Rebecca San-

defur, did not accept that idea. She sought

to ﬁnd out whether low-income individ-

uals encountered legal problems and, if

so, whether they wanted attorney assis-

tance, by asking low-income individuals.

72 Dædalus, the Journal of the American Academy of Arts & Sciences

The New Legal

Empiricism &

Its Application

to Access-to -

Justice Inquiries

Deploying a well-executed set of focus

groups, she found that the answers to the

two questions identiﬁed above were com-

plex and nuanced. The evidence suggests

that low-income individuals frequently

encounter legal problems, but even when

they recognize those problems as legal

(not always), they generally prefer to in-

volve neither lawyers nor courts. Cost is

often not the primary reason for their re-

luctance to turn to formal law.

Neither Sandefur’s research nor the

rcts on the effectiveness of different lev-

els of legal services support the idea that

legal services are worthless or that fund-

ing for legal services should be cut. This

rigorous research does suggest, however,

that standard narratives that exist in the

U.S. legal profession are distorted in ways

that matter.

The previous discussion suggests that

the new legal empiricism has already pro-

vided much to challenge assumptions

common in the U.S. legal profession.

The approach could also offer much in

the way of guidance for the profession’s

policy-makers, regulators, funders, re-

formers, and revolutionaries. Challenges

to accepted truths are helpful, but the

approach should also deliver construc-

tive ideas. Early indicators are that it

will be able to do so. Here are some ques-

tions about which, with adequate fund-

ing and with the political will among the

members of the legal profession, a new

legal empiricism could provide useful

guidance:

What would be the effects of partial de-

regulation of the U.S. legal profession?

◌

Can limited licensed legal technicians,

or other kinds of nonlawyer legal pro-

fessionals, provide effective services

at a cost accessible to low-income in -

dividuals?

◌

Can online legal-service providers like

LegalZoom, or free online legal-ser-

vice providers, or nonproﬁts that pro-

vide a hybrid of online and tradition-

al lawyer services provide an effec-

tive way for low-income individuals

to beneﬁt from the justice system?

◌

Would changing legal ownership

rules allowing lawyers, or unsuper-

vised nonlawyer legal professionals,

to work as salaried employees of cor-

porations with convenient locations

(think paralegals in ofﬁces at Wal-

mart) improve access for low-income

individuals and families?

What would be the effects of moving

certain disputes to online or app-based

adjudication?

Can algorithms and scoring systems, ad-

ministered by human beings or comput-

ers implementing artiﬁcial intelligence

programs, improve the functioning of

courts, legal-services ofﬁces, court ad-

ministrators, and other key actors with-

in the justice system, as is already occur-

ring in the medical profession?

Are there effective ways to divert indi-

viduals accused of crimes away from

the traditional criminal law system?

It is impossible to overstate the impor-

tance of these questions to the modern

justice system. The new legal empiricism

has much to offer.

If the new legal empiricism has already

exploded some of the myths that previ-

ously masqueraded as truths, and if it has

much to offer for the future, why has the

U.S. legal profession yet to embrace it, and

what can be done about the situation?

Because medicine and other disciplines

have incorporated rigorous empiricism

into their understandings of what counts

as true, it cannot be that the new legal

empiricism is inimical to the judgment-

based reasoning that the legal profession

offers in solving legal problems. Nor can

148 (1) Winter 2019 73

D. James

Greiner

it be that rigorous empiricism is incon-

sistent with professional ethics, legal or

otherwise. Again, the medical example

shows as much. Rigorous empiricism can

coexist with or alongside professional

judgment.

In the 1930s, when medicine began to

turn to rigorous empiricism (particular-

ly the

rct), medicine was less a science

than an individualized craft, especially as

it was practiced outside major cities and

teaching/research facilities.

And in this

period, published papers in both med-

icine and law began to make use of ran-

domization to conduct empirically rig-

orous studies, suggesting that the intel-

lectual foundation for transformation of

the legal system into an evidence-based

ﬁeld was present then, just as it was in

medicine.

In prior work, Andrea Matthews and I

speculated that perhaps lawyers (and thus

judges, who in the United States are or-

dinarily former lawyers) resist rigorous

empiricism because 1) they are trained to

pursue goals that clients provide and their

thought processes are therefore funda-

mentally instrumental (in the service of

advocacy) as opposed to analytical; and

2) there may be social value to having law-

yers appear certain when they argue and

to having judges appear certain when they

make decisions, even when there is little

basis for that certainty, and to appear cer-

tain, lawyers and judges must convince

themselves that they are.

The ﬁrst obser-

vation, if true, might make it hard for law-

yers and judges to embrace the new legal

empiricism because they are trained more

to argue than to analyze, and thus instinc-

tively seek persuasion rather than truth.

The second observation, if true, might

make it hard for lawyers and judges to em-

brace this approach because certainty in-

hibits a desire for rigorous investigation.

Neither observation/argument is close

to bulletproof. The duties of corporate

general counsels are less about advocacy

and more about strategic decision-making

and policy-making than those of, say,

courtroom litigators. Yet we see little if any

evidence of rigorous empiricism in corpo-

rate counsel ofﬁces, despite the supposed

existence of market forces that might put

a premium on using a new legal empiri-

cism to discover money-saving truths.

When Matthews and I provided our spec-

ulation, we stated that these two obser-

vations were likely insufﬁcient to explain

fully the U.S. legal profession’s resistance

to evidence-based thinking.

Medicine’s partial turn to evidence-

based thinking reﬂected leadership shown

by particular members of the medical

profession; these leaders worked pri-

marily in urban teaching centers and had

strong connections to federal agencies.

The legal profession needs leaders who

can transform its thinking about what

counts as useful knowledge. Law schools

should create environments that intro-

duce students to the issues discussed here.

Legal academics aim to prepare the next

generation of leaders in the U.S. legal pro-

fession to lead. A crucial element of that

training is to teach about the need for,

and about how to work with researchers

to expand, the new legal empiricism.

74 Dædalus, the Journal of the American Academy of Arts & Sciences

The New Legal

Empiricism &

Its Application

to Access-to -

Justice Inquiries

endnotes

Thomas Pynchon, Gravity’s Rainbow (New York: Viking Press, 1973), 251.

Laura and John Arnold Foundation, Developing a National Model for Pretrial Risk Assessment (Hous-

ton and New York: Laura and John Arnold Foundation, 2013).

Figures to support the statements about oversubscription are on ﬁle with author, from a sur-

vey of 2014 cases in an eight-county area around Akron, Ohio. The statements regarding law-

yers’ focus on whether they can win are the result of conversations with legal aid attorneys

around the country, including in two Ohio locations.

Harry M. Marks, The Progress of Experiment: Science and Therapeutic Reform in the United States, 1900–

1990 (Cambridge: Cambridge University Press, 2000).

See Laura and John Arnold Foundation, Developing a National Model for Pretrial Risk Assessment; and

Jon Kleinberg, Himabindu Lakkaraju, Jure Leskovec, et al., “Human Decisions and Machine

Predictions,” NBER Working Paper No. 23180 (Cambridge, Mass.: National Bureau of Eco-

nomic Research, 2017), http://www.nber.org/papers/w23180.

D. James Greiner and Cassandra Wolos Pattanayak, “Randomized Evaluation in Legal Assis-

tance: What Difference Does Representation (Offer and Actual Use) Make?” The Yale Law

Journal 121 (8) (2012): 2175–2179, note 154.

Ken Smith, Kelly Thayer, and Kathy Garwold, Final Report on the Assessment of Telephone-Based Legal

Assistance Provided by Pennsylvania Legal Aid Programs Funded under the Access to Justice Act (Traverse

City, Mich.: The Resource for Great Programs, 2012), https://www.paiolta.org/wp-content

/uploads/2014/05/Report-on-Telephone-Based-Legal-Assistance.pdf.

Rebecca L. Sandefur, “The Importance of Doing Nothing: Everyday Problems and Responses

of Inaction,” in Transforming Lives: Law and Social Process, ed. Pascoe Pleasence, Alexy Buck, Nigel

J. Balmer, et al. (Norwich, United Kingdom: The Stationary Ofﬁce, 2007). See also Rebecca

L. Sandefur, “The Fulcrum Point of Equal Access to Justice: Legal and Non-Legal Institu-

tions of Remedy,” Loyola of Los Angeles Law Review 42 (4) (2009): 949.

See Washington State Bar Association, “Limited License Legal Technicians,” updated Septem-

b er 24, 2018, https://www.wsba.org/for-legal-professionals/join-the-legal-profession-in

-wa/limited-license-legal-technicians.

See LegalZoom, https://www.legalzoom.com/; and Upsolve, https://upsolve.org/.

See American Bar Association, “Rule 5.4: Professional Independence of a Lawyer,” https://

www.americanbar.org/groups/professional_responsibility/publications/model_rules_of_

professional_conduct/rule_5_4_professional_independence_of_a_lawyer.html.

See “Online Claims Service being Piloted in England and Wales,” Out-Law, April 10, 2018, https

://www.out-law.com/en/articles/2018/april/online-claims-service-piloted-england-wales/.

See Taylor Kubota, “Stanford Algorithm can Diagnose Pneumonia Better than Radiolo-

gists,” Stanford News, November 15, 2017, https://news.stanford.edu/2017/11/15/algorithm

-outperforms-radiologists-diagnosing-pneumonia/.

See Lone Star Justice Alliance, https://www.lonestarjusticealliance.org/.

D. James Greiner and Andrea Mathews, “Randomized Control Trials in the United States Le-

gal Profession,” Annual Review of Law and Social Science 12 (2016): 295.

Marks, The Progress of Experiment.

James Burns Amberson Jr., B. T. McMahon, and Max Piner, “A Clinical Trial of Sanocrysin

in Pulmonary Tuberculosis,” American Review of Tuberculosis 24 (1931): 401; and Frederick J.

Gaudet, George S. Harris, and Charles W. St. John, “Individual Differences in the Sentenc-

ing Tendencies of Judges,” Journal of Criminal Law and Criminology 23 (5) (1933).

Greiner and Matthews, “Randomized Control Trials in the United States Legal Profession.”

Marks, The Progress of Experiment.