A Multi-dimensional Evaluation of the Houston Food Bank's Food

A Multi-dimensional Evaluation of the Houston Food Bank’s Food Prescription

Program

A brief report

Contributors to the report:

UTHealth School of Public Health

Shreela Sharma, PhD, RD, MA Nalini Ranjit, PhD

Henry Shelton Brown, PhD Jennifer Aiyer, MPH, RDN

Houston Food Bank

Nicole Lander, RDN Kenia Way, MS

Danielle Sanders, MPH, RDN Gwendolyn Sims, MEd, RD

Acknowledgements: We would like to acknowledge the Houston Food Bank,

healthcare organizations and food pantries who participated in this project. We would

also like to acknowledge the following individuals for their contributions on the

project.

• Robert McGowan, MPH

• Jemima John, PhD

• John Wesley McWhorter, DrPH, RD, CSCS

• Heidi McPherson, MPH

• Esther Liew, MSW, RDN

• Jack Toups

BACKGROUND

Food Prescription (Food Rx) programs are gaining popularity and increasing attention from

funders, policy makers, healthcare systems, social service agencies, and researchers, as a

strategy to both improve health outcomes and address social determinants of health among those

most vulnerable.

In this report, we present the results from our comprehensive, multi-

dimensional real-world investigation of a Food Rx program implemented by the Houston Food

Bank (HFB) in collaboration with multiple HCPs across the Greater Houston region.

In the HFB food prescription model, low-income patients at risk

for/diagnosed with chronic disease are enrolled by HCPs into the

Food Rx program. Enrollment consists of providing eligible

patients, in addition to their standard medical care, a prescription

voucher that can be redeemed at designated HFB-affiliated food

pantries called Food For Change markets located across the

region. The voucher allows up to 12 healthy food redemptions over

a 6-month period.

In 2020, The University of Texas Health Science School of Public

Health (UTHealth SPH) obtained funding from the Rockefeller Foundation to conduct a

comprehensive evaluation of HFB’s Food Rx program across 21 HCPs across Houston. As part of

our evaluation, we conducted

(a) a clinical effectiveness analysis to evaluate if utilization of the HFB Food Rx program led

to improvements in selected clinical markers

(b) a cost-effectiveness analysis to estimate societal benefits of implementing the Food Rx

program,

provider and staff, and HFB food pantry staff with the program.

(d) a data processes and quality study to examine and identify opportunities and

shortcomings in data and process flows

FINDINGS

Clinical effectiveness analysis:

For the clinical effectiveness study, we examined clinic visits data and pantry visit data collected

by the Houston Food Bank from HCPs and HFB-affiliated food pantries (i.e. FFC markets) between

May 2018 and March 2021. These data were provided to UTHealth under the terms of a Data Use

Agreement.

Downer S, Berkowitz SA, Harlan TS, Olstad DL, Mozaffarian D. Food is medicine: actions to integrate

food and nutrition into healthcare. BMJ. 2020-06-29 2020:m2482. doi:10.1136/bmj.m2482

Figure 1: HFB Food Rx formula

This kernel density map in Figure 2 shows the location of the 28 FFC markets/pantries (red dots)

and mobile trailer market/pantries (light blue dots) where patients can redeem their prescription

for food, the healthcare clinic locations (blue dots), and the “heat map”, or density, of the patients’

residence (shading in the background ranging from blue (sparse) to red (dense). The take-away

here is both the scope of the program geographically, while also noting that the healthcare

clinics, markets and participant residence densities overlap well for the most part, but there are

some areas where coverage of either clinics or pantries may be sparse relative to density of

number of patients residing in that area

We evaluated changes in clinical markers (HbA1c, systolic and diastolic blood pressure (DBP),

lipids (LDL), and BMI) in enrolled patients from the time of enrollment to the next recorded clinic

visit (average of about 6 months). Our approach was to conduct a complete case analysis for our

analysis of change in clinical markers. Thus, we could only include patients that had at least two

clinic visits recorded. Of the 2,028 unique patient records made available to us, less than half

were available for our complete case analysis.

Exposure status: Although all eligible patients receive food prescription vouchers at their

healthcare providers, only about half (53%) the patients redeemed one or more vouchers (i.e.,

visited a pantry) at any time between consecutive healthcare visits. We utilized a quasi-

experimental design in which patients who redeemed a voucher at least once between the first

and second clinic visits were considered “Exposed”, and patients that did not redeem any

voucher between clinic visits were treated as “Controls”. Analyses confirmed that Exposed

patients did not differ significantly from Control patients in socio-demographics or in severity of

disease at baseline, as further detailed below.

Description of patient population: Patients were eligible to participate in the Food Rx program if

they were food insecure at the time of enrollment and if their baseline clinical markers were

consistent with elevated disease risk. The average age of enrolled patients was 57 years. Nearly

70% of the patients were female. Two in five of the patients had less than high school education,

and only about 31% had any type of employment. The population was largely composed of

patients of color (56% Hispanic, 32% African-American), and over 80% of patients reported food

insecurity at baseline. The sociodemographic composition is largely comparable across Exposed

Figure 2: Heat map of Food Rx

participant place of residence

overlay with 13 FFC food

market trailers, 15 FFC food

markets/pantries, 21

Healthcare Partners

and Control groups. Finally, at baseline the HbA1c was slightly higher at 8.5% among the Exposed

group as compared to 8.2% among those in the Control group.

Key results: The Food Rx model utilized by HFB demonstrated a clinically meaningful decrease in

HbA1c levels among Exposed patients that redeemed their vouchers (-0.52%) as compared to

Control patients who did not redeem their voucher and received only standard medical care (-

0.24%), for a net difference of 0.28% (p=.007). Statistically significant reductions in systolic blood

pressure (-3.2, p<0.001) and diastolic blood pressure (-2.5, p=0.028) favoring the Exposed group

were also found. Thus, we found statistically significant greater decreases in HbA1c, SBP and DBP

among the Exposed who participated in the Food Rx program and redeemed their voucher, as

compared to those who were eligible and enrolled but did not participate. No differences across

Exposed and Control groups were seen in levels of LDL or BMI.

Figure 3: Magnitude of A1c decline by exposure status

Furthermore, a dose-response effect of visits on HbA1c reductions was evident. A

clinically significant reduction in HbA1c (> 0.50 percent points) was more likely among patients

that completed 7 or more visits in a 6-month period. (Figure 4)

Figure 4: Probability of >0.5% decline in A1c over 6 months, by number of visits

P-values shown in Figure 4 reflect the difference in probability of clinically significant improvements

in HbA1c across the indicated dose level, and the no visit dose level.

Cost-effectiveness analysis:

Subsequent to the clinical-effectiveness analysis, we conducted a cost-effectiveness analysis of

Food Rx using HbA1c as our health outcome to predict future health gains and averted medical

costs. We conducted a cost-utility analysis, or CUA.

The formula for CUA is as follows.

CUA =[Cost of Food Rx-Averted Medical Costs due to Food Rx]

QALYs with Food Rx – QALYs without Food Rx

The CUA was informed by three pieces of data (a) cost data for Food Rx, (b) estimates of medical

costs averted due to Food Rx, and (c) estimates of incremental QALYs with Food Rx.

For costs of Food Rx, we worked with the Houston Food Bank to arrive at estimates of Food Rx

provision. For the duration of the Food Rx implementation between 2018-2021, HFB estimated

that the cost of redemptions for each patient redeeming 12 vouchers over 6 months is up to $180,

assuming $15 per redemption per patient. The incremental cost of Food Rx does not include

micro-costing for the HFB retail centers (procurement, shipping, transportation etc) because

those costs occur without Food Rx. The averted medical costs and incremental Quality-Adjusted

Life Years (QALYs) come from our Markov Chain model. For our analyses, we applied a 20%

offset to this amount to account for food costs that would have been borne by patients without the

benefit of the Food Rx redemptions, and we used a range of redemption costs ranging from $15,

$20, $25 and $30 per redemption per patient. To estimate incremental QALYs from Food Rx, we

used our model, based on the CDC-RTI Diabetes Cost-Effectiveness Model

, which is a Markov

simulation model of disease progression and cost-effectiveness for type 2 diabetes over a

person's lifetime. This model simulates disease progression based on annual transitions between

states that represent health outcomes of common diabetes complications, and provides estimated

QALYs and costs for each possible trajectory. Each year of survival adds a life year, which is

quality adjusted. Costs for disease events such as heart attacks and strokes accrue. This allowed

us to estimate the number of incremental Quality Adjusted Life Years (QALYs) with Food Rx,

compared to treatment as usual, as well as associated costs.

Two other assumptions we made in our cost-effectiveness calculation were: first, all costs and

benefits are discounted at 3% to account for a preference for immediate benefits and delayed

costs, and second, the willingness to pay threshold for each added QALY is $50,000. From the

clinical effectiveness analysis, use of Food Rx (average of 6 visits a month) reduced HbA1c Levels

by 0.28 percentage points, which was assumed to be sustained after 3 years at one half that level.

Key results: Use of Food Rx versus control resulted in 0.07 incremental QALYs, corresponding to

$288 in averted medical costs. Overall, we conclude that HFB Food Rx is cost-saving and cost-

effective as long as the cost per redemption does not exceed $30 in cost with a 20% offset.

Mixed methods assessment of patients, HCP providers and staff and HFB food pantry staff:

We conducted a mixed methods assessment of patients that redeemed their prescriptions,

participating HCP providers and staff, HFB food pantry staff in order to obtain a comprehensive

360-degree view of factors informing implementation success of Food Rx. Additionally, we

interviewed patients who were eligible and enrolled but did not redeem their prescription to

understand factors associated with lack of redemption since this constituted 47% of those enrolled.

Data were collected through semi-structured phone interviews, open-ended forms, and brief

online surveys. In all, data were obtained from 18 patients (10 patients that redeemed vouchers,

and 8 patients that did not redeem vouchers), 252 responses by HCP staff, and 20 food pantry staff.

Themes in patient data:

(1) Recognition of healthcare commitment to value-based care through food prescription

Overall participants felt grateful for HCP efforts to prioritize provision of healthy food through the

food prescription program, which demonstrated HCP support of their personal health needs and

commitment to value-based care. Patients described both their decision to enroll and the program

purpose as necessary to support their health.

(2) Perceived need for ongoing outreach and communication from HCPs

Participants described gaps in messaging, including not knowing what to expect, unclear

eligibility status, and lack of follow-up communication from HCPs. Recommendations for

improvement included extending reminders and improving reach, as well as closing the time gap

between program enrollment and first food pantry visit.

(3) Recognition of individual patient challenges for program engagement

Individual personal challenges effected participants ability to engage fully in the program,

including lack of transportation, especially during Covid-19. One unexpected response from non-

participants was that they felt other people needed the food and services more than they did.

(4) Need for food pantry adjustments to provide food prescriptions

While participants enjoyed the choice, nutrition education, and variety of healthy food offered by

the food pantries, at times they also observed inconsistent nutrition education, food availability

and limited or unpredictable market hours across different types of pantries. In addition, the low

density of markets across the Houston area presents a possible barrier to those without

transportation.

“First, it [enrollment in the program] was because

we were unemployed. That was the first reason,

and also because of bad eating habits, since you

can’t always buy vegetables or even fruit.”- Food Rx

participant

“The only reason [for non-participation] is I don't

have no transportation. I guess I do make it to my

appointment, but I have a bike. And they offer you a

lot of food, and it's very hard to bring it in back to

the house in a bike.” – Food Rx participant

Themes in HCP providers and staff:

(1) Salience of value-based care strategy

Recognition of food prescription programming as an important VBC strategy paramount to

addressing food and health needs in a vulnerable population. Providers noted the appreciation

for providing the program to meet patient food needs that supports their health.

(2) Patient and food pantry barriers to success

Healthcare providers and staff heard from participants that transportation is a barrier to program

utilization. Additional feedback identified food pantries with variable hours and wait times, with

changing food variety and availability as other issues for participants. Ongoing input and

feedback from both organizational and participant voices is needed.

(3) Importance of inter-organization care coordination

Healthcare providers and staff felt strongly that inter-organization care coordination (i.e., between

healthcare, social service agencies and food pantries) was needed for effective and standardized

implementation of the program, including leveraging ongoing communication to work through

implementation solutions.

(4) Needed workflow integration within HCPs

Healthcare clinics are time strapped and though seen as valuable, Food Rx program

implementation will require explicit and reinforced integration into existing workflows including

both personnel and time.

"Food Rx helps with participants to use their new knowledge about healthy eating by providing

them with fruits and vegetables to go home and eat more healthy. It helps relieve the monetary

burden, and encourage them to try fruits and vegetables that otherwise they would not be able to

buy." – healthcare provider

“we are also having some patients with challenges going themselves because of work, so

working to try to get them to write another shopper on their behalf when they register." –

healthcare provider

Perceptions of HFB food pantry staff:

Top perceived barriers among participants to Food Rx visits:

• Lack of transportation

• Fears about government knowledge of personal information provided

• Inability to cook or prepare food provided

• Perceive that other clients need the program more than the current enrolled clients do

• Don't have time to come (competing priorities)

Top perceived benefits among participants to Food Rx visits:

• Provides better access to healthy foods

• Helps them eat more fruits and vegetables

• Provides exposure to try new healthy food with low risk/no risk

• Improves knowledge of nutrition and healthy eating

• Helps feed their family and children

• Helps families save money on food so they can use that money for other items

Similar to patients, staff at HFB-affiliated Food Rx food

pantries also perceived the Food Rx program

primarily as a food distribution program, rather than

as a health intervention. Of particular interest was

stronger relationships with HCPs, and a stated desire to

know more about the health condition of their patients in

order to better meet participants’ needs. Moreover,

although the Food “prescription” includes both the

provision of food and nutrition education, roughly half

the staff did not provide the nutrition education

component reinforcing a disconnect between program

objectives and the nutrition education component of the

intervention. Further, there is some ambiguity about whether nutrition education is the

responsibility of health care providers or food pantry staff.

Common themes and implications across all three stakeholder groups – providers, patients

and food pantry staff:

(a) Strong support for the Food Rx program across all stakeholder groups, but a less than complete

understanding of the purpose of the program and how it relates to health, with both food pantry

staff and patients seeing it more as food distribution program rather than a health promotion

initiative. This narrow view of program purpose may partly explain both low redemption returns,

and the limited provision of nutrition education by food pantry staff.

(b) A perceived need for warm referral and care coordination protocols to improve redemption

and retention of Food Rx participants. Currently, the healthcare provider has no way of following

up with either the patient or with the food bank (effectively, the food pharmacy) after the patient

leaves the healthcare premises after the enrollment visit until the patient arrives at the food Rx-

designated food pantry for their redemption. These types of challenges in communication and

follow up may reduce patient motivation to redeem their prescription, and provider and pantry

investment in the program’s outcomes

Food Rx redemption. Thus, to the extent that it is possible, there is need for health care providers

and potentially, pantry staff, to offer referral and support services to address these other needs.

“Well, I mean, I guess probably whenever people visit the clinic, maybe another friendly

reminder there. Maybe a flyer. That's a different way of letting people know that, "You know

what? You still have your pickups." – patient who was eligible and enrolled in Food Rx but

did not redeem their voucher

Data processes and quality study:

As part of our systems and operations study, we evaluated data and information flow, loss of data

across the system, and data quality issues. Several areas for improvement were identified and

reported back to HFB and the HCPs participating in the program. These include procedures to

improve data integrity and usability, and processes to improve program implementation. Our

recommendations are summarized in the next two paragraphs.

Recommendations to improve data integrity. These included: (a) Use of more failsafe data

management systems, standardization and automation. For example, replacing paper ID cards

electronic IDs linked to smartphones could prevent ID transcription errors, as well as guard

against loss of ID cards (b) Streamlining the type of data collected (individual level rather than

aggregate) as well as content (more information needed on nutrition education provided) would

reduce wastage, while maximizing the information content of the data, c) Streamlining the systems

of record where the data are being collected, shared and stored, as well as real-time monitoring

to assess data quality and integrity (e.g. response rate on surveys to assess missingness), as well

as program implementation fidelity are needed.

Recommendations to improve data usability. These included (a) Establishing shared metrics

important to all stakeholder groups and related reporting structures, b) Establishing regular data

reporting periods and standards for HCPs, as well as a communication plan for patient callback to

ensure participation, document reasons for non-participation, and a feedback loop to address any

non-response and non-participation issues. This would ensure intervention fidelity and also that

sufficient post-program data is available (c) More consistency across health care systems in

screening and tracking food insecurity and other SDOH (d) standardization of eligibility criteria

across providers (e) Use of automated procedures to merge health care visits data with

redemption data on a daily basis, to maximize the availability of usable data at any time.

Recommended changes in data and process flows: (a) Strengthen care coordination - Set up a warm

referral system that allows health care provider staff to connect patient with staff at a food pantry

upon enrollment, and at predesignated intervals thereafter, to motivate the first visit to a pantry

and to prevent patient fall-off thereafter (b) Set up communication systems and plans within

healthcare organizations, food bank and pantry staff to communicate with patients (c) Set up

rational costing systems that can be updated on a regular basis. Because Food Rx is only one of

many programs administered by healthcare provider staff and pantry staff, it is important to know

at all times what proportion of fixed and variable costs should be attributed to Food Rx. (d) Better

workflow integration, training and capacity building across provider and pantry staff is needed

for greater implementation efficiency.

Stakeholder implications

Our Food Rx evaluation has implications for findings to various stakeholders that share an interest

in expanding food prescription programs and integrating them into health care delivery. These

are summarized below, separately for each category of stakeholder.

Healthcare Payers and Providers: Healthcare payers and providers should consider including Food

Rx programs as a value-based care reimbursement strategy. The particular voucher-based model

we explored, where food distribution is managed by an agency outside the healthcare provider,

is feasible, cost-effective, and clinically effective. For optimal functioning, several system-wide

changes will have to be instituted, including (i) Socializing all agents to the value of the program,

and frame as prescription for improving diet quality and health outcomes, not just food access (ii)

Integrating program activities into workflow of healthcare organizations so that it is part of the

quality metrics. (iii) Establishing communication protocols and feedback channels between

HCPs, food bank, food pantries, and patients to motivate redemption and keep patients engaged.

(iv) Motivating collection of clinic data (clinic outcomes, prescription compliance, and SDOH) at

regular intervals to facilitate monitoring and implementation.

Funders and Policymakers: With the mounting evidence of effectiveness of food prescription

programs from studies such as ours and others, as Medicaid expansion is considered by states,

policymakers should support CMS reimbursement for food prescription programs to improve

health outcomes among those at highest risk. Although that the voucher-based model is feasible,

cost-effective, and scalable, neither healthcare systems nor social service agencies like the food

bank may have the expertise to smoothly incorporate outcomes measurement from Food Rx

programs in their existing workflow. Funders should require the partnership of academic or other

non-profit research-focused agencies to build the data backbone and infrastructure necessary for

ongoing monitoring and evaluation. Policymakers and funders should also look into supporting

research to investigate low redemption rates, and low return rates of patients to healthcare

providers, using strategies such as making food more accessible, e.g., by including grocery retail

stores in the distribution chain. Another area of research is to find ways to measure and optimize

food provision costs and increase access by increasing outlets.

Researchers: There continue to be gaps in our knowledge about clinical effectiveness and

implementation barriers to Food Rx programs. Most research examines effects over a 6-month

period. Further research is needed to examine what is a sufficient dose, and whether there is

decay of effects over time. To this end, researchers should also be prepared to assist in structuring

standardized food prescription program fidelity and evaluation metrics to inform program

evaluation. Additionally, we need research that emphasizes implementation outcomes, including

factors affecting program participation, and alternative models of food provision. Researchers

should also focus on building partnerships with community organizations that are involved in Food

Rx program. Such cross-sector partnerships with healthcare and community-based organizations

can focus on facilitating research conduct (recruitment and implementation), optimizing data

processes (streamlining data collection and program valuation methodology guided by validated

frameworks), and assisting organizations with developing key performance indicators (KPIs) for

program implementation. Finally, as mentioned above, another area of research is to find ways to

measure and optimize food provision costs and increase access by increasing outlets for food

prescription redemption by leveraging existing resources such as grocery retail stores.

Advocates, Implementers, Community Groups

Community coalitions and advocates that have their ear to the ground can more effectively gather,

reflect and implement community perspectives. Thus, they can:

● Obtain community perspectives regarding community needs, perceived challenges, how

to best mitigate barriers to program implementation, and how to address social disparities.

● Collaborate with other stakeholders on meaningful solutions to support long-term

program sustainability through community driven infrastructure and initiatives to support

these programs, for instance, involvement of local retail.

● Establish relationships and shared processes across sectors to support all aspects of food

prescription programming and evaluation.

● Support dissemination of research and findings to wider audiences using attributive

science communication techniques, including “common-speak” and sharing the lived

experience.

Conclusions

It is often noted that translation of research into evidence-based policies and programs suitable

for public use can take anywhere from fifteen to twenty years. This project sought to narrow this

translation gap by both providing further evidence of clinical and cost effectiveness of Food Rx

programs, and by highlighting pitfalls and potential solutions in the dissemination and

implementation aspects of a large voucher-based Food Rx program.

Funding Acknowledgements: This paper is based on research funded by The Rockefeller

Foundation. The findings and conclusions contained within are those of the authors and do not

necessarily reflect positions or policies of The Rockefeller Foundation.