250 E Street SW
Washington, D.C. 20525
202-606-5000/ 800-942-2677
Frequently Asked Questions:
AmeriCorps State and National
Evaluations
AmeriCorps State and National
July 2024
1
Table of Contents
Changes from Previous Version ......................................................................................................... 3
Background .......................................................................................................................................... 3
Evaluation Requirements .................................................................................................................... 4
1) Where can I find the evaluation requirements for AmeriCorps State and National
grantees? .......................................................................................................................................... 4
2) What are the evaluation requirements for ASN grantees? ..................................................... 4
Formula Grantees ........................................................................................................................ 4
Competitive Grantees ................................................................................................................. 4
3) What time period must my evaluation cover? ......................................................................... 5
4) How does AmeriCorps calculate the $500,000 threshold for grantees? ............................. 5
5) Will changes in approved funding in continuation years affect the $500,000 threshold? . 6
Large Grantee Evaluation Requirements .......................................................................................... 6
6) What does AmeriCorps expect of grantees that receive an average annual ASN grant of
$500,000 or more and are required to conduct an independent impact evaluation? ........... 6
7) Are Large grantees required to conduct an impact evaluation in every grant cycle? ........ 7
8) I am a Large grantee who has not conducted an impact evaluation at my own project
site(s), but I am implementing an intervention/program model that has already been
rigorously evaluated at other sites. Do I still need to conduct an impact evaluation? ............ 7
Small Grantee Evaluation Requirements ........................................................................................... 7
9) What does AmeriCorps expect of grantees that receive an average annual AmeriCorps
grant of less than $500,000, or that receive an Education Award Program (EAP) or No Cost
Slot grant, and are required to conduct either an independent or internal evaluation? ........ 7
Timing of Evaluations .......................................................................................................................... 8
10) When should evaluations be completed and/or submitted? .............................................. 8
11) I have already completed an evaluation for my program. Do I have to keep conducting
new evaluations during every grant cycle? ................................................................................... 9
12) How should I implement my evaluation during my three-year grant cycle? ...................... 9
13) What happens if a grantee does not conduct an evaluation that successfully fulfills the
AmeriCorps evaluation requirements by the end of their second three-year grant award? .. 9
Alternative Evaluation Approaches ................................................................................................ 10
14) What if my program can’t meet the evaluation requirements as established? ............... 10
15) What types of AEAs are available? ....................................................................................... 10
16) How do I request an AEA? .................................................................................................... 10
17) How will I know if my AEA request is approved? ............................................................... 10
18) I was approved for an AEA during my current competitive grant cycle. Can I continue
using the same AEA for my next competitive grant cycle? ...................................................... 10
2
Submission and Review of Evaluation Plans .................................................................................. 10
19) What should be included in my evaluation plan? .............................................................. 10
20) What will AmeriCorps do with the evaluation plan submitted with the application? .... 11
21) What are the evaluation plan review results categories? .................................................. 11
22) What should I do if I need to change my approved evaluation plan? ............................. 12
23) What is the process for submitting revised plans for formal approval? .......................... 12
Submission and Review of Evaluation Reports .............................................................................. 13
24) What should be included in my evaluation report? ........................................................... 13
25) What will AmeriCorps do with evaluation reports submitted with grant applications? 13
26) What happens if a grantee that is required to submit an evaluation report does not
submit it by the due date? ........................................................................................................... 14
27) How are evaluation reports used post-award? ................................................................... 14
28) What happens if a grantee’s evaluation report shows null or negative findings? .......... 14
Other Questions on Evaluation ....................................................................................................... 14
29) What is the difference between performance measurement and evaluation? ............... 14
30) Is it required that evaluations measure the impact of the primary service activity on the
service beneficiaries or the community? May evaluations measure the impact of the
program on AmeriCorps member development? ................................................................... 15
31) My program design has changed so much it’s basically a new program. Am I still
required to evaluate it? ................................................................................................................ 16
32) Are multi-site grantees expected to evaluate all of their sites? ........................................ 16
33) I am an organization that has both a national direct grant and one or more state
subgrants, all of which are using the same program model. Can I use the evaluation of my
national direct grant to cover the evaluation requirements for my state subgrants? ........... 16
34) For grantees, including multi-focus intermediaries, that serve as umbrella organizations
for many different types of service activities (e.g., support mentoring, health, public safety,
and environmental programs), what should they evaluate? ................................................... 17
35) How much should I budget for evaluation? ........................................................................ 17
36) Are organizations that receive AmeriCorps funding from more than one funding stream
(i.e., ASN, VISTA, or AmeriCorps Seniors) required to conduct separate evaluations for
each program? .............................................................................................................................. 18
37) Are AmeriCorps grantees required to submit evaluation instruments to the Office of
Management and Budget? .......................................................................................................... 18
38) How does the language in the regulations, which states that AmeriCorps may
“supersede these requirements with an alternative evaluation approach, including one
conducted by the Corporation at the national level” apply to evaluation requirements? ... 18
39) Where can I locate an independent evaluator and what should be the selection criteria
for choosing one? ......................................................................................................................... 19
40) Where can I access additional information on evaluation? ............................................... 20
3
41) How can I get additional help to build my organization’s evaluation capacity? ............. 20
For State Commissions ..................................................................................................................... 21
42) Does participation in a statewide evaluation fulfill the evaluation requirements? ......... 21
43) Is a state commission required to submit the evaluations for its formula programs to
AmeriCorps? ................................................................................................................................. 21
44) When a commission submits a former formula program to AmeriCorps as a state
competitive application, is the application required to include an evaluation plan and/or
report? ............................................................................................................................................ 21
Resources ........................................................................................................................................... 22
Changes from Previous Version
The previous Frequently Asked Questions: AmeriCorps State and National Evaluations was
last updated in 2019. While there are some changes to many of the FAQs, please note the
following:
• Overall:
o Aligned with AmeriCorps branding.
o Updated hyperlinks.
• New:
o FAQ 21
o FAQ 23
• Clarified/Revised:
o FAQ 3
o FAQ 12
o FAQ 15
o FAQ 19
o FAQ 20
o FAQ 22
o FAQ 24
o FAQ 38
o FAQ 40
o FAQ 41
• Removed:
o FAQs related to 2019 or earlier.
o FAQs related to Data Collection Plans.
Background
These FAQs are intended to explain the AmeriCorps State and National evaluation
requirements and AmeriCorps’ expectations for ASN grantee evaluations. Additionally, the
4
FAQs provide information for ASN applicants and grantees/subgrantees to help them plan
and conduct successful evaluations of their AmeriCorps-supported programs. The goal of
this policy guidance is to help ensure that the evaluations conducted by ASN
grantees/subgrantees are an effective and appropriate tool to help them improve their
program interventions and build their evidence base. Unless otherwise noted, the guidance
in this document is for competitive ASN grantees.
Evaluation Requirements
1) Where can I find the evaluation requirements for AmeriCorps State and
National grantees?
The evaluation requirements for AmeriCorps State and National grantees can be found in
AmeriCorps regulations 45 CFR § 2522.500-.540 and .700-.740
.
2) What are the evaluation requirements for ASN grantees?
Formula Grantees
AmeriCorps state formula grantees are required to follow the evaluation requirements
established by their respective state service commission. Applicants for state formula grants
should contact their state commission for their grant evaluation requirements.
Competitive Grantees
If an ASN state competitive, national direct, or native nation grantee has received at least
three years of competitive funding for a project, they are required to submit an evaluation
plan when they recompete for competitive AmeriCorps funding for the same project.
If an ASN state competitive, national direct, or native nation grantee has received at least six
years of competitive funding for a project, they are required to submit an evaluation plan and
evaluation report when they recompete for competitive AmeriCorps funding for the same
project.
The type of required evaluation depends on the size and type of competitive grant awarded.
AmeriCorps state competitive, national direct, and native nation grantees (with the exception
of Education Award Programs) that receive an average annual AmeriCorps grant of $500,000
or more as documented in the federal share of the grant budget, must conduct an
independent impact evaluation. For further information on the requirements for grantees in
this category, see the Large Grantee Evaluation Requirements
section below.
AmeriCorps state competitive, national direct grantees, and native nation grantees that
receive an average annual AmeriCorps grant of less than $500,000 as documented in the
federal share of the grant budget, as well as all AmeriCorps Education Award Program and
No Cost Slot grantees, are required to conduct either an internal or an independent
evaluation, which does not need to be an impact evaluation. For further information on the
5
requirements for grantees in this category, see the Small Grantee Evaluation Requirements
section below.
Figure 1: Flow chart describing evaluation requirements.
3) What time period must my evaluation cover?
Evaluations must cover at least one year of AmeriCorps-funded service activity for the same
project. In this context, “one year” refers to activities that take place during one program year.
Depending on the program design, these activities may or may not span a full 12 months.
The AmeriCorps-funded service activity covered by the evaluation can be from any time
period. For example, a grantee may choose to evaluate data collected during a previous
competitive grant cycle for the same project. However, the evaluation report must be
completed or published during the current grant cycle in order to meet the grantee’s
evaluation requirements for that cycle.
4) How does AmeriCorps calculate the $500,000 threshold for grantees?
The $500,000 threshold is based on the average annual funding the grantee receives from
AmeriCorps as documented in the federal share of the grant budget (the AmeriCorps share
6
only, not the program’s total budget with matching funds) during the three-year grant cycle in
which the evaluation is conducted. When a grantee recompetes for funding, if the grantee is
requesting $500,000 per year or more from AmeriCorps in their recompete grant application
budget, the grantee will be considered a Large grantee and will be required to submit an
evaluation plan that will meet the requirements for a Large grantee in the upcoming three-
year grant cycle.
5) Will changes in approved funding in continuation years affect the
$500,000 threshold?
If a grantee is awarded a different amount of AmeriCorps funding in years 2 and/or 3 of a
grant cycle, it will affect the three-year average funding used to calculate evaluation
requirements for that grant cycle. If the change will move the grantee above the $500,000
average funding threshold, they will need to revise their previously-approved evaluation plan
to meet the Large grantee requirements. If the change will move the grantee below the
$500,000 average funding threshold, the grantee will have the option of revising their
previously-approved evaluation plan to meet the Small grantee requirements.
Large Grantee Evaluation Requirements
6) What does AmeriCorps expect of grantees that receive an average
annual ASN grant of $500,000 or more and are required to conduct an
independent impact evaluation?
AmeriCorps state competitive, national direct, and native nation grantees who receives an
average of $500,000 or more per year from AmeriCorps during a three-year grant cycle are
expected to conduct an independent impact evaluation during that grant cycle.
An evaluation is considered independent if it uses an external evaluator who has no formal or
personal relationship with, or stake in, the administration, management, or finances of the
grantee or of the program being evaluated.
An impact evaluation is designed to provide statistical evidence of how well a program works
and what effect it has on service recipients and/or service participants compared to what
would have happened in the absence of the program (i.e., evaluations that include a
comparison or control group). In addition, high-quality evaluations must be designed to
provide evidence of a causal relationship between program activities and outcomes (
45 CFR
§ 2522.700). Grantees must use an experimental or quasi-experimental design. The
evaluation method should match the size, scale, and purpose of the program.
An impact evaluation should be guided by measurable and clearly defined research
questions that identify the effect of program participation on program service recipients
and/or service participants for specific outcomes. These outcomes should be included in the
grantee’s theory of change and/or logic model.
7
7) Are Large grantees required to conduct an impact evaluation in every
grant cycle?
Large grantees that have already conducted an impact evaluation for the same project can
request an Alternative Evaluation Approach
to conduct a different type of independent
evaluation. See the AEA Guidance for details about what must be included in this request.
The subsequent independent evaluation activities should build on the findings from the
impact evaluation to strengthen the evidence base for their program and make continuous
improvements to the program.
8) I am a Large grantee who has not conducted an impact evaluation at
my own project site(s), but I am implementing an intervention/program
model that has already been rigorously evaluated at other sites. Do I still
need to conduct an impact evaluation?
Large grantees who are replicating an evidence-based intervention with fidelity can request
an Alternative Evaluation Approach
to conduct a different type of independent evaluation.
See the AEA Guidance for details about what must be included in this request. If AEA
approval is granted, the grantee is strongly encouraged to conduct a well-designed
implementation study.
Small Grantee Evaluation Requirements
9) What does AmeriCorps expect of grantees that receive an average
annual AmeriCorps grant of less than $500,000, or that receive an
Education Award Program (EAP) or No Cost Slot grant, and are required
to conduct either an independent or internal evaluation?
Grantees with an average annual grant under $500,000, as well as those that receive
Education Award Program (EAP) or No Cost Slot grants, are required to submit an evaluation
that may be conducted by either an internal or an independent evaluator.
An evaluation is considered internal if it is conducted by an individual from the grantee’s own
organization or other stakeholders, such as board members, partners, or volunteer affiliates.
Small grantees are not required to work with an independent individual or entity, such as a
university or research firm, to conduct their evaluation. However, grantees are encouraged to
use evaluators with training and/or experience with rigorous evaluations and may use an
independent evaluator, if preferred.
While AmeriCorps encourages Small grantees to use the most rigorous evaluation design
feasible, they are not required to conduct an experimental or quasi-experimental evaluation.
When selecting a study design, grantees should consider the various options and select the
design most appropriate for the program. Grantees may opt for a process, outcome, or
impact evaluation. However, AmeriCorps strongly encourages grantees to build on outcome
data collected through existing performance measurement efforts and implement a design
8
that includes assessment of program outcomes and a study comparison group, which can
help to determine whether outcomes can be attributed to the program.
Timing of Evaluations
10) When should evaluations be completed and/or submitted?
AmeriCorps competitive grantees must:
• submit an evaluation plan with each recompete application after their initial three
years of competitive funding;
• conduct an evaluation during their subsequent three years of competitive funding;
and
• submit a report of the findings from their evaluation when recompeting for funding
again.
First time applicants for AmeriCorps funding are not required to submit an evaluation plan
with their application and grantees are not required to conduct an evaluation during their
initial three-year grant period. However, it is highly recommended that grantees begin the
evaluation planning process during their initial three-year grant cycle so that they have a
strong evaluation plan and are prepared to conduct a high-quality evaluation if selected for
funding during a subsequent grant award period. AmeriCorps encourages grantees to
proactively use evaluation resources and assistance made available by AmeriCorps. Refer to
the Evaluation Core Curriculum course, “Laying the Groundwork for Your First Evaluation”, on
the AmeriCorps Evidence Readiness Resources
page for additional information.
Competitive cycle
Submit with application:
Activities during awarded grant cycle:
First
Nothing related to
evaluation requirements
Prepare to submit an evaluation plan
that will meet your evaluation
requirements
Second (or having
received three years of
competitive funding)
First evaluation plan Conduct your evaluation according to
your plan, and draft your second
evaluation plan
Third (or having
received six years of
competitive funding)
First evaluation report
and second evaluation
plan
Conduct your second evaluation
according your second plan, and draft
your third evaluation plan
Further years of
competitive funding
Follow the pattern
above
Follow the pattern above
9
11) I have already completed an evaluation for my program. Do I have to
keep conducting new evaluations during every grant cycle?
Yes. Grantees who continue to recompete for funding must continue to submit evaluation
plans for each grant period, as well as evaluation reports documenting the results of their
most recent evaluation, with their recompete applications. AmeriCorps expects that each
subsequent evaluation will be developmentally appropriate for the program’s lifecycle and
will build on the findings from previous evaluation efforts in order to strengthen the evidence
base for the program and make continuous improvements to the program.
12) How should I implement my evaluation during my three-year grant
cycle?
There is no one way a grantee should implement an evaluation, as each evaluation is
different. Some grantees may need to develop and pilot data collection instruments, whereas
others may use previously validated data instruments or use data from a previous year of
AmeriCorps-funded service activity. Some grantees may have made connections with an
external evaluator during their first competitive grant cycle, whereas other grantees may be
starting outreach once they are in their second grant cycle.
Regardless of where you are in executing your evaluation, you must allot enough time to hire
an evaluator (if needed), develop and pilot data collection instruments (if needed), collect
data for at least one year of AmeriCorps-funded service activity (see above
), analyze the data,
and write the evaluation report with your findings in time to submit the report with your next
recompete application. Please note: this means grantees do not have three full years to
complete evaluation activities.
If the outcomes being studied in the evaluation require longer-term measurement that will
extend beyond this timeframe, grantees may request an AEA on the basis of timing. See the
AEA Guidance
for more details.
13) What happens if a grantee does not conduct an evaluation that
successfully fulfills the AmeriCorps evaluation requirements by the end of
their second three-year grant award?
If a grantee has not completed an evaluation that fulfills their AmeriCorps evaluation
requirements, they should detail in the application narrative the evaluation activities that they
have carried out to date along with any evidence that they have gathered on the impact of
their program when they submit a recompete application. The grantee should also provide
an evaluation plan with their application that details their evaluation activities in the next
grant cycle. The evaluation plan must be designed to fulfill the evaluation requirements.
Recompeting grantees should keep in mind that evaluation reports and evaluation plans are
a required part of the AmeriCorps funding process. If a grantee that is required to submit an
evaluation report and/or evaluation plan fails to do so, AmeriCorps will take this into
consideration in making funding decisions. AmeriCorps reserves the right to not consider an
application that does not include the requisite evaluation report.
10
Alternative Evaluation Approaches
14) What if my program can’t meet the evaluation requirements as
established?
Grantees may be eligible to apply for approval of an AEA. An AEA effectively changes the
type of evaluation you can conduct and still meet your evaluation requirements. For example,
a Large grantee is required by the regulations to conduct an independent impact evaluation.
However, an AEA may allow for a Large grantee to conduct an independent non-impact
evaluation instead. Review the AEA Guidance
for more information about eligibility.
15) What types of AEAs are available?
Refer to the AEA Guidance for AEAs currently available to grantees.
16) How do I request an AEA?
You must complete and submit the AEA Request Form. Any AEA requests submitted without
using this form will be denied.
AEA requests should be submitted along with your evaluation plan when you submit your
recompete grant application as described in the Notice of Funding Opportunity. Requests
submitted outside of the recompete grant application process may be considered. Such
requests must be submitted as email attachments to [email protected]
. See
the AEA Guidance for details.
17) How will I know if my AEA request is approved?
AmeriCorps will notify you via email about whether your AEA request is approved or denied.
Please retain AEA approval notifications as grant records along with your approved
evaluation plan. If AmeriCorps denies your AEA request(s), you must conduct an evaluation
that meets the established evaluation requirements for your grant.
18) I was approved for an AEA during my current competitive grant cycle.
Can I continue using the same AEA for my next competitive grant cycle?
AEA approvals are for one three-year grant cycle only. You must submit a new AEA request
the next time you recompete for competitive funding.
Submission and Review of Evaluation Plans
19) What should be included in my evaluation plan?
All ASN competitive grantees who are required to submit an evaluation plan (see above),
regardless of grant size or type, must use the Evaluation Plan Template that accompanies the
Notice of Funding Opportunity. The template specifies the elements required in the plan.
11
20) What will AmeriCorps do with the evaluation plan submitted with the
application?
AmeriCorps will review and approve evaluation plans for all applicants who were required to
submit a plan that are selected to receive a competitive grant.
Evaluation plans will be reviewed to ensure that the plan meets the requirements based on
the size of the grant or approved AEA (if applicable) and covers at least one year of
AmeriCorps-funded service activity for the same project. Evaluation plans will also be
reviewed to note strengths and weaknesses that may affect the quality of the evaluation.
Reviewers compile their feedback in the Evaluation Plan Review Form.
All grantees who have evaluation plan requirements will receive an EPRF, which notes
whether the plan has been approved, is approved with recommendations, needs revisions,
or needs to be submitted.
Evaluation plans must be approved by the date specified in the Grant Terms and Conditions
(typically the August after the initial year of the grant award). Once the plan is approved,
grantees will be required to implement the evaluation specified in the approved plan during
the remainder of the three-year grant cycle.
21) What are the evaluation plan review results categories?
Evaluation plans fall into one of four categories:
• Approved, No Need to Resubmit – the evaluation plan, if executed as written, should
meet the grantee’s evaluation requirements and result in a high-quality evaluation.
There may be some recommendations noted in the EPRF.
• Approved with Recommendations, Resubmission Optional – the evaluation plan
meets the basic evaluation requirements, but requires revisions in order for the
grantee to conduct a high-quality evaluation. Grantees receive feedback that, if
implemented, can improve the quality of the plan itself and its implementation. It is
recommended that the grantee review the feedback and address concerns in the
evaluation. We encourage grantees to reach out for support as they incorporate
feedback through their evaluator or through the Evaluation TA portal
. Grantees are
not required to submit a revised plan for approval. However, grantees may revise their
plan and submit it for review if they would like additional feedback; revised plans
should be sent to
to address the suggested revisions may affect the desired learning objectives,
program improvement goals, and evidence level target.
• Not Approved, Re-Submit – the evaluation plan does not meet the requirements for
the evaluation that must be conducted for this grant cycle; at least one element in the
evaluation plan related to your requirements was either omitted or not clearly
addressed. The plan will need to be revised to meet the requirements and
resubmitted to [email protected]
. Grantees will also receive feedback
about the quality of their plan. We recommend the grantee also address the initial
12
feedback in the revised plan in order to receive additional feedback from reviewers.
Grantees may address feedback with their evaluator or through the
Evaluation TA
portal.
• Not Submitted, Submit – the grantee was required to submit an evaluation plan with
their application but did not. They will need to submit their plan as soon as possible
for review to [email protected]
.
This process allows grantees to move forward with an evaluation that works for their
program, even if it is not perfect when first submitted to AmeriCorps.
We also hope that grantees will actively reach out to the Evaluation TA portal both to address
feedback on their plan and to get insight from evaluation experts as the grantee implements
their evaluation.
22) What should I do if I need to change my approved evaluation plan?
The revised plan must align with AmeriCorps’ established evaluation requirements or the
requirements specified in an approved AEA for your current grant cycle.
If you change your approved evaluation plan significantly, such that it affects whether the
evaluation will meet evaluation requirements, you must submit a revised version of the plan
(with the proposed changes in track-changes mode) as an attachment to
(and cc your portfolio manager). AmeriCorps will review
and notify you if the evaluation plan is approved or needs to be revised. If the plan needs to
be revised, you can decide whether to implement the previously approved evaluation plan or
address the issues in the changed plan.
If you change your approved evaluation plan by incorporating feedback to strengthen your
evaluation, you may submit a revised version of the plan (with the proposed changes in track-
changes mode) as an attachment to [email protected]
(and cc your portfolio
manager). AmeriCorps will review and provide further feedback to your already approved
plan.
23) What is the process for submitting revised plans for formal approval?
If a grantee’s evaluation plan receives either Not Approved, Re-submit or Not Submitted,
Submit status, they will need to address the feedback in the EPRF in their evaluation plan to,
at minimum, meet their evaluation requirements.
Direct grantees must resubmit their revised plan to [email protected]
(and cc
their portfolio manager) for formal approval. The revised plan will go through review as
stated
above.
Commissions should send the EPRF to their subgrantee(s) as appropriate. Subgrantees then
revise their plans to address the feedback. Once the subgrantee is ready to formally submit
their revised plan, they submit it to their state commission. The state commission must send
the revised evaluation plan to [email protected]
(and cc their portfolio
13
manager) for formal review. AmeriCorps will not formally review evaluation plans sent directly
by subgrantees. The revised plan will be reviewed as stated above
.
Grantees are formally approved when their EPRF states the evaluation plan is in either
Approved or Approved with Recommendations status.
Submission and Review of Evaluation Reports
24) What should be included in my evaluation report?
Evaluation reports should include:
• an introduction to the project;
• the research questions;
• a description of the evaluation design and methods; and
• a section on the findings and results of the evaluation.
If the grantee conducted an experimental or quasi-experimental design, the report should
also include statistical evidence of the effect of the program compared to what would have
happened in the absence of the program (i.e. p-values, confidence intervals).
Evaluation reports should include a title page with the AmeriCorps grant number(s) for the
project(s) that was/were evaluated, the name of the project(s) evaluated, the evaluator’s
name, the evaluator’s organization name, and the report’s completion date.
For more information about what to include in an evaluation report, refer to the Evaluation
Core Curriculum module “Analysis and Reporting” on the AmeriCorps
Evidence Readiness
Resources page.
25) What will AmeriCorps do with evaluation reports submitted with grant
applications?
Expert evaluators will assess the quality of the evaluation design and implementation, the
strength of its findings, and whether the evaluation met AmeriCorps requirements. These
assessments may be used to inform AmeriCorps’ consideration of the selection criteria and
for the purpose of assessing the quality and outcomes of your program (45 CFR § 2522.740
).
Refer to the Notice of Funding Opportunity for additional information on how evaluation
reports will be used during the review process.
AmeriCorps will provide feedback on required evaluation reports for all applicants.
14
26) What happens if a grantee that is required to submit an evaluation
report does not submit it by the due date?
Evaluation reports are a required component of the AmeriCorps competitive application
process. If a grantee that is required to submit an evaluation report fails to do so by the
application due date, AmeriCorps will take this into consideration when making funding
decisions. AmeriCorps reserves the right not to consider an application that does not include
the requisite evaluation report.
27) How are evaluation reports used post-award?
Evaluation reports should be used by organizations to strengthen their program and their
impact. Not only can evaluations measure the impact on participants and beneficiaries, but
they can also provide feedback on the extent to which program implementation aligns with
the program model and whether impacts differ for different aspects of the program or for
different populations. These findings should guide program improvement, adjustment, and
future action.
Evaluation reports are used by AmeriCorps staff in three ways: First, to identify training and
technical assistance needs to foster continuous improvement and identify programmatic
areas that need attention. Second, to identify and share promising practices and models that
merit replication. And third, to strengthen the evidence base for the impact of national
service. AmeriCorps posts grantee evaluation reports on the Evidence Exchange
in order to
share best practices and promote the use of evidence-based program models.
28) What happens if a grantee’s evaluation report shows null or negative
findings?
Evaluation findings are one way that applicants can demonstrate evidence of a program’s
impact, and as such may be used in consideration of the selection criteria and to assess the
quality and outcomes of a proposed program model. However, applications with evaluations
that show null or negative findings will not automatically be excluded from competitive
consideration. Applicants should use relevant sections of the application narrative to
demonstrate how they have learned from the evaluation findings and made program
improvements to address weaknesses or unexpected findings. Applicants should also
develop evaluation plans for the coming grant cycle that will build on the findings from the
previous evaluation and help the program continuously improve. Grantees are encouraged
to work with an experienced evaluator or use AmeriCorps evaluation technical assistance
resources to identify how the evaluation findings can be used as a learning tool.
Other Questions on Evaluation
29) What is the difference between performance measurement and
evaluation?
According to the Government Accountability Office’s (GAO) performance measurement and
evaluation glossary, performance measurement is “the ongoing monitoring and reporting of
15
program accomplishments, particularly progress towards pre-established goals.” Evaluations,
on the other hand, are “individual systematic studies conducted periodically or on an ad hoc
basis to assess how well a program is working.” While performance measures track whether a
program has met its objectives, goals, or targets, program evaluations analyze a wider range
of information, including important contextual information on the program, beneficiaries, and
the external environment, than is feasible to track on an ongoing basis.
Performance measurement and evaluation both include systematic data collection, and
performance measurement data may be included in analysis conducted for an evaluation. For
example: a reading tutoring program identifies the need and a theory of change for
addressing that need, e.g., students are reading below grade level and the tutoring program
can help them achieve reading success. The program develops performance measures and
identifies a primary outcome to measure their ability to address the need, e.g., the primary
outcome is to have students reading at or above grade level by the end of one year. The
program tests students’ reading ability as part of their performance measurement activities
and gathers evidence of the extent to which the students who participate in the tutoring
program improve their reading ability and are reading at grade level after one year. The
program then designs an evaluation that will allow them to compare the outcomes for the
students who participate in the tutoring program with a matched group of students that does
not receive the tutoring services. By comparing the outcomes for the two groups, the
evaluation can determine whether the students’ improved reading skills can actually be
attributed to the tutoring program and not to other factors, such as regular reading
improvement or classroom instruction. The greater the capacity of the evaluation to control
for the differences between the students who receive tutoring and the comparison group and
their experiences (outside of participation in the tutoring program), the stronger the case can
be made that the improvement in tutored students’ reading ability, when compared to the
other group, was the direct result of the tutoring program.
Additional information on how evaluation differs from performance measurement can be
found in the AmeriCorps regulations 45 CFR § 2522.700
.
30) Is it required that evaluations measure the impact of the primary
service activity on the service beneficiaries or the community? May
evaluations measure the impact of the program on AmeriCorps member
development?
Evaluations must measure the impact of one or more significant program activities, but not
necessarily the primary service activity. AmeriCorps encourages grantees to conduct
evaluations that will provide rigorous evidence of beneficiary impact and demonstrate that
the program is an effective means to solve community problems. However, AmeriCorps also
recognizes that the impact of program participation on AmeriCorps members may represent
a significant component of a grantee’s theory of change. For this reason, an evaluation
focused on member outcomes is allowable.
If you are interested in evaluating members, please be sure to include member activities and
outcomes in your theory of change and logic model.
16
31) My program design has changed so much it’s basically a new
program. Am I still required to evaluate it?
It may not be developmentally appropriate for grantees to conduct an impact evaluation if
the program design has undergone considerable changes. In such cases, a Large grantee
may request an AEA based on program structure. Large grantees whose AEA requests are
approved will still be required to conduct an independent evaluation that is appropriate for
the program’s life cycle (e.g., a process or outcome evaluation). Small grantees that have
changed considerably are also required to conduct an independent or internal evaluation
that is appropriate to the program’s life cycle (e.g., a process or outcome evaluation).
When reviewing requests for an AEA as a result of changes in program design, AmeriCorps
will assess whether the program submitted for competitive funding consideration satisfies
AmeriCorps’ definition of “same project” (45 C.F.R.§ 2522.340
). A program will be
considered the “same project” if AmeriCorps cannot identify a meaningful difference
between the two programs based on a comparison of the following characteristics, among
others:
• The objectives and priorities;
• The nature of the services provided;
• The program staff, participants, and volunteers involved;
• The geographic locations in which the services are provided;
• The populations served; and
• The proposed community partnerships.
32) Are multi-site grantees expected to evaluate all of their sites?
AmeriCorps recommends that multi-site grantees evaluate a representative sample of
operating sites, if not all of the sites. If a sample of sites are selected, the evaluation results
should articulate how the results can be generalized to the other sites, and should describe
any key elements of variation (e.g., operations, size, type of location, program activity) among
the included and excluded sites. The most important factors are that the sites chosen for
evaluation are appropriate within the context of the evaluation design and methodology and
can serve as a representative sample for all of the program sites.
33) I am an organization that has both a national direct grant and one or
more state subgrants, all of which are using the same program model.
Can I use the evaluation of my national direct grant to cover the
evaluation requirements for my state subgrants?
State subgrants are legally distinct from national direct grants and so have separate
evaluation requirements, even if they are utilizing the same program model or intervention. In
order for an evaluation to meet the requirements for a particular grant or subgrant, the
17
evaluation must cover activities that take place at site(s) supported by that specific
grant/subgrant. Therefore, if the national direct evaluation includes sites supported by the
national direct grant AND sites supported by a state subgrant, then the national direct
evaluation can be used to meet the requirements for both grants. The evaluation report
should state which sites are associated with the national direct grant and state subgrant(s) to
ensure the report meets the requirements. Otherwise, the state subgrant must conduct a
separate evaluation of its own site(s).
34) For grantees, including multi-focus intermediaries, that serve as
umbrella organizations for many different types of service activities (e.g.,
support mentoring, health, public safety, and environmental programs),
what should they evaluate?
Grantees that support multiple types of service activities are not required to evaluate every
activity and should work with their evaluator to determine what to evaluate.
Several factors may be considered in deciding which type(s) of service activities should be
evaluated and how they should be evaluated. For example, grantees may want to focus on
the primary service activity as defined in the application. Grantees may also want to evaluate
elements of the program’s theory of change that are common to all the service activities, such
as volunteer leveraging, capacity-building results, or the ways in which AmeriCorps members
add value to operating sites or service locations. In addition, grantees should consider
program maturity, preliminary evidence of effectiveness, and the feasibility of conducting a
high quality evaluation for the different types of service activities.
35) How much should I budget for evaluation?
The cost of evaluations varies widely and will depend on the type of study design, the size of
the study, the level of expertise and experience of the evaluator, and data collection
expenses.
The more rigorous the level of evidence the evaluation is designed to provide or detect, the
higher the evaluation costs are likely to be. For example, randomized control trials, which use
an experimental study design, tend to be more expensive than other evaluations, but also
tend to yield the highest level of valid and rigorous evidence.
Evaluation costs tend to be driven by the type and amount of data collected. For example, an
evaluation collecting a large amount of new data from beneficiaries will typically cost more
than an evaluation collecting a smaller amount of new data, or than one that uses existing or
administrative data.
For more information about budgeting for evaluation, refer to the Evaluation Core
Curriculum module “Budgeting for Evaluation” on the AmeriCorps
Evidence Readiness
Resources page.
18
36) Are organizations that receive AmeriCorps funding from more than
one funding stream (i.e., ASN, VISTA, or AmeriCorps Seniors) required to
conduct separate evaluations for each program?
Organizations are required to evaluate each of their awarded projects in accordance with the
evaluation requirements of the AmeriCorps funding stream that supports that project.
Depending on the characteristics of the organization and programs, it may be possible for
the grantees to develop a single evaluation design capable of meeting the evaluation
requirements of more than one funding stream. In addition, it is possible that part of the
evaluation requirements may be covered if the project has been included in a national or a
statewide evaluation. If an individual organization receives more than one type of funding,
grantees should work with their state service commission and/or AmeriCorps staff to
determine if a separate evaluation is required for each funding stream.
37) Are AmeriCorps grantees required to submit evaluation instruments to
the Office of Management and Budget?
No. These requirements apply only to Federal Executive Departments and agencies,
branches of the military and other establishments of the Executive Branch of the federal
government. While not required, AmeriCorps encourages grantees to pursue institutional
review board clearance for evaluations collecting data from human subjects. IRBs are tasked
with reviewing evaluation and research plans to ensure that researchers protect the rights of
subjects participating in the planned research. IRB clearance may be especially important if
grantees seek to publish their work in academic journals. IRBs may be associated with
universities, localities (e.g., a local government or school district), or independent research
firms.
38) How does the language in the regulations, which states that
AmeriCorps may “supersede these requirements with an alternative
evaluation approach, including one conducted by the Corporation at
the national level” apply to evaluation requirements?
AmeriCorps’ Office of Research and Evaluation periodically undertakes national evaluations.
We encourage programs to participate if invited to do so. State or national grantees who
participate in these evaluations may meet their evaluation requirements through their
participation. In order to meet requirements, grantees must request an AmeriCorps National
Evaluation AEA. See the AEA Guidance
for details.
AmeriCorps will notify grantees of such national evaluation efforts as they emerge.
AmeriCorps has engaged grantees in Life Cycle Evaluations and Return on Investment
evaluations. Due to resource constraints, most AmeriCorps evaluation studies will focus on
certain types of programs. For example, AmeriCorps may be interested in conducting a study
of mentoring programs, and only a nationally representative or random sample of programs
conducting these activities may be asked to participate in the study.
19
39) Where can I locate an independent evaluator and what should be
the selection criteria for choosing one?
Universities are good sources for evaluators. Peers and state service commissions may be
able to provide recommendations or a list of college and university contacts that have
evaluation expertise. The AmeriCorps Office of Research and Evaluation holds research grant
competitions for institutions of higher education, and these universities could serve as local
resources. A listing of current and previous grantees can be found on the
AmeriCorps
Research Grantee Activities and Insights page.
National conferences such as those hosted by the Association for Research on Nonprofits and
Voluntary Action, the American Evaluation Association, and other nonprofit organizations can
also be excellent sources for referrals to evaluators. There are many entities dedicated to
conducting program evaluations and these typically have the breadth and depth of expertise
and resources to carry out rigorous evaluations. The American Evaluation Association website
provides a list of research firms/evaluators available by state.
Evaluator qualifications are critical to a successful evaluation that will improve the program’s
level of evidence. Here are some criteria to consider when selecting an evaluator:
• Years of experience;
• Successful completion of similar types of evaluations for similar social service;
programs;
• Experience in evaluating similar types of programs in similar settings;
• Capacity and/or access to other resources necessary to manage the scale and scope
of the evaluation (e.g., staff has the time to commit to the project, the organization has
the resources to collect and analyze breadth of data needed);
• References from previous clients; and
• Absence of investment in the program to be evaluated (i.e., independence is essential
to avoid potential or perceived bias on the part of the evaluator).
In planning for an evaluation, grantees should identify the qualifications required for the
evaluator and then assess potential evaluators on the extent to which their previous
experience and training provide them with the background necessary to meet those
qualifications. When selecting an evaluator, grantees should also consider any potential
conflicts of interest. For example, there is potential and perceived conflict of interest if the
evaluator played a role in designing the program, or if the person supervising the evaluator is
connected to the program in some way. Finally, AmeriCorps offers additional advice on
hiring an external evaluator in the Evaluation Core Curriculum module, “Managing an
External Evaluation” on the Evidence Readiness Resources
page.
Grantees must follow all applicable federal and state procurement regulations when hiring an
external evaluator. Refer to 2 CFR § 200.317 – 200.320
for more information.
20
40) Where can I access additional information on evaluation?
Evaluation resources, including evaluation policies, the evaluation plan template, and AEA
Guidance, are available under the Evaluate my program
resources of the AmeriCorps
grantee pages of the AmeriCorps website.
The AmeriCorps Evaluation Core Curriculum is located on the Evidence Readiness Resources
page of the AmeriCorps website. The Evaluation Core Curriculum consists of general
resources to support organizations in their learning and knowledge capacity building in four
phases of evaluation: planning; implementing; analyzing and reporting; and using
evaluations for action and program improvement.
41) How can I get additional help to build my organization’s evaluation
capacity?
AmeriCorps offers two types of evaluation technical assistance: the evaluation TA portal and
intensive TA. This evaluation TA is available free of charge to ASN grantees and competitive
subgrantees.
Grantees can submit a request anytime for individualized TA from an evaluation expert using
the evaluation TA request portal
.
Examples of the types of requests that may be made through the TA Portal include:
• Answering questions on evaluation design and approach.
• Providing feedback on evaluation materials and data collection tools.
• Advising on data collection and/or analysis procedures.
• Discussing approaches for identifying an external evaluator.
• Requesting assistance with evaluation plans that have not yet met AmeriCorps
requirements for approval.
• Requesting assistance with developing evaluation plans that will be submitted the
next time they recompete for funding.
Eligibility requirements for requesting TA are:
• Grantees and subgrantees must currently be competitively funded by AmeriCorps.
• Grantees and subgrantees conducting external evaluations may only submit requests
after consulting with their external evaluator.
• State subgrantees may only submit requests after consulting with their State
Commission.
21
Requests may be submitted at any time during the grant cycle, are not limited by the
grantee’s funding amount, and will not affect current or future awards. Grantees are not
limited in the number of requests they can make and are encouraged to seek assistance
proactively.
If a grantee’s evaluation plan seems to require a significant lift in order to be formally
approved, they may be invited to participate in intensive TA. This type of TA involves meeting
2-5 times for up to an hour per meeting with TA providers to review, improve, and revise their
evaluation plan so the evaluation report is more likely to meet requirements and be high-
quality. Direct grantees and state commissions will receive invitations to participate in
intensive TA in the fall of the first year of the grant cycle.
Please note: review of an evaluation plan by the TA providers does not constitute submitting
a plan for formal approval. Formal approval only occurs when the plan is submitted by either
the direct grantee or state commission to AmeriCorps at [email protected]
.
See above for more information.
For State Commissions
42) Does participation in a statewide evaluation fulfill the evaluation
requirements?
If a program participates in a statewide evaluation, that program may be exempt from the
requirement to conduct their own internal or independent evaluation for the next grant cycle
if the evaluation provides findings that fulfill the grantee’s evaluation requirements.
The program should describe its participation in the statewide evaluation when presenting
their evaluation plan for that grant cycle. Results must be disaggregated by program to
ensure consistency across the entire state portfolio, although individual results may not be
made public due to reasons of confidentiality. Nevertheless, the program is required to
submit its evaluation results as part of its recompete application in order to meet the
evaluation requirements.
43) Is a state commission required to submit the evaluations for its formula
programs to AmeriCorps?
No. However, AmeriCorps is very interested in developing best practices in evaluation design
and implementation. If a state commission has an example of a strong evaluation from a
formula grantee that they would like to share with AmeriCorps, we encourage them to do so.
44) When a commission submits a former formula program to AmeriCorps
as a state competitive application, is the application required to include
an evaluation plan and/or report?
If the grantee has never received competitive funding, the grantee is not required to submit
an evaluation plan or report in the first competitive grant cycle.
22
If the grantee has received competitive funding in the past, requirements are determined as
follows:
• If the grantee has received at least three years of competitive funding for the same
project, an evaluation plan is required.
• If the grantee has received at least six years of competitive funding for the same
project, an evaluation plan and an evaluation report are required.
AmeriCorps recognizes that some grantees move from competitive to formula and back
again. Commissions are encouraged to develop evaluation requirements for formula
grantees that will keep them on track to meet competitive evaluation requirements.
Resources
Evaluation TA Portal
Evidence Readiness Resources
Evaluate My Program
• Evaluation Plan Template
• AEA Guidance
• AEA Request Form
Regulations
• Evaluation Requirements: 45 CFR § 2522.500 - .540 and .700 - .740
• Same Project: 45 CFR § 2522.340
• Procurement Standards: 2 CFR § 200.317 – 200.320
Evidence Exchange
Produced and published at US taxpayer expense.
