Transcript EVALUATION
WHAT WILL BE COVERED 1. 2. 3. 4. 5. CNCS evaluation expectations Impact evaluation Choosing an external evaluator Evaluation plans Evaluation reports CNCS EVALUATION EXPECTATIONS What are the evaluation requirements for AmeriCorps grantees? • AmeriCorps State Formula Grantees are required to complete the evaluation requirement as established by their respective State Service Commission. • Serve Alabama is currently in the process of establishing evaluation policies. BUT WHAT ABOUT THE $500,000 THRESHOLD? If you are a… The following evaluation requirements apply… State Competitive grantee with an average annual CNCS grant under $500,000 Internal or Independent Evaluation State Competitive grantee with an average annual CNCS grant of $500,000 or more Independent Impact Evaluation National Grantee with an average annual CNCS grant under $500,000 Internal or Independent Evaluation National Grantee with an average annual CNCS grant of $500,000 or more Independent Impact Evaluation State and National Education Award Program (EAP) grantee, regardless of funding amount State Formula grantee Internal or Independent Evaluation Evaluation as specified by your State Commission IMPACT EVALUATION • An impact evaluation should provide statistical evidence of the impact of the program compared to what would have happened in the absence of the program (i.e. evaluations that include a comparison or control group). CHOOSING AN EXTERNAL EVALUATOR -CNCS SUGGESTED CRITERIA Criteria to Consider in Selecting an Evaluator 1. Years of experience 2. Successful completion of similar types of evaluation for similar social service programs 3. Capacity and/or access to other resources necessary to manage the scale and scope of the evaluation 4. References from previous clients 5. Absence of investment from the program to be evaluated CHOOSING AN EXTERNAL EVALUATOR -GENERAL SUGGESTIONS Look for an evaluator who 1. preferably a member of the American Evaluation Association 2. has good research skills but who also has the ability to communicate at the grassroots level 3. has good technical skills in data management and data analysis 4. will work in partnership with program stakeholders to design useful and used evaluations WHAT IS AN EVALUATION PLAN • A written document that details the program model being evaluated • Describes and justifies the evaluation approach selected • Provides instructions for the evaluation / a guide for each step of the evaluation process KEY COMPONENTS OF AN EVALUATION PLAN 1. Introduction 2. Description of intervention/impact theory (logic model) 3. Evaluation methods (design, data collection, analysis) 4. Data analysis 5. Timeline 6. Budget WHAT IS AN EVALUATION REPORT It is a written document that objectively describes all of the steps involved in completing the evaluation. • Program Background • Evaluation Purpose, Methods, and Procedures • Evaluation Results • Programmatic Achievements • Lessons Learned The document also provides a transparent basis for: • Understanding the program’s accountability to theory of change • Decision-making on policies and programs • Drawing lessons for program improvement KEY COMPONENTS OF A PROGRAM’S EVALUATION REPORT 1. Executive Summary 2. Background and Purpose 3. Evaluation Methods Uses (Design, Data Collection, Analysis) 4. Results 5. Conclusions and Recommendations 6. References and Appendices CNCS SUGGESTED RESOURCES • The American Evaluation Association • http://www.eval.org/ • The Evaluation Center • http://www.wmich.edu/evalctr/ • Innovation Network’s Point K Learning Center • http://www.innonet.org/ • Digital Resources for Evaluators • http://www.resources4evaluators.info/ • The CNCS Knowledge Network • https://www.nationalserviceresources.gov/ • The Kellogg Foundation’s Evaluation Handbook • http://www.wkkf.org/resourcedirectory/resource/2010/w-k-kellogg-foundationevaluation-handbook Questions