Copyright © 2011 American Institutes for Research All rights reserved.

2013 Washington 21st CCLC Evaluation March 1 Webinar

Neil Naftzger and Samantha Sugar March 1, 2013

2

Topics

• • • Review site coordinator and staff survey data collection efforts and address any questions project directors have about the process Review suggestions for modifying the leading indicator reports Highlight the new local evaluation guidelines

3

Survey Timeline

• • • Site coordinator and staff survey data collection interface is now open Emails to project directors containing survey links were sent on

Friday, February 22

Complete surveys from site coordinator and afterschool program staff are due

April 5th

4

Survey Administration Process

 Once logged into the Project Director Survey Management page:        Ensure the centers listed for a given grant are correct and send any corrections needed to [email protected]

Under the name of each center, a survey Web link

unique

center is provided to each This link needs to be

forwarded

center to the site coordinator for that When the link is clicked on, the respondent will be asked to create a

survey account

and then be prompted to take the survey in question Site coordinators and staff will utilize the

same link

to create a survey account and take the survey for their role in the program You can return to the Project Director Survey Management page to view the

completion status

of surveys and send

email reminders

ideally, a minimum of

12 staff

per site should be identified per center

5

Questions About Leading Indicators

  Site coordinator survey will contain questions about the leading indicator reports and the advanced planning with data sessions held last fall Interested in learning what impact these efforts have had, if any, on internal program improvement efforts 

How have you or your staff used the leading indicator reports and/or the tools provided in the advanced training with data session to inform quality-related discussions in your program?

6

Leading Indicators

• • • The collection of site coordinator and staff survey data represents the start of the leading indicator reporting process for 2012-13 Goal is to make leading indicator reports available in PPICS by

August 15, 2013

Now is the time to make any changes to the leading indicator reports to make them more useful for programs

7

Suggestions

• • • • Provide more clarity on the presentation and interpretation of

youth outcome

data Allow for the presentation of leading indicator data at the

grantee

level Include a more

dashboard-like

of the report summary at the top Allow access to data obtained from state data warehouses to further support local evaluation efforts

8

Questions

•

What additional changes would you recommend be made to the leading indicator reports?

•

What changes should be made to the rollout of the reports and processes to support report utilization?

9

PPICS Data Collection Schedule

10

PPICS Webinars

 Two Webinars will be held in April and May 2013 to provide a review of PPICS reporting requirements 

Friday, April 19th

General Overview of APR Reporting Requirements at 10:00 am Pacific 

Friday, May 3rd

at 10:00 am Pacific - Focus on the WA Attendee Module

11 •

Local Evaluation Guidelines Revisions - Purpose

More structure with more freedom • Reduce redundancies • Improve the quality of reports

12 • •

Local Evaluation Guidelines Revisions - Method

Use multiple data sources to justify revisions • • • • Local Evaluator Interviews Project Director Interviews Survey data AIR review of local evaluation reports Review of draft revisions by LIAG

13 • • • • • • • •

Local Evaluation Guidelines Revisions – Findings

Program goals & objectives Remove requirement on more cumbersome data elements Guidance on how to present data Guidance on how to include other sources of data Inclusion of a logic model/theory of change Guidance on support provided by local evaluator Need for annual meeting for local evaluators Need for exemplars of quality reports

14

Local Evaluation Guidelines Revisions – Changes Made

• • Evaluation Process Guidance on timelines, goals and objectives, evaluation plans.

Guidance on submitting and reviewing reports • • Report Content Required elements Recommended elements • Report Format Guidance on expectations

15

Local Evaluation Guidelines Revisions

Available by Friday, March 8 th !

16

For Additional Help