Transcript Document
Workshop Evaluation Worth the Effort! NLTAPA Conference Boise, ID Evaluation Overview • Purpose: To determine the effectiveness of a training program. Evaluation Overview • Four-level model developed by Donald Kirkpatrick (1998). • Reaction • Learning • Behavior • Results Evaluation Overview (continued) • Each successive evaluation level is built on information provided by the lower level. Level 1 Level 2 Level 4 Level 3 Reaction (Level 1) What they thought and felt about the training. Level 1 Level 1 Courses expanded, or condensed Instructors adjusted Courses removed from the program pending further review Courses expanded, or condensed Stormwater Management Participant Comments (Fall 2010) Too long and too deep of explanations. Too much info that doesn’t concern our needs. Maintenance section is too repetitive. Instructors Adjusted Winter Maintenance Participant Comments (Fall 2012) •Instructor should be someone that worked in this field at public works level. •Instructor seems to lack practical experience in this field. •Less talk about cost. More on winter maintenance. (That's what our bosses are for– keeping cost down.) Courses Removed • Make half days. • Break into different sections. • Break into different levels – decision planning vs. on-going Traffic Signals Participant Comments maintenance. (Fall 2012) Learning – Level 2 Pre and Post Quiz • Developed to evaluate the instruction and content of our training programs (NOT the student) • Consists of 6 questions • Note your answers on the back of the evaluation sheet Quiz Example Question 1. Denying physical access to someone because of their impairment is: a. Sometimes necessary b. Discrimination c. Permitted by law in certain circumstances d. Unimportant compared to making roads safe for cars Learning Course content modified Quiz questions refined “Un-Learning” in Winter Maintenance Behavior - Level 3 Enhanced skills and knowledge identified on the job o Newer process o 6 month evaluation Level 3 Results – Level 4 Cost savings of approximately $5,000 related to remediation of current signage $1,500 of savings in equipment expenditures $30,000 of expenses avoided that might have been incurred if gas line had to be relocated after completion Lawsuits avoided Questions ... LTAP – Local Technical Assistance Program Pennsylvania Department of Transportation Bureau of Planning and Research 400 North Street, 6th Floor Harrisburg, PA 17120 • E-mail: [email protected] • Web Site: www.ltap.state.pa.us