Transcript NOCTI Overview - MnSCU CTE - Career Technical Education
NOCTI Overview
Amie Birdsall and Patricia Kelley
February 23, 2012
Discussion Topics
NOCTI Overview Products and Services Selecting an Assessment CTEDDI Q&A
NOCTI Overview
Over 40 years serving the CTE community Non-profit organization Owned and operated by the 50 states Each state is a member of NOCTI’s Consortium of States Works with both secondary and post-secondary populations Nocti Business Solutions (formerly The Whitener Group) provides services to industry
NOCTI Overview - Educational Partners
NOCTI Products & Services
Assessments Over 80 Job Ready tests offered 30 Pathway tests in 10 clusters available Over 90 private industry employment and/or advancement tests delivered 20 teacher-level assessments
Supplemental Products Study guides for all Job Ready and Pathway assessments
Countdown Kits available for all Job Ready assessments
Successful Schools-Successful Programs Guide Professional Learning Opportunities Also Offered!
Selecting an Assessment
Determine purpose of assessment Determine audience Determine results desired Research available options
Selecting an Assessment
Selecting an Assessment
Assessment blueprints are available on NOCTI’s website at no cost!
www.nocti.org
Assessment Blueprints
• Blueprints include: • Competency List • Content • Distribution Sample items/jobs
Assessment Blueprints
Content distribution for written and performance Sample multiple choice questions Sample performance job
Job Ready Assessments
Assess technical skills at the occupation level (e.g., Accounting, Carpentry, Pre-Engineering) Measure aspects of occupational competence such as factual and theoretical knowledge Target secondary and post-secondary program completers
NOCTI Job Ready Assessment Blueprint
Safety Plumbing Math General Plumbing Practices Joining Pipes and Fittings Drainage Vent Installation Water Supply and Distribution Hot Water Systems Traps Installing Fixtures Plumbing Service and Repair 8% 10% Plumbing 8% 9% 4% 6% 7% 7% 8% 9% 24%
NOCTI Job Ready Assessment Blueprint
Number of Questions: 222 Areas Covered: Accounting
NOCTI Pathway Assessment Blueprint
General Architecture & Construction Technical Skills Construction Academic Foundations Systems Ethics and Legal Responsibilities Communication Information Technology Applications Problem Solving, Critical Thinking, and Decision Making Leadership and Teamwork Safety, Health, and Environment Employability and Career Development Construction Pathway 15% 15% 15% 6% 6% 5% 6% 7% 8% 8% 9%
Assessment Results
NOCTI’s Standard Score Reporting Package: Individual Score Reports
Group Score Reports
Comparative Data
Other Reports Available: Task-Link Reports
Integrated Academic Reports
What data do I receive from NOCTI?
Score Reports Online Test Administration
Immediate Score Reports Aggregate Reports: 2 business days after user codes are released from Client Services Center
Paper/Pencil Administration
Aggregate Reports: 10 business days after receipt of correctly completed answer sheets
What data do I receive from NOCTI?
Immediate Score Reports
Immediate Student Scores Displayed to student if option is designated Emailed to Site Coordinator upon completion of testing (can be forwarded to instructor)
NOCTI Standard Score Reports
Individual Score Reports
• • • • Provided for each test taker Scores by duty and total Pre-/post-test comparisons Comparative data
NOCTI Standard Score Reports
Group Score Report
• • Provided for the testing group Duty and total scores for each participant
NOCTI Standard Score Reports
Analysis of Scores Report
• • • • Provided for the testing group Comparative data Criterion referenced cut score Other statistical information
NOCTI Additional Reports: Task-Link Report Task-Link Report
• Includes: • Number of items • presented for each task Average correct by • group Number correct for each test taker
Project Goals
• • • • • Understand data and assessments Use data effectively Improve learning and skill acquisition for students Plan for continuous improvement of learning and instruction Share ideas and collaborate with other educators
Instructional Improvement Cycle
Classroom Example A Nursing Assisting Class
Areas on a Nursing Assisting Test 8% 8% 28% 13% 10% 13% 10% 10%
Basic Nursing Assisting Infection Control Communication Skills Personal Care Skills Mental Health/Social Needs Client’s Rights Basic Restorative Devices Employability Skills
Individual Nursing Pretest
Nursing Assistant Pretest Areas Basic Nursing Assisting Infection Control Communication Skills Personal Care Skills Mental Health/Social Needs Client’s Rights Basic Restorative Devices Employability Skills Total Pretest % Correct Student 1
54.6
30.0
41.1
36.4
53.8
36.4
50.0
27.3
43.4
Posttest % Correct Student 1
- - - - - - - --
Nursing Assisting Pretest
Nursing Assistant Pretest Areas Basic Nursing Assisting Infection Control Communication Skills Personal Care Skills Mental Health/Social Needs Client’s Rights Basic Restorative Devices Employability Skills Total Student 1 % Correct Student 2 % Correct Student 3 % Correct Student 4 % Correct Student 5 % Correct Student 6 % Correct Class Average % Correct
54.6
45.5
42.7
27.3
27.3
54.5
41.9 30.0
41.1
36.4
53.8
36.4
50.0
27.3
43.4
20.0
17.6
54.5
38.5
36.4
60.0
18.2
38.4
10.0
23.5
45.5
42.3
38.3
50.0
18.2
35.9
20.0
23.5
45.5
38.5
36.4
60.0
27.3
38.5
10.0
13.2
36.4
20.8
22.5
30.0
9.1
22.8
30.0
35.3
54.5
53.8
45.5
80.0
27.3
48.8
20.0 25.7 45.5 41.3 35.9 55.0 21.2 38.0
Nursing Assisting Posttest
Nursing Assistant Pretest Areas Basic Nursing Assisting Infection Control Communication Skills Personal Care Skills Mental Health/Social Needs Client’s Rights Basic Restorative Devices Employability Skills Student 1 % Correct Student 3 % Correct Student 5 % Correct
100.0
91.7
100.0
92.3
97.4
84.8
87.5
87.5
92.4
94.5
100.0
54.5
72.5
66.4
60.2
78.5
76.2
72.4
55.3
52.5
60.3
44.7
45.2
37.4
Total
93.9
79.4
58.4
Group
86.4
79.2
95.0
69.2
71.8
64.6
53.1
79.7
76.8
Site
79.5
78.2
83.8
65.8
69.8
67.6
61.9
75.6
73.7
State Nation
72.5
70.7
64.3
51.6
63.1
58.1
55.6
66.6
64.1
75.5
70.6
66.4
52.9
64.1
57.8
56.5
67.0
65.2
What the Data Reveal
• • • • Achievement gaps Differences between subgroups Trends Progress
Collecting Data – More Than NOCTI
• • • Includes process of collecting, analyzing, and interpreting information Determines the degree to which students are becoming competent and meeting standards Occurs at many levels (e.g., classroom and program assessments to state and national assessments)
What Data Exist?
• • • National State District/System • • • School Program Teacher • • • Class Student Standards
Icebergs
What lies beneath the data scores that influence those scores?
What Questions Can the Data Answer?
• What is the data designed to answer?
• What other data may be needed?
• What roadblocks are there to interpreting the data?
Plan for Learning Improvement • • Start with results: Analyze the previous year’s goals, comparing against local unit tests and end-of-course tests, at the student level and classroom level.
Triangulate: Find relationships between test data and attendance and discipline referrals.
Plan for Learning Improvement • • • Continued: Disaggregate data by subgroups (e.g., socioeconomic status or those receiving free/reduced lunch, or Individualized Education Program/IEP).
Use information from advisory groups or parents.
Set goals for the year as well as long-term goals.
Plan for Learning Improvement • • • Continued: A common denominator is to improve learning through teamwork.
Build school department and state data teams.
Review annual assessment data with a faculty team to isolate achievement gaps or groups of students with high and low performance.
Plan for Learning Improvement Continued: • Schedule school and department data team • meetings to discuss data and review progress. Discuss and develop strategies, such as curricular alignment with academic and industry standards, curricular sequence, timing, and depth of instruction.
Monitor Student Progress
• Use a progress monitoring system for all students.
• Graph students’ curricula-based scores. • Identify effective instructional components.
Thank You
Amie Birdsall [email protected]
Pat Kelley [email protected]