Transcript Financial Report 2001-2002 - National Association for

Innovations in
Career-Tech Assessment
Presented by:
Sandra Pritz, Senior Consultant
National Occupational Competency
Testing Institute
Why should career-technical
programs assess academic
skills?
• State/national academic standards
demand attention.
• High school students must pass
academic tests for graduation.
• Workplace demands for technical
literacy are increasing.
• Academics in context tend to be
student-friendly.
How Should Career-Technical
Programs Assess Academic Skills?
• In the context of the students’ careertechnical program
• In alignment with state and national
academic standards
• In ways that help students translate to
other tests they will need to take
• Embedded In their career-technical tests,
such as end-of-course or program
Alternative assessments
• Ask students to perform, create,
produce something
• Elicit applications of higher-level
thinking and problem-solving skills
• Connect with tasks that are meaningful
in terms of the related instruction
• Engage students in authentic realworld tasks
Adapted from A Practical Guide to Alternative Assessment
(Herman,Aschbacher, and Winters, ASCD,1992)
Portfolio Assessment
• Involves assessing a collection of
student work
• Establishes a purpose, criteria (for
what gets put in, when, and by whom),
and how the collection or individual
pieces will be evaluated
• Engages students in assessment
through gathering evidence of their
progress and achievements that meet
the criteria
Value Added to CareerTechnical Programs by Tests
• External and effective
• Critical competencies
benchmarked to industry
standards
• Correlated to state and national
academic standards
Effective Tests• are based on standards
• provide transparent links between
standards, learning objectives,
and learning activities
• help translate what is required
into terms the learner can
address
• complement projects, portfolios,
alternative assessments
Critical Competencies Are
Central
• Students should recognize them as
learning objectives with standards,
move toward self-assessment.
• Teachers should teach to them in
learning activities.
• They should be assessed—in
multiple ways—with the results used
to help focus instruction, for the
group and for individuals.
Process to Use Multiple Tests
to Improve Instruction
• Written tests—practice at all
levels by including in
assignments; mirror types of
items on HSTW tests, include
academics embedded in the
technical industry/program.
Electronic Technology Written
Students must be able to
comprehend the following:
•Read diagrams
•Recognize symbols
•Apply Ohm’s Law
Math Calculations needed:
•Solve parallel and series
circuits.
•Solve algebra equations
•Manage inverse relationships
Performance Tests
- Practice by including scenarios in
assignments
- Mirror real job tasks, time, tool
selection, procedures
- Open-ended, can apply a rubric, and
can easily assess at higher levels
- Some aspects yes/no, some scaled,
some qualitative judgment by expert
Precision Machining
Performance
1. You will receive a piece of cold rolled steel 1.125”
diameter x 6.62 long.
2. Following industrial safety rules, machine the part
on the lathe according to the specifications given on
the drawing provided. Check tools for condition and
sharpness.
Requires student to:
-evaluate drawing to develop a work plan
-select speeds and feeds
- read blueprints
Coupling Written and
Performance Tests
• Reflects reality better than one or the
other alone
• Provides for multiple forms of assessment
• Gives all students a chance to succeed
• Enables mutual reinforcement of learning
• States and schools using NOCTI tests
have documented improvement in
instruction from using analyses of test
results.
Academic Skills Measured
in Context
Academic
Skills
Career and Technical
Skills
Academic Skills
+ Career Technical
Skills
Comprehensive
Assessment
Assessing Academic Skills in
Context
• NOCTI tests of occupational
competencies also test some
embedded academics.
• Math, science, and language arts
are required to complete most
occupational items and
performance tasks.
Needs in Reading
• Comprehend and interpret the
materials of the occupational field
• Compare, analyze, synthesize
technical materials into a new set of
information
Needs in Math and Science
• Use analysis techniques
• Solve problems that require
integration of more than one math
concept and/or multiple steps
• Estimate and check answers for
reasonableness
• Solve open-ended problems
Science -- Carpentry
• 4% of the written test items and 14% of
the jobs of the performance test match
NSES science standards at the 9-12
grade level.
• Example question: Concrete wall forms
must be made __________
(A. strongest at the top, B. strongest in
the middle, C. strongest at the bottom,
D. uniform throughout).
NOCTI Academics Correlation
Project
• 7 NOCTI job-ready tests correlated
to academic standards for math,
science, and language arts
- National standards (NCTM,
NSES, NCTE)
- State standards (NY, CA, FL)
• Several business/industry tests
correlated to national standards
Academics Identified
• Written tests contain embedded
academics that match
standards in all three
disciplines.
• Performance tests contain a
higher percentage of embedded
academics than written tests,
and the items tend to be at
higher levels of proficiency.
Employability Skills
• Competency lists like SCANS
can be used to embed matching
employability skills in tests
• Bloom’s taxonomy* can be used
to embed higher order skills in
tests
* Bloom’s taxonomy progresses from
knowledge to comprehension,application,
analysis, synthesis, and evaluation
Bloom’s Taxonomy -Carpentry
Written
Performance
Knowledge
37%
14%
Comprehension
25%
0%
Application
27%
14%
Analysis
10%
0%
Synthesis
1%
72%
Evaluation
0%
0%
SCANS Skills -- Carpentry
Written
46%
Performance
86%(1 or
more)
Resources
6%
Information
8%
Systems
5%
50%
Technology
27%
86%
Sample of Carpentry Written Test Item
with Technical and Academic Content
at Proficient Level
If a benchmark is established at 100’-0” and
the bottom of the foundation is at 96’-4”,the
bottom of the foundation is _______ the
benchmark. (A. 6’-4” above, B. 3’-8” above,
C. 6’-4”below, D. 3’-8” below)
Math: proficient, 2 national standard
matches
Reading: basic, 1 national standard match
Bloom’s: analysis SCANS: acquires and
evaluates information
Sample Carpentry Performance
Job At Advanced Technical Level
with Integrated Academics
Using the appropriate tools, equipment, and
material, install an interior window frame in
the rough opening provided. Case the top
and part of both sides of the frame using
mitered corners. Use a table saw or power
miter box to cut trim.
Math: proficient; 4 national standard matches
Reading: advanced; 1 national standard match
Bloom’s: synthesis SCANS: selects technology
Ways NOCTI can help
• By providing performance as well as
written tests, NOCTI can help assess
academics at higher levels.
• NOCTI can respond to the need to
reinforce academic and employability
instruction through assessment, can
customize tests and test reports to
isolate academics scores.
• NOCTI can provide states the service
of cross-correlating its tests to their
state standards.
Ways NOCTI can help (cont.):
Workplace Readiness Test
• NOCTI Workplace Readiness
Test measuring employability
skills is being up-scaled
• A large and stellar Subject
Matter Expert Team has rated
the critical competencies
• Career Cluster Foundation
Skills will be assessed
Ways NOCTI can help (cont.):
Workplace Readiness Test
• Employability skills are focus of NOCTI
best-selling Workplace Readiness Test
since 1996.
• Up-scaled test in process to meet
changed needs of business & industry
– Research and synthesis of standards literature
(e.g. MSSC, SCANS, EFF)
– Review by >30 leaders, 3 focus groups
– New assessment to be developed by fall
– Will be formatted to align with career clusters
For Tech Prep Programs
• NOCTI’s critical competency lists
can be used to document curriculum
alignment for secondary and postsecondary articulation.
• NOCTI can validate student mastery
of competencies at the secondary
level for transition to and possible
credit at the post-secondary level.
States Career Cluster Initiative
Moves to Assessment Stage
• NASDCTEc has named NOCTI a
developer and the provider of its cluster,
pathways, foundation tests.
• NOCTI is administering the test
developed for the National Health
Sciences cluster.
• NOCTI is developing 3 pathways tests in
construction for PA, to be used
nationally.
• NOCTI is developing a Workplace
Readiness foundation assessment.
What Do I Receive
from NOCTI?
• Score Reports
– Coordinator’s Report
– Individual Student Reports
– Customized Reporting Available—
e.g. integrated academics
• Statistical Information
–
–
–
–
Group Data
Site Data
State Data
National Data
• Student Certificates
• Personalized Contact Assistance
Note:
All results are
broken down into
duty, category
and total scores.
0.0
Pre-Test Scores
Post-Test Scores
Total
Welding Costs
Welding Qualifications &
Procedures
Measuring Instruments
Gas Tungsten Arc
Welding
Gas Metal Arc & Flux
Cored Arc Welding
Shielded Metal Arc
Welding
Oxyfuel Welding &
Brazing
Power Source Principles
Safety
Welding Consumables
Welding Inspection
Welding Terms/Blueprint
Reading
Heat
Treat/Metallurgy/Distortion
Control
Thermal Cutting
Processes
Pre and Post Test Gain Analysis
Job Ready Welding Written Assessment
100.0
90.0
80.0
70.0
60.0
50.0
40.0
30.0
20.0
10.0
Thank You
Contact NOCTI for more information and
to discuss your interests--www.nocti.org or 1-800-334-6283