Transcript Alternate Assessments on Alternate Achievement Standards

Alternate Assessments on
Alternate Achievement
Standards
Student Population
Jacqueline F. Kearns, Ed.D.
Elizabeth Towles-Reeves, MS
The Assessment Triangle & Validity Evaluation
Marion & Pellegrino (2006)
OBSERVATION
Assessment System
Test Development
Administration
Scoring
INTERPRETATION
VALIDITY EVALUATION
Empirical evidence
Theory & logic (argument)
Consequential features
COGNITION
Student Population
Academic content
Theory of Learning
Reporting
Alignment
Item Analysis & DIF/Bias
Measurement error
Scaling and Equating
Standard Setting
Cognition Vertex Validity Questions
1)
2)
Is the assessment appropriate for the students
for whom it was intended?
Is the assessment being administered to the
appropriate students?
Both are important for the validity evaluation
More Different Than Alike
SOURCE: Education Week analysis of data from the U.S. Department of Education, Office of Special Education Programs, Data Analysis System, 2002-03
Issues in Teaching/Assessing
Students in Alternate
Assessments










Varied levels of symbolic communication
Attention to salient features of stimuli
Memory
Limited motor response repertoire
Generalization
Self-Regulation
Meta-cognition
Skill Synthesis
Sensory Deficits
Special Health Care Needs

Kleinert, H., Browder, D., & Towles-Reeves, E. (2005). The assessment triangle and students
with significant cognitive disabilities: Models of student cognition. National Alternate Assessment
Center, Human Development Institute, University of Kentucky, Lexington. (PDF File)
Previous Data



165 Students across 7 states
Extensive documentation through 111 item inventory
Findings suggest:
 64% routinely use verbal language
 46% routinely understand pictures used to represent
objects
 11% don’t understand pictures used to represent
objects.

Almond & Bechard (2005) An In Depth Look at
students who take alternate assessments: What do we
know. Colorado EAG.
Learner Characteristics
Demographic Variables

Learner Characteristics (all on a continuum of skills):










Expressive Language
Receptive Language
Vision
Hearing
Motor
Engagement
Health Issues/Attendance
Reading
Mathematics
Use of an Augmentative Communication System (dichotomous
variable)
Methodology

Four partner states chose to participate

States 1, 2, and 3:


State 4:


gathered data in the administration process for their AAAAS via scannable document (i.e., bubble-sheet)
gathered data using Zoomerang, an online survey package.
N= 7,075
States & LCI Response Rates
State Geography Demographic
State
1
State
2
State
3
State
4
Eastern
Eastern
Rural
Suburban
UrbanSuburban
Urban
Western
Rural
North
Eastern
Sample
N
3595
Response
Rate
75%
2793
100%
468
91%
219
47%
Alternate Assessment Participation
Rates : % Total population
State 1
.959%
State 2
1.14%
State 3
.766%
State 4
.55%
IDEA Categories
Multiple Disabilities- 22.63%
0.80%
1.00%
Autism- 14.80%
0.03%
3.07% 1.50% 0.03%
5.27%
14.80%
Emotional Disability- 7.13%
3.90%
7.13%
Mental
Retardation
Autism
Multiple
Disabilities
22.63%
Mental Retardation (including
MMD-FMD)- 47.57%
Other Health Impairment- 5.27%
47.57%
Specific Learning Disability- 3.90%
Other- 3.07%
Traumatic Brain Injury- 1.50%
Orthopedic Impairment- 1.00%
Speech Language- 0.80%
Visual Impairment- 0.03%
Deaf-Blind- 0.03%
Level of Expressive Language and
Presence of Other Complex Characteristics
4.57%
3.94%
2.20%
2.35%
0.21%
0.21%
0.00%
State State State State
1
2
3
4
PreSymbolic
0.25%
State State State State
1
2
3
4
Emerging Symbolic
“Most significant cognitive disabilities”
0.00%
0.00%
0.06%
0.04%
State State State State
1
2
3
4
Symbolic
Expressive Language
Expressive Language
90.0%
80.0%
Percentage
70.0%
60.0%
Presymbolic
Emerging Symbolic
Symbolic
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
State 1
State 2
State 3
State
State 4
Receptive Language
Receptive Language
90.0%
80.0%
70.0%
Percentage
60.0%
Uncertain response
Alerts to input
Requires cues
Follows directions
50.0%
40.0%
30.0%
20.0%
10.0%
0.0%
State 1
State 2
State 3
State
State 4
Elementary School Grade BandExpressive Langague
Middle School Grade BandExpressive Language
10.70%
12.70%
20.30%
67.00%
17.00%
1
2
2
72.30%
3
3
High School Grade Band-Expressive
Language
9.70%
17.70%
1
72.60%
1
2
3
Use of Augmented Communication
Number of Students not using ACS
90.0%
80.0%
70.0%
Percentage
60.0%
50.0%
Presymbolic
Emerging Symbolic
40.0%
30.0%
20.0%
10.0%
0.0%
State 1
State 2
State 3
State
State 4
Reading
Reading
60.0%
50.0%
Percentage
40.0%
No awareness
Aware of text
Reads basic sight words
Basic understanding
Critical understanding
30.0%
20.0%
10.0%
0.0%
State 1
State 2
State 3
State
State 4
Mathematics
Mathematics
60.0%
50.0%
No awareness
Percentage
40.0%
Counts by rote to 5
1:1 correspondence
30.0%
Does computational procedures
with or without a calculator
20.0%
Applies computational procedures
10.0%
0.0%
State 1
State 2
State 3
State
State 4
Percentages of Students with High Skill Levels in Expressive Communication,
Receptive Language, Reading, and Mathematics
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
Series1
State 1 State 2 State 3 State 4
Skill level 1
43% 36% 40% 47%
State 1 State 2 State 3 State 4
Skill level 2
13% 12% 14% 29%
Elementary Grade Band-Reading
13.20%
Middle School Grade Band-Reading
17.90%
21.70%
18.90%
1
46.20%
19.00%
17.10%
2
3
4
High School Grade Band-Reading
18.70%
15.00%
47.00%
2
3
46.20%
4
19.40%
1
1
2
3
4
Elementary Grade Band-Math
Middle School Grade Band-Math
3.50%
3.70%
35.60%
19.10%
16.60%
1
11.90%
9.30%
2
46.40%
3
24.10%
4
29.60%
5
2
3
4
5
High School Grade Band-Math
6.00%
16.10%
6.20%
1
52.50%
1
19.10%
2
3
4
5
Who are the Kids?










Represent ~1% or less of the total assessed population
All disability categories were represented but primarily 3 emerge,
 Mental Retardation
 Multiple Disabilities
 Autism
Highly varied levels of expressive/receptive language use
Most students in the population use symbolic communication
Level of symbolic language distribution is similar across grade-bands
Only about 50% of the pre and emerging symbolic language users use ACS
Pre-symbolic expressive language users are more likely to have additional complex
characteristics.
Most of the population read basic sight words and solve simple math problems with a
calculator.
Lack of skill progression in reading across grade bands (elementary, middle & high)
Skill progression apparent in mathematics across grade bands but still small
Limitations




Only four state participants
Small sample size
Global items in reading and math
Participation rates at 1% or less
Cognition Vertex:
Validity Evaluation Essential Questions





Who is the population being assessed?
How do we document and monitor the population?
What do we know about how they learn (theory of learning)
academic content?
What do our assessment results tell us about how the
population is learning academic content?
Are our data about the population and theory of learning
consistent with student performances on the assessment?


If not, what assumptions are challenged?
What adjustments should be made?



Participation
Theory of Learning
Student Performance
References










Agran, M., Fodor-Davis, Moore, & Martella, (1992). Effects of peer-delivered self-instructional training on a lunch-making task
for students with severe disabilities. Education and Training in Mental Retardation, 27, 230-240.
Billingsley, F., Gallucci, C., Peck, C., Schwartz, I., & Staub, D. (1996). "But those kids can't even do math: An alternative
conceptualization of outcomes in special education. Special Education Leadership Review, 3 (1), 43-55.
Brown, L., Nisbet, J., Ford, A., Sweet, M., Shiraga, B., York, J., Loomis, R. (1983). The critical need for non-school instruction in
educational programs for severely handicapped students. Journal of the Association of the Severely Handicapped. 8, 71-77.
CAST (2002).
Fox, (1989). Stimulus Generalization of skills and persons with profound mental handicaps. Education and Training in Mental
Retardation, 24,219-299.
Haring, N. (1988). Generalization for students with severe handicaps: Strategies and solutions. Seattle, WA: University of
Washington Press.
Hughes, C. & Agran, M. (1993). Teaching persons with severe disabilities to use self-instruction in community settings: An
analysis of the applications. Journal of the Association for Persons with severe Handicaps, 18, 261-274.
Hughes, C., Hugo, K., & Blatt, J. (1996). Self-instructional intervention for teaching generalized problem-solving with a functional
task sequence. American Journal of Mental Retardation, 100 565-579.
Westling, D., & Fox, L. (2004). Teaching Students with Severe Disabilities. Columbus: Pearson (Merrell).
Whitman, T. L. (1990). Self-regulation and mental retardation. American Journal on Mental Retardation, 94, 347-362.
Contact Information
Jacqueline Kearns, Ed.D.





1 Quality Street, Suite 722
Lexington, Kentucky 40507
859-257-7672 X 80243
859-323-1838
[email protected]
Elizabeth Towles-Reeves, MS





1 Quality Street, Suite 722
Lexington, Kentucky 40507
859-257-7672 X 80255
859-323-1838
[email protected]
www.naacpartners.org