Transcript Slide 0

Florida Association of
Science Supervisors
Michele Sonnenfeld
NAEP State Coordinator
Florida Department of Education
October 2006
0
National Assessment of
Educational Progress (NAEP)
“The Nation’s Report Card”
NAEP’s two major goals are to:
1. Measure and compare student
achievement in states and other
jurisdictions
2. Report changes in achievement of
4th, 8th, and 12th graders over time
1
NAEP
• Has produced National results for
grades 4, 8, and 12 since 1969
• Has produced state-level results at
grades 4 and 8 since 1990
• Has produced results for Trial Urban
Districts (TUDA) since 2002
• Does not produce student-, school-,
or district-level results, except for the
10 TUDAs
2
Current Organizational
Structure of NAEP
US Department of Education
Institute of Education Services
National Assessment
Governing Board (NAGB)
National Center for
Educational Statistics
Assessment Division
National Assessment of
Educational Progress
3
Contractors:
ETS, Westat, AIR,
NCS Pearson, GMRI,
Hager Sharp, HumRRO,
And ESSI
Federal Legislative Reference
No Child Left Behind Act of 2001
(NCLB)
•Section 111(c)(2) of NCLB requires
states/districts who receive Title I
funding to participate in biennial
State NAEP assessments of 4th and
8th graders in Reading and
Mathematics, beginning with the
2002-2003 school year.
4
State Legislative Reference
Added in 1990
Florida State Statute 1008.22 (2)
“The Commissioner of Education
shall direct Florida school districts
to participate in the administration
of NAEP . . . both for the national
sample and for any state-by-state
comparison programs .”
– State legislation requires all selected
schools to participate in all NAEP
assessments.
5
NAEP Components
National Assessment of Educational Progress
Main NAEP
TUDA
(Public)
(Grades 4 & 8)
NATIONAL
(Public & Private)
(Grades 4, 8, & 12)
STATE
(Public)
(Grades 4 & 8)
6
Long Term Trend NAEP
NATIONAL
(Public & Private)
(9-, 13-, & 17-year olds)
NAEP Frameworks
• The American Institute of Research
(AIR), with the help of teachers,
develops the NAEP frameworks.
• The National Assessment Governing
Board (NAGB) approves and adopts
the frameworks after they have been
reviewed and critiqued by teachers.
• Each framework describes the
content and format of the 4th, 8th, and
12th grade assessments.
• There are frameworks for each of the
content areas assessed by NAEP.
7
NAEP Science Frameworks
• Current science framework was
used to guide the 1996, 2000, and
2005 assessments
• A new framework, approved in
2005, will be used to direct future
assessments.
8
NAEP Science Frameworks
Current science framework requires
assessment in
– Earth science
– Physical science
– Life science
9
NAEP Design:
Item Development
• Items are based on the subject area
frameworks approved by the National
Assessment Governing Board (NAGB).
• Items are developed by teachers, subject
area specialists, and assessment experts.
• Items meet professional standards for
testing.
• Experts review items for technical/statistical
quality, content quality, bias, and sensitivity.
10
How Florida’s School
Sample is Selected
• Step 1: Lists of schools are obtained from
the National Center for Education Statistics
Common Core of Data (CCD).
• Step 2: Schools are grouped into strata by
two primary characteristics:
– Type of location (central city, urban fringe, rural)
– Minority enrollment (white, African American,
Hispanic)
• Step 3: Within strata, schools are ordered
by a measure related to achievement.
• Step 4: A proportional sample is selected
systematically from the ordered list.
11
Florida’s Measure of
Achievement
Grade 4
Mathematics average scale scores
for all students, grade 4, in 2005
Grade 8
Median household income (because
the match rate for achievement
scores for 8th grade was less than
70%)
12
How is the Student Sample
Selected?
• A list of all 4th or 8th grade students enrolled
at the sampled schools is submitted
electronically to Westat from the FLDOE,
based on the October FTE data.
• 90% of selected students at each school
must participate in the assessment or makeup sessions are scheduled at schools not
meeting this benchmark.
• Students exempted by their parents are not
replaced and are not used to calculate the
assessment participation rate.
13
NAEP 2007
• NAEP assessments will be administered in Florida
from January 22 through February 21, 2007.
• Each student takes only a portion of the questions,
answering two 25-minute sections of subject-area
questions.
• Students also answer questions related to their
background and their home or school experiences in
the subject of their assessment.
• Assessment is administered by a team from Westat,
the NAEP administration contractor for the US DOE,
and National Center for Education Statistics (NCES).
• Parents must be notified prior to the assessment.
Refusals must be in writing.
14
Sampling
Schools and Students
• ~350 Florida public schools in 49 districts will
participate in some type of NAEP assessment in
2007.
• Grade 4 – Reading and Mathematics:
Florida’s sample will include 167 schools.
Schools can elect to test all students (up to 120).
If there are more than 120 4th graders, NAEP will
select a sample of 62 students to assess.
• Grade 8 – Reading, Mathematics, and Writing:
NAEP will select a sample of 93 students at each of
the 161 Florida schools included in the sample.
• Grade 12 – Reading, Mathematics, and Writing:
Florida’s sample will include 31 high schools, and
75 students from each school will be selected to
participate.
15
NAEP Inclusions and
Accommodations
• NAEP has included students needing accommodations
since 1998.
• Students with disabilities (SWD) and English language
learners (ELL) are to be included in NAEP based on an
IEP/504 Accommodations Plan/ELL Plan team decision.
• Most frequent accommodations with NAEP:
1. Extra testing time
2. Individual or small group administrations
3. Large Print booklets
• Does not include reading aloud of passages or
questions on the NAEP reading assessment.
• Assessment must be completed in one day.
• Assessment is not available in Braille unless the
district or state provides it.
16
Analysis and Reporting
National, State, and Trial Urban District
Assessment (TUDA) results
17
–
NAEP scores are reported in two ways –
by Scale Scores and by Achievement
Levels
–
No student, classroom, or school results
are provided
–
No district results are provided except
for the TUDAs
–
Subgroup, state, and national
comparisons can be made
–
Relationships between student
proficiency and certain background
variables are available
NAEP Scale Scores
Scale scores provide a comparison of student
performance with regard to a group (e.g., the
nation) or a subgroup (e.g., ELL) on a set of items.
Scale scores:
•
Reading and Mathematics scale score range
is 0-500
•
Science and Writing scale score range
is 0-300
Scale scores are reported as an average for
groups of students and may obscure progress or
problems related to student performance across
distribution of scores.
18
2005 National NAEP Science
Average Scale Scores
Grades 4 and 8
National NAEP average science scale scores
were 4 points higher in 2005 than in 1996 at
Grade 4. At Grade 12, scale scores were 3
points lower. There has been no change at
grade 8. * indicates change is significant.
19
NAEP 2005
Hands-On Science Tasks
Approximately half of the students
assessed in science participated in
a hands-on task (kits).
Grade 4 – Seeds, Unknown Powder,
Markers
Grade 8 – Markers, Powders, Soil
Tests
Grade 12 – Soil Tests, Antacid,
Pendulum
20
NAEP 2005
Hands-On Science Tasks
• Students use the kits to conduct an
investigation and record their findings in
their booklet
• Kits are self-contained, including
whatever safety equipment may be
needed
• Students are instructed how to clean up
when they are finished with their kit
• Certain pieces of the kits are left at the
school for their use
21
Florida Compared to the Nation
NAEP 2005 Science Scale Scores
Grade 4
Of the 44 states and one jurisdiction that participated in the 2005
4th grade science assessment, students’ average scale scores in
Florida were higher than that of 10 jurisdictions, not significantly
different from that of 12 jurisdictions, and lower than that of 22 jurisdictions.
22
Florida Compared to the Nation
NAEP 2005 Science Scale Scores
Grade 8
Of the 44 states and one jurisdiction that participated in the 2005
8th grade science assessment, students’ average scale scores in
Florida were higher than that of 3 jurisdictions, not significantly different
from that of 8 jurisdictions, and lower than that of 33 jurisdictions.
23
NAEP
Achievement Levels
NAEP Achievement Levels (ALs) try to identify
what students should know and be able to do at
various achievement levels.
• The NAEP achievement levels are Advanced,
Proficient, Basic, and below Basic. Each is
determined by cutpoints established by professional
educators and NAGB.
• The number or percentage of students in the
population at large or in subpopulations can be
reported by the four achievement levels.
• Results show how different groups are performing in
relation to each other and over time.
24
Florida Compared to the Nation
NAEP 2005 Science at or Above Proficient
Grade 4
Of the 44 states and one jurisdiction that participated in the 2005
Grade 4 science assessment, the percentage of students scoring at or
above Proficient in Florida was higher than that of 8 jurisdictions, not
significantly different from that of 15 jurisdictions, and lower than that of
21 jurisdictions.
25
Florida Compared to the Nation
NAEP 2005 Science at or Above Proficient
Grade 8
Of the 44 states and one jurisdiction that participated in the 2005 Grade 8
science assessment, the percentage of students scoring at or above
Proficient in Florida was higher than that of 3 jurisdictions, not significantly
different from that of 11 jurisdictions, and lower than that of 30 jurisdictions.
26
Florida NAEP Achievement Levels
Grade 8 Science 1996 and 2005
Florida’s scores have remained unchanged since 1996.
27
Florida’s Performance Compared with the Nation
NAEP 2005 Science Grades 4 and 8
Percentage of States Florida Outperformed
Grades 4 and 8 in 2005 Based on
Race/Ethnicity
90
80
83
Grade 4
Grade 8
70
60
50
48
50
40
38
30
18
20
11
10
0
Hispanic
African
All
Students American 36 States
45 States 36 States Included
Included Included
28
African
All
Students American Hispanic
15 States 35 States 33 States
Included Included Included
Florida’s Performance Compared with
the Nation
NAEP 2005 Grade 8 Science
Percentage of States for which
Florida’s Change from 1996 to 2005
was Greater
by Race/Ethnicity
50
44
45
40
Percent
35
36
32
30
25
20
15
10
5
0
All Students
37 States Included
29
African American
28 States Included
Hispanic
16 States Included
2005 NAEP Grade 4 Science
Race/Ethnicity
Grade 4 Average Scale Scores for Florida
and the Nation by Race/Ethnicity
161
White
161
130
African
American
128
Florida
132
162
Asian/Pacific
Islander
156
0
30
Nation
144
Hispanic
50
100
150
200
2005 NAEP Grade 4 Science
Race/Ethnicity
• In 2005, the average scale score of African
American students was 31 points lower than the
average scale score of white students. In the
nation, the average scale score for African
American students was 33 points lower than the
average scale score of white students.
• In 2005, the average scale score of Hispanic
students was 17 points lower than the average
scale score of white students. This
performance gap was narrower than that of the
nation (29 points).
31
2005 NAEP Grade 8 Science
Race/Ethnicity
Grade 8 Average Scale Scores for Florida
and the Nation by Race/Ethnicity
155
White
159
118
123
African
American
131
Hispanic
Nation
127
149
Asian/Pacific
Islander
155
0
32
Florida
50
100
150
200
NAEP Grade 8 Science
1996 and 2005
Race/Ethnicity
Grade 8 Science Average Scale
Scores for 1996 and 2005 for Florida
and the Nation by Race/Ethnicity
180
160
154 155
158 159
140
117 118
120
119 123
130 131
128 127
100
1996
80
2005
60
40
20
0
FL White
33
Nation
White
FL African
American
Nation
African
American
FL
Hispanic
Nation
Hispanic
NAEP Grade 8 Science
1996 and 2005
Race/Ethnicity
• In 2005 and 1996, the average scale
score of Florida’s African American
students was 37 points lower than the
average scale score of white students.
The National gap declined from a 39
point difference in 1996 to a 36 point
difference in 2005.
• In 2005 and 1996, the average scale
score of Hispanic students was 24 points
lower than the average scale score of
white students. The National gap
increased from 30 points to 32 points.
34
NAEP AND FCAT
• FCAT – measures student
performance on selected
benchmarks, as defined by
Florida’s SSS
• NAEP – reports on the performance
of groups of students at a given
time - and across time - without
specifying how a subject should be
taught
35
2005 FCAT, Florida NAEP, and
National NAEP
Science Grades 4 and 8
Achievement Levels
FCAT – Level 3 and Above
NAEP – Proficient and Above
35
33
33
Grade 4
30
26
Grade 8
27
27
25
Percent
21
20
15
10
5
0
FCAT
36
FL NAEP National
NAEP
FCAT
FL NAEP National
NAEP
2005 FCAT, Florida NAEP, and
National NAEP
Science Grades 4 and 8
• Percentage of students performing
at Proficient and above in science is
very similar for FCAT Grade 5 and
Grade 4 Florida NAEP and National
NAEP.
• Percentage of students scoring at
Proficient and above on the Grade
8 science FCAT is greater than on
the Grade 8 Florida NAEP and
National NAEP.
37
2005 FCAT Grade 5 and
Florida NAEP and National NAEP Grade 4
Science
Race/Ethnicity
Achievement Levels
FCAT – Grade 5, Level 3 and Above
NAEP – Grade 4, Proficient and Above
33
26
27
All Students
45
38
38
White Students
24
17
Hispanic Students
10
12
African American
Students
7
7
0
10
National NAEP
38
20
30
Florida NAEP
50
40
FCAT
2005 FCAT Grade 5 and
Florida NAEP and National NAEP
Grade 4 Science
• Grade 4/5 students in all three
racial/ethnic groups had higher
average scale scores in science
and a greater percentage achieving
Proficient and above on the FCAT
than they did on Florida NAEP and
National NAEP
39
2005 FCAT, Florida NAEP, and National NAEP
Grade 8 Science
Race/Ethnicity
Achievement Levels
FCAT – Grade 8, Level 3 and Above
NAEP – Grade 8, Proficient and Above
33
All Students
21
27
46
White Students
32
38
24
14
Hispanic Students
10
13
African American
Students
6
7
0
10
National NAEP
40
20
30
Florida NAEP
40
FCAT
50
2005 FCAT , Florida NAEP, and
National NAEP
Grade 8 Science
• A greater percentage of Grade 8
white, African American, and
Hispanic students scored Level 3 or
above on the FCAT than scored
Proficient and above on Florida
NAEP and National NAEP
41
The Nation’s Report Card
http://www.nces.ed.gov/nationsreportcard/
42
NAEP Websites
Florida’s NAEP website:
http://firn.edu/doe/sas/naephome.htm
NAEP website:
http://nces.ed.gov/nationsreportcard/
NAEP Data Explorer:
http://nces.ed.gov/nationsreportcard/nde/
NAEP Questions Tool:
http://nces.ed.gov/nationsreportcard/itmrls
43
NAEP Coordinator
Contact Information
Michele Sonnenfeld
NAEP State Coordinator
Florida Department of Education
(850) 245-0787
SC 205-0787
FAX (850) 245-0781
SC 205-0781
[email protected]
44