Transcript ASCA National Model - Arkansas School Counselor Association

Accountability System
Using Data to Drive School
Counseling Activities
[email protected]
© 2003 by The Education Trust, Inc.
Mark Kuranz,
ASCA National Model
© 2003 by The Education Trust, Inc.
Accountability System

Results Report –
–
–
–
–
Program
Guidance Curriculum
Closing the Gap
Impact Over Time
School Counselor Performance Standards

The Program Audit
© 2003 by The Education Trust, Inc.

How are students different
as a RESULT of what you do?
 Results
Report –
© 2003 by The Education Trust, Inc.
–Program
–Guidance Curriculum
–Closing the Gap
–Impact Over Time
What Is Your Relationship
with Education Data?
Nonexistent?
Reactive?
© 2003 by The Education Trust, Inc.
Proactive?
The Power of Data


Provide objective snapshots of the
students, school, community
Surface evidence of access or equity
issues
Break old myths – eliminate denial

Create urgency/energy for change

Provide direction – data driven
decisions
© 2003 by The Education Trust, Inc.

The Power of Data



Use as an accountability tool
Focuses resources where they are
most needed
Supports grant writing efforts
© 2003 by The Education Trust, Inc.

Challenge existing behavior,
funding patterns, programs, &
policies
Using Data to Spur
Systemic Change

School Counselors must be proficient in:
–
–
–
–
School Counselors must use data to:
- Recognize barriers to learning
– Point out the system inequities
– Advocate for system change
– Create urgency for change
© 2003 by The Education Trust, Inc.

Accessing data
Analyzing data
Interpreting data
Presenting data
How Not to Use Data
To place blame

To focus on the past

To maintain the status quo
© 2003 by The Education Trust, Inc.

What Data Should
You Collect
© 2003 by The Education Trust, Inc.
Data has many gap closing and
diagnostic uses and can be
categorized differently
Demographic Data
What do you want to know?
© 2003 by The Education Trust, Inc.
Student demographics: what are the
characteristics of our students?
 Gender
 Ethnicity
 Socio-economic status (free/reduced
lunch)
 Limited English Proficiency
 Family configuration
 Mobility
Rhode Island Shifts in
Student Demographics
100%
90%
80%
11.5
1
4
7.5
13.1
14.8
1
3.2
7.7
1
3.2
8.1
70%
60%
50%
77.2
75.4
73.4
30%
20%
American Indian,
non-Hispanic
Asian and Pacific
Islander, nonHispanic
Black, nonHispanic
White, nonHispanic
10%
0%
1997
1999
2001
Source: US Census Bureau, Population Projections, in Education Week, September 27, 2000.
© 2003 by The Education Trust, Inc.
40%
Hispanic
Portland PS
Demographics Today
100%
2
90%
10
80%
11
70%
American Indian
60%
Asian and Pacific
Islander
50%
Hispanic
17
30%
60
Black
20%
White
10%
0%
2003
© 2003 by The Education Trust, Inc.
40%
Student Demographics
30
25
20
% Free Lunch
10
Pike High School
Indianapolis, IN
5
03-0
4
00-0
1
96-9
7
0
© 2003 by The Education Trust, Inc.
15
Attainment Data
What do you want to know?
© 2003 by The Education Trust, Inc.
Attainment: How many make it to - and
beyond key points in the system?
 Advance to next grade
 Transition from middle school to high
school
 Graduation rate
 Type of high school diploma
 Matriculation to an institution of
higher education
 Persistence beyond freshman year
 Earning a college diploma
Highest Educational Attainment
for Every 100 Kindergartners
Graduate from
High School
Obtain at Least a
Bachelor’s Degree
Asians
Latinos
Whites
87
90
63
93
50
74
32
65
18
51
11
33
(Age 25-29)
Source: US Department of Commerce, Bureau of the Census. March Current Population
Survey, 1971-2001, In The Condition of Education 2002
© 2003 by The Education Trust, Inc.
Complete at Least
Some College
African
Americans
Get Local Graduates
Public K-12
Enrollment
Totals
2001-02 NCES Data
158,046
6.3%
3.2%
8.9%
.4%
81.1%
8,603
© 2003 by The Education Trust, Inc.
African American 8.1%
Asian
3.2%
Hispanic
14.8%
Native American 1.0%
White
73.4%
Graduates
Ninth Grade Retention Rate Reduction as a
Result of School Counselor Interventions
2001
29.75%
13.47%
32.26%
21.85%
18.16%
24.54%
26.61%
49.28%
36.09%
36.97%
26.04%
20.20%
34.68%
67.74%
22.66%
36.20%
43.10%
2002
15.45%
9.09%
8.73%
20.56%
8.67%
12.57%
31.18%
38.99%
21.34%
35.51%
21.16%
11.67%
25.33%
41.18%
20.29%
29.58%
22.31%
© 2003 by The Education Trust, Inc.
JE FFE R S ON COU NTY S CH OOLS
ATH E R TON H I G H
BALLAR D H I G H
CE NTR AL H I G H
D OS S H I G H
E AS TE R N H I G H
FAI R D ALE H I G H
FE R N CR E E K TR AD I TI ONAL H I G H
I R OQU OI S H I G H
JE FFE R S ONTOWN H I G H
MOOR E TR AD I TI ONAL H I G H
PLE AS U R E R I D G E PAR K H I G H
S E NE CA H I G H
S H AWNE E H I G H
S OU TH PAR K TAPP
S OU TH E R N H I G H
WAG G E NE R TR AD I TI ONAL H I G H
WE S TE R N H I G H
0
30
20
2001
2002
10
© 2003 by The Education Trust, Inc.
Western
Waggener
Southern
S. Park
Shawnee
Seneca
Pleasure
Moore
Jeff
Iroquois
Fern
Fairdale
Eastern
Doss
Central
Ballard
Atherton
Ninth Grade Retention Rate Reduction as a
Result of School Counselor Interventions
70
60
50
40
Achievement Data
What do you want to know?
Achievement: What does achievement look like at
different levels and with different groups of students?
 Overall Achievement
– Grade point average

© 2003 by The Education Trust, Inc.

– Standardized test scores, SAT, ACT, State tests
– Passing all subjects
Periodic assessment
– Semester grades
– End of course tests
Ongoing classroom assessment
– Class assignment grades
– Tests
Achievement-Related Data






© 2003 by The Education Trust, Inc.

Course enrollment patterns
Discipline referrals
Suspension rates
Alcohol, tobacco and other drug violations
Attendance rates
Parent involvement
Extracurricular activities
MPS Grade 9 Reading
MBST Grade 9 Passing Reading
80
60
40
20
0
© 2003 by The Education Trust, Inc.
y
ry on rt h elt ut h est rn
a
n
u dw
e dis No ev o
b
w
H E
S t h sh
s
a
o
o
u
o
a Br
R
So W
MBST Grade 9
Passing Reading
African American and Latino
17 Year Olds Do Math at Same Levels
As White 13 Year Olds
100%
0%
250
300
350
White 8th Graders
African American 12th Graders
Latino 12th Graders
Source: NAEP 1999 Long Term Trends Summary Tables (online)
© 2003 by The Education Trust, Inc.
200
State Data
Black eight-graders and white fourthgraders had almost identical scores in
math on a national standardized test
in 2003.
© 2003 by The Education Trust, Inc.
Milwaukee Journal Sentinel, May 23, 2004
Student Behavior Data
What do you want to know?
© 2003 by The Education Trust, Inc.
Student Behavior Choices: What are
students doing?
 Attendance
 Discipline referrals
 Classroom behavior
 Homework completion
 Enrollment patterns
– Algebra in 9th grade
– Upper level math and science
– Honors, AP, college credit
Low-Income Students are Less Likely to
be Enrolled in a College Prep. Track
Percent Enrolled
90
65.1
48.8
20
Socio-Economic Status
Low
Medium
High
Source: US Department of Education, National Center for Education Statistics. National Education Longitudinal Study of 1988:
Second Follow-Up, 1992 in: A Profile of the American High School Senior in 1992. (p. 36) Washington, DC: US Department of
Education, June 1995.
© 2003 by The Education Trust, Inc.
28.3
Even Low Achieving
Students Go On To
Postsecondary
% Entering
Postsecondary*
58%
Middle
75%
High
93%
Overall
76%
* Postsecondary includes any college, university, vocational, or
technical education entered by 2000.
Source: USDOE, NCES. Coming of Age in the 1990s: The Eighth-Grade Class of 1988 12 Years Later, NCES
2002-321. Washington, DC:2002.
© 2003 by The Education Trust, Inc.
Eighth Grade
Achievement Level
Low
Opportunity Gap Data
What do you want to know?
Opportunity Gaps: Are some students
provided more opportunities than
others?




© 2003 by The Education Trust, Inc.

Access to rigorous curriculum
Access to quality teachers
School policies and climate
Special Education Screening
and Placement
Participation in Support Services
– Tutoring, Mentoring
Equity?
© 2003 by The Education Trust, Inc.
Program Evaluation Data
What do you want to know?
 Process
© 2003 by The Education Trust, Inc.
data
 Perception data
 Results data
Process data
What do you want to know?



Evidence that event occurred
How activity was conducted
Did the program follow the prescribed
practice?
© 2003 by The Education Trust, Inc.

“What you did for whom”
Process Data - Examples


586 9th grade students received the
“The Four Year Plan” guidance lessons
All 4th and 5th (112) grade students
participated in the “bus buddy” (4)
guidance lessons
© 2003 by The Education Trust, Inc.

Weekly (32) academic support groups
with 12 students each were held
Adding Process Data
Perception Data
Pre
Activity
Process Data
Question or
Demonstration
D
a
t
e
Results Data
Post
D
# a
% t
e
#
%
How will
student
behavior
change?
Pre
D
a
t
e
Post
D
# a
% t
e
#
%
© 2003 by The Education Trust, Inc.
Perception Data
What do you want to know?


“What others think, know or demonstrate”
data.
Measures competency achieved, knowledge
gained or attitudes beliefs of students

Pre-post
Competency achievement
Surveys
Evaluations
Measures what students are perceived to have
gained in knowledge
© 2003 by The Education Trust, Inc.
–
–
–
–
Perception Data - Examples

Competency Achievement
– Every student in grades 9-12 completed a 4 year plan
– Every 10th grade student completed an interest inventory

Knowledge Gained

Attitudes or Beliefs
– 74%of students believe fighting is wrong
– 29% of students feel safe at school
– 78% know the name of their school counselor
© 2003 by The Education Trust, Inc.
– 89% of students demonstrate knowledge of promotion/
retention criteria
– 92% can identify Early Warning Signs of violence
Adding Perception Data
Perception Data
Pre
Activity
Process Data
Question or
Demonstration
D
a
t
e
Results Data
Post
D
# a
% t
e
#
%
How will
student
behavior
change?
Pre
D
a
t
e
Post
D
# a
% t
e
#
%
© 2003 by The Education Trust, Inc.
Results Data
What do you want to know?




“So WHAT” data
– Attendance
– Behavior
– Academic achievement
© 2003 by The Education Trust, Inc.
Hard data
Application data
Proof your program has (or has not)
positively impacted students ability to
utilize the knowledge, attitudes and skills
to effect behavior
Results Data - Examples


Graduation rates improved 14% over
three years
Attendance improved among 9th grade
males by 49%
© 2003 by The Education Trust, Inc.

42 students avoided retention
Adding Results Data
Perception Data
Pre
Activity
Process Data
Question or
Demonstration
D
a
t
e
Results Data
Post
D
# a
% t
e
#
%
How will
student
behavior
change?
Pre
D
a
t
e
Post
D
# a
% t
e
#
%
© 2003 by The Education Trust, Inc.
ACADEMIC
Academic Results
Interventions (6-8)
Career
Personal/ Social
120
100
80
60
on retention
off retention
20
0
6th
7th
8th
grade grade grade
72 students avoided retention
© 2003 by The Education Trust, Inc.
40
Possibilities…
There are schools that show things
can be different? Find schools like
yours that have:

Closed the opportunity gaps
Closed the achievement gap
© 2003 by The Education Trust, Inc.

Resources





© 2003 by The Education Trust, Inc.

American School Counselor Association –
www.schoolcounselor.org
Education Trust web Site www.edtrust.org
State Department of Education
Local School Districts
No Excuses - Heritage Foundation
www.noexcuses.org
National Center for Educational Statistics
– nces.ed.gov
© 2003 by The Education Trust, Inc.
Learn to Analyze
and Interpret Data
Data Analysis:
Keep It Simple
Descriptive Data Powerful

Numbers and percentages



The benchmark is ALL
Usually working with entire population
Easier for educators and public to
understand
© 2003 by The Education Trust, Inc.
Rationale for Use
Basic Ways to Analyze Data

Start with simple statistics - averages, %’s
Spencer Owen School Corporation ISTEP Tests
Math
Eng/LA
02-03
© 2003 by The Education Trust, Inc.
100
90
80
70
60
50
40
30
20
10
0
Basic Ways to Analyze Data
Whi
te
Hisp
anic
Asia
n
Blac
k
mer
ican
Nati
ve A
Student
Staff
© 2003 by The Education Trust, Inc.
70%
60%
50%
40%
30%
20%
10%
0%
Data Over Time
What do you want to know?
 Immediate
 Intermediate
range (Impact Over Time)
© 2003 by The Education Trust, Inc.
 Long
Data Over Time Examples

Immediate – data measures
the immediate impact
- pre-post test, p.70-71
Intermediate
Long-range
– Longitudinal
- data “stretch over time”, p.73
© 2003 by The Education Trust, Inc.
– data collected
over a short period of time
- improved grades after
counseling group, p. 72-73
Basic Ways to Analyze Data


Start with simple statistics - averages, %’s
Longitudinal = “stretching” data out over time
Spencer Owen School Corporation ISTEP Tests
Math
English
98-99
99-00
00-01
01-02
02-03
© 2003 by The Education Trust, Inc.
100
90
80
70
60
50
40
30
20
10
0
Bus Buddy Program Results
Bus Referrals
97-98 98-99 99-00 00-01 01-02 02-03
© 2003 by The Education Trust, Inc.
100
90
80
70
60
50
40
30
20
10
0
Basic Ways to Analyze Data



Start with simple statistics - averages, %’s
Longitudinal
Disaggregate= “slicing” a piece of data
--Victoria Bernhardt
© 2003 by The Education Trust, Inc.
The Power of Disaggregated Data
Disaggregation is not a
problem-solving
strategy. It is a
problem-finding
strategy
Basic Ways to Analyze Data



Start with simple statistics - averages, %’s
Longitudinal
Disaggregate= “slicing” a piece of data
African American Asian
32%
52%
Latino Native American White
31%
27%
66%
© 2003 by The Education Trust, Inc.
Average MPS Graduation Rate
46%
Basic Ways to Analyze Data
Disaggregate
70%
60%
50%
Graduation
Rate
te
W
hi
n
H
is
pa
ca
ni
n
c
Am
er
ic
an
As
ia
Af
ri
iv
e
Am
er
ic
an
0%
© 2003 by The Education Trust, Inc.
40%
30%
20%
10%
N
at

© 2003 by The Education Trust, Inc.
Examples of Data to Examine
Test Scores
Achievement
State
National
Enrollment
Honors/AP Classes
 College Track
 Special Education
 LEP

Attendance
 Absences
Retention Rates

By Subject Area
 By Grade Level
 By Gender, Ethnicity
Post Secondary
Plans

By Classroom
 Types of Problems
 Gender

By Gender
 By Ethnicity
 By SES

GPA/Class Rank
By Gender
 By Ethnicity
 By SES

Special Education
 By Gender

By Ethnicity
 By SES
Dropout Rate
Grade Levels
 Gender, Ethnicity…
 Reasons Why

© 2003 by The Education Trust, Inc.
Tardies
 By Grade Level

Discipline
Graduation Rate
Data Interpretation
Look for:
Pictures Patterns Gaps


© 2003 by The Education Trust, Inc.

Questions:
What problems or needs surface?
What achievement gaps exist?
What opportunity gaps do the
data suggest?
Uncovering the
Opportunity Gaps




© 2003 by The Education Trust, Inc.

Access to challenging courses
Access to support services
Access to resources
Percentage of high quality teachers
in school
Distribution of teacher talent
Percentage of High Scoring Students
Placed in Algebra in One Southern
California School District
100%
100%
51%
88%
83%
50%
42%
11%
0%
Top Quartile CTBS
African American
Second Quartile CTBS
Asian
Source: The Achievement Council, Inc. Los Angeles, CA. Unpublished. 1991
Latino
White
© 2003 by The Education Trust, Inc.
16%
Equity?
© 2003 by The Education Trust, Inc.
Data Driven Counseling Programs
70% Attendance Rate for
Low SES Students
Small
Group
Mentors
Tutoring
Bully
Proofing
Program
Individual
Counseling
Phone
Contact
Manager
Of
Resources
Classroom
Guidance
Behavior
Management
© 2003 by The Education Trust, Inc.
Student Focused
Academic Achievement –
Student Focused
AVERAGE GRADE
INCREASE - 18.2%
60
67.28
58.49
45.52
40
20
0
Semester 1
Quarter 3
Semesters
AVERAGES
Quarter 4
© 2003 by The Education Trust, Inc.
Grades
80
Data Driven Counseling Programs
Change
Daily
Schedule
Student
Focus
Groups
System Focused
Advisory
Program
Change
Attendance
Policies
Advocate
for
Task
Force
Team
With
Parents &
Community
© 2003 by The Education Trust, Inc.
70% Attendance Rate for
Low SES Students
Lead
Lead
Advisory
Disaggregate
Task
Training
Data By
Force
Teacher
Highlights of Achievement


The Protective Schools model was
implemented at all eight schools.
CCBG program was implemented in
every school
© 2003 by The Education Trust, Inc.

Student to counselor ratio at CLEAR
schools was reduced from 641/1 to
354/1.
Longitudinal Study
Questions
1) Will the use of the Protective School’s Model have
any impact on academic progress and attendance for
the participating students?
© 2003 by The Education Trust, Inc.
2) Will lowering the student to counselor ratio and
fully implementing the CCBG program have an
increase on academic progress and attendance for the
students?
Reading
district
project students
2000-01 2001-02 2002-03
2000-01 – 3rd grade,
2001-02 – 4th grade,
2002-03 – 5th grade
© 2003 by The Education Trust, Inc.
54
53
52
51
50
49
48
47
46
45
44
Math
56
54
52
50
district
project students
48
46
42
40
2000-01 2001-02 2002-03
2000-01 – 3rd grade,
2001-02 – 4th grade,
2002-03 – 5th grade
© 2003 by The Education Trust, Inc.
44
Math
56
54
52
50
district
project students
48
46
42
40
2000-01 2001-02 2002-03
2000-01 – 3rd grade,
2001-02 – 4th grade,
2002-03 – 5th grade
© 2003 by The Education Trust, Inc.
44
Attendance
97
96
96
95
95
district
project students
94
94
93
92
2000-01 2001-02 2002-03
2000-01 – 3rd grade,
2001-02 – 4th grade,
2002-03 – 5th grade
© 2003 by The Education Trust, Inc.
93
School Climate Results
Examples of preliminary gains are as
follows:
 Teachers reported an increase in student
safety at the project schools. (Source: TUSD
School Quality Survey)
Students responded that students of
different races & ethnic backgrounds are
getting along better at their school.
(Source: TUSD School Quality Survey)
© 2003 by The Education Trust, Inc.


Parents indicated an increase in
satisfaction with their school. (Source: TUSD
School Quality Survey)

March and April 2003)
© 2003 by The Education Trust, Inc.
All Principals indicated increased contact
with students and a more consistent
program (in classrooms, groups and
interventions) with increased counseling
staff. Principals consider the counselor
indispensable. (Source: Principal interviews –

each school2003)

my/03
© 2003 by The Education Trust, Inc.
5th grade students reveal that they see
positive school changes, have gained
many academic and personal skills from
their counselor and have a personal vision
that includes continuing their education.
(Source: Focus groups conducted in March and April at
Principal Comments


© 2003 by The Education Trust, Inc.

Principals saw counselor as the
primary change agent for the school
Principals understood how the
counselor can contribute to student
achievement
Principals acknowledge that the school
counselor knows every student and is
the soul or heart of the school.
Minority AP Enrollment in One Jefferson
County High School - System Focused
© 2003 by The Education Trust, Inc.
Results Report
How are students different
as a RESULT of what you do?




© 2003 by The Education Trust, Inc.

What does the data tell you?
Was the program successful?
What worked?
What did NOT work?
What needs to be changed?
Results Report- A Tool
For

Advocate for systemic change





© 2003 by The Education Trust, Inc.

Ensuring program was carried out
Every student was served
National Student Standards are addressed
Developmentally appropriate
Documenting process, perception, results data;
immediate, intermediate, and long range impact of
program
Analyze effects
Share successes

Results Report Examples

Closing the Gap Results Report, p. 117
© 2003 by The Education Trust, Inc.

Guidance Curriculum Results Report,
p.118
Accountability …
Getting Started


Use data that are already being
collected such as attendance,
behavior, grades, or other scores.
Keep it simple – percent change,
pre to post changes.
© 2003 by The Education Trust, Inc.

Use what you are already doing
(e.g., small group or one
classroom).
Accountability …
Getting Started




© 2003 by The Education Trust, Inc.

Show several years of growth
Connect to student standards
Don’t measure EVERYTHING
Make decisions based on needs of
district, site and access to data
Goal is to learn from data, not be
evaluated on data
We need to “be the change
we want to see happen.
We are the leaders
we have been waiting for.”
© 2003 by The Education Trust, Inc.
– Gandhi
“Without the possibility of
action, all knowledge comes
to one labeled ‘file and
forget,
-- Ralph Ellison, Invisible Man
© 2003 by The Education Trust, Inc.
and I can neither file nor
forget.”
Accountability System
Using Data to Drive School
Counseling Activities
[email protected]
© 2003 by The Education Trust, Inc.
Mark Kuranz,