Transcript Stricom 1.21.9 C - National Center for Research on

C RE SS T/U C LA
1
Quality School Portfolio
Decision Support System:
Transforming Data into Information
John J. Lee
Senior Researcher
UCLA Graduate School of Education & Information Studies
Center for the Study of Evaluation (CSE)
National Center for Research on Evaluation, Standards, and Student Testing (CRESST)
C RE SS T/U C LA
2
The Quality School Portfolio
(QSP)
 Without tools to investigate and understand
results, more testing will not impact
educational practice
 Software for use by educators to organize,
collect, interpret, and respond to complex data
 UCLA, OERI, Stuart, and Joyce Foundation
support
C RE SS T/U C LA
3
QSP—Making Use of Testing
and Other Information
Data Manager
Resource Kit
C RE SS T/U C LA
4
QSP Goals
 Transform test and other results into usable,
actionable form
 Create coherent information structure for all
members of the educational community
(policymakers, administrators, teachers, and
parents)
 Encourage using information as the basis for
making decisions about educational options
and services
 Raise capacity and efficacy of educators
 Communicate findings in intuitive displays
C RE SS T/U C LA
Learning to Use Results:
QSP Features
 Relies on external and internal sources of
information
 Easy access and display
 Custom queries
 Disaggregation
 Report preparation
5
C RE SS T/U C LA
6
Applying QSP
 Use for school monitoring and improvement
 For distance learning assessment reporting
 For distributed program evaluation (LA’s BEST;
Artful Learning)
 Disaggregation
 Individual longitudinal records
C RE SS T/U C LA
7
Disaggregation
How can the school focus on particular populations?
System Groups
Custom Groups
Combo Groups
C RE SS T/U C LA
8
Conditional Disaggregation:
Eight Filters
 Hispanic +
 Boys +
 9th Grade +
 Spanish L1 +
 Redesignated English
 Magnet schools +
 GPA below 2.0 +
 Less than 2 years in the school
C RE SS T/U C LA
9
Gauge Display
C RE SS T/U C LA
10
Scatterplot
C RE SS T/U C LA
Quick School Comparisons
11
C RE SS T/U C LA
School Performance Reports
12
C RE SS T/U C LA
Resource Kit Menu
13
C RE SS T/U C LA
14
Resource Kit Functions
 Clustered topics of resources
 Selection or modification of instruments in a
cluster given a description
 Supports Acrobat Reader and Rich Text
Formats
 Customized entry of standard surveys, etc., to
permit local use of evaluation data and
simultaneous export of information to
centralized sources
C RE SS T/U C LA
15
QSP Resource Kit
 Local data to answer local questions
 Staff and student surveys, interview and
observation protocols
 Data entry mechanisms, e.g., student work—
in process (digital portfolios)
 Online assessments or surveys
 Transfer into Data Manager
C RE SS T/U C LA
16
Resource Kit Summary Page
C RE SS T/U C LA
17
Extensions
 QSP Web (currently in development)
 Data analysis from descriptive to growth
projection
 Flexible report wizard
 Improved Web training component
C RE SS T/U C LA
18
User Challenges
 Learning to ask productive questions
 Looking at measures over time
 Level of technical sophistication, e.g.,
hierarchical models, trajectories
 Reports appropriate to various audiences
and stakeholders
C RE SS T/U C LA
Continuing Research: A
Sample of Impact Questions
 Who is helped? What kinds of schools,
teachers, and students? How do experience,
beliefs, knowledge condition impact?
 Do administrative practices change at school,
district and board levels?
 What happens in classrooms? Is student
growth affected? For how long? What kind of
combination of information is needed
(CRT/NRT, process)?
19