Transcript Surveys of the Enacted Curriculum

Surveys of the
Enacted Curriculum
Cary Cuiccio
Lisa Palacios
March 6, 2009
Copyright © 2008 Learning Point Associates. All rights reserved.
Learning Point Associates
Our Vision
An education system that works for all learners.
Our Mission
We deliver the knowledge, strategies,
and results to help educators make
research-based decisions that produce
sustained school improvement.
2
www.learningpt.org
Research has found that faculty in successful
schools always question existing
instructional practice and do not blame lack
of student achievement on external causes.
— Carl Glickman,
2002
www.learningpt.org
What if?
…there was a powerful tool that compared teacher’s
reflections of content instruction to state
standards, benchmarks and assessments?
… you could compare how your teachers teach
content compared to how others teach across the
nation?
… you could have consistency across grade levels on
both time teachers spend on content as well as
depth of instruction?
www.learningpt.org
What if?
… there was a tool that compared teacher’s reflected
instructional practices to other teachers’ across a
school, district or the nation?
… you could use data on instructional quality and
content to guide professional development?
… you could use anonymous teacher data to start a
powerful school discussion about teachers’
curricular and professional development needs
based on data and observed differences between
student achievement scores and schools
curriculum?
www.learningpt.org
Introducing: The SEC
 Online web-based surveys in Mathematics, English
Language Arts, Social Studies and Science.
 Groups data that are reported in user-friendly
charts.
 Assists in facilitating group discussions for
improved instruction across classrooms, schools
and districts.
 A practical, reliable set of data collection tools
used by teachers.
www.learningpt.org
7
www.learningpt.org
SEC Collaborative
• State Departments of Education (selected)
• National Organizations (College Board,
ACT, NAEP, WIDA)
• Council of Chief State School Officers
• Wisconsin Center for Education Research
• Learning Point Associates
• TERC
www.learningpt.org
History of the SEC
The Surveys of Enacted Curriculum (SEC)
Collaborative Project was initiated in 1998.
Surveys are available K-12 for Mathematics,
Science, English Language Arts and
Reading, and Social Studies.
www.learningpt.org
History of SEC (cont.)
Over the past ten years the Council of Chief
State School Officers (CCSSO) and their
partners have worked with states to
implement the SEC tools for data collection,
analysis, and reporting.
www.learningpt.org
Research on the SEC
• Reform-Up-Close
(Porter, Kirst, Osthoff, Smithson, Schneider, 1993)
Validation of teacher self-report survey data.
• Upgrading Mathematics
(Gamoran, Porter, Smithson, White, 1997)
First content analysis of assessment using content language.
Predictive validity of alignment index comparing instruction & assessments
• Data on Enacted Curriculum
(Blank, Porter, Smithson, 2004)
Use of SEC data to facilitate school improvement efforts
First content analysis of state standards
• MSP-PD Study
(Blank, Smithson, Porter, Nunnaley, Osthoff, 2006)
Use of SEC data for program evaluation
www.learningpt.org
The ELA Survey Part One:
Instructional Content and
Cognitive Demand
Copyright © 2008 Learning Point Associates. All rights reserved.
Instructional Content
A tool to address a key question:
Is classroom instruction aligned with state
content standards and assessments?
• Content Emphasis
• Cognitive Expectations
www.learningpt.org
Instructional Topics
Topics provide the framework for data
collection and comparison.
Topics include the range of topics found in
English Language Arts classrooms, K–12.
Stable framework—not intended for a
particular state or curriculum; rather with
the content area in mind generally.
Neutral matrix—intends to include all
possible topics that could be taught.
www.learningpt.org
www.learningpt.org
K-12 English Language Arts Reading
Content Areas
100 Phonemic awareness
1000
Elements of Presentation (Verbal
and Written)
200 Phonics
1100
Writing applications
300 Vocabulary
1200
Language Study
400 Text and print
features
1300
Listening and Viewing
500 Fluency
1400
Speaking and Presenting
600 Comprehension
1500
Forms of Text
700 Critical Reasoning
1600
Genre (fiction or non-fiction)
800 Author's craft
1700
Sources of Text
900 Writing processes
1800
Choice
www.learningpt.org
www.learningpt.org
K-12 English Language Arts Reading
Specific Content
600
Comprehension
601
Word meaning from context
602
Phrase
603
Sentence
604
Paragraph
605
Main idea(s), key concepts, and sequence(s) of events
606
Descriptive elements (e.g., detail, color, condition)
607
Narrative elements (e.g., events, characters, setting, and plot)
608
Persuasive elements (e.g. propaganda, advertisement, and emotional appeal)
609
Expository or informational elements (e.g., explanation, lists, and organizational patterns such as description,
cause-effect, and compare-contrast)
610
611
Technical elements (e.g., bullets, instruction, form, sidebars, etc.)
Electronic elements (e.g., hypertext links, animations)
612
613
Strategies (e.g., activating prior knowledge, questioning; making connections, predictions; inference, imagery,
summarization, re-telling)
Self-correction strategies (e.g., monitoring, cueing systems, and fix-up)
614
Metacognitive processes (e.g., reflecting about one's thinking)
615
Interpreting maps, graphs, charts
616
Test-taking strategies
690
Other
www.learningpt.org
Cognitive Expectations
State standards and assessments have
inherent cognitive expectations.
Instructional activities also have
cognitive expectations.
The SEC measures the cognitive
expectations of instruction.
www.learningpt.org
Cognitive Expectations
English Language Arts and Reading
Memorize and Recall
Perform Procedures and Explain
Generate, Create, and Demonstrate
Analyze and Investigate
Evaluate
www.learningpt.org
Data Interpretation
Part One:
Instructional Content/Cognitive
Demand aligned to State Standards
• Contour Maps
• Tile Charts
Copyright © 2008 Learning Point Associates. All rights reserved.
Survey Instrument: Alignment
Subject Content
Student Specific Topics
Time Spent on Topic
Student Expectations
www.learningpt.org
Reading SEC Contour Maps
State J Grade 8
Mathematics Instruction
Number Sense
Operations
Measurement
Algebraic Concepts
Geometric Concepts
Data Analysis
0.1
0.08
0.06
0.04
0.02
0
Instructional Technology
Memorize Communicate Connect
Perform
Conjecture
Memorize
Communicate
Connect
Perform
Conjecture
www.learningpt.org
www.learningpt.org
The map on this side will be
your data, and only you can
access it. That’s why this data
is from an entire grade.
This map is the NY state
standards specific to your
grade. You can also look at
the NY state assessments.
Alignment describes
how well your
instruction aligns to
standards or
assessments. 1.0 is
perfect; anything
above .5 is fine.
This data is
from 134
Grade 3
teachers.
You read this like
a topographical
map. Darker
areas are peaks
and represent
more time spent
on that topic at
that cognitive
demand level.
Here, the 3rd
grade teachers
are spending
more time on
phonemic
awareness and
phonics, and at
more levels of
cognitive
demand, than
the standards
require.
You can also click
on any subtopic
and get another
map comparing
your teaching to
standards and/or
assessments at a
“fine grain” level.
Same here in
terms of
topographical
maps. Teachers
are spending less
time on speaking
and presenting
www.learningpt.org
than the
standards require.
www.learningpt.org
www.learningpt.org
www.learningpt.org
www.learningpt.org
www.learningpt.org
www.learningpt.org
www.learningpt.org
www.learningpt.org
www.learningpt.org
www.learningpt.org
www.learningpt.org
Activity
Copyright © 2008 Learning Point Associates. All rights reserved.
The Survey Part Two:
Instructional Practices
Copyright © 2008 Learning Point Associates. All rights reserved.
Instructional Practices
A tool to address key questions:
Are instructional practices optimized for
student achievement?
Are the professional development efforts
aligned with needs and priorities?
Are teachers’ opinions and belief in line with
district and school initiatives?
www.learningpt.org
Instructional Practices
 Homework
 Instructional influences
 Classroom
assessments
 Professional
development
 Opinions and beliefs
 Personal
characteristics
 Classroom
Instructional
preparation
 Formal course
preparation
www.learningpt.org
Taking the Survey
Copyright © 2008 Learning Point Associates. All rights reserved.
www.seconline.org
www.learningpt.org
Completing the Online Survey
Approximately 60-90 minutes to compete.
High-speed Internet connection required.
May be completed in multiple sittings.
Data are saved as each section is
submitted.
www.learningpt.org
Data Interpretation
Part Three:
Powerful Discussions
Copyright © 2008 Learning Point Associates. All rights reserved.
And so you just threw everything together? Matthews, a posse is
something you have to ORGANIZE.
www.learningpt.org
What does the SEC show?
Intersection of what is taught with what is tested.
Assessment
Taught,
tested, and in
the standards.
Instruction
Standards
Intersection of what
is tested with what
is in the standards.
Intersection of what is taught with what is in standards.
www.learningpt.org
Traditional Model for
School Improvement
Identify
Shortcomings
in Student
Performance
Identify “Best
Practices” for
Achievement
Problems
www.learningpt.org
Better School Improvement
Identify
Shortcomings
in Student
Performance
The Surveys
of Enacted
Curriculum
Review
Instructional
Program Data
Relative to
Concerns
Identify
Appropriate
Programmatic
Solutions
www.learningpt.org
Applications for SEC
Aligning standards, assessments,
and curriculum
Improvement of instruction
Interpreting student assessment results
Needs assessment and program evaluation
Indicators system for monitoring progress
www.learningpt.org
Possible Data Analysis
Outcomes
Targeted professional development
Changes in environmental factors
Positive climate of change
Continuous improvement process
Review of curriculum materials
Other data-driven decisions and actions
www.learningpt.org
What SEC isn’t -Individual Teacher Evaluation
• The “wall” of 3
A tool to judge alignment without
school/district context and/or discussion
with staff.
The only piece of data for measuring
alignment of teacher instruction.
www.learningpt.org
“Regardless of what a state policy requires
or what a district curriculum spells out,”
says Andrew C. Porter of Vanderbilt
University, “the classroom teacher
ultimately decides” what to teach.
Education Week, October 8, 2003
www.learningpt.org
Think, Pair, Share
Discuss what you’ve learned with the person
next to you and make a few notes.
• What was the purpose today?
• Similarities in your thoughts, perspectives?
• Based on what you read – how could the SEC
help to improve your classroom, school or
district?
www.learningpt.org
Cary Cuiccio
P: 718-483-4279
E-Mail: [email protected]
Lisa Palacios
P: 630-649-6601
E-Mail: [email protected]
54
www.learningpt.org