Transcript programs.ccsso.org

Surveys of Enacted Curriculum:
An Introduction to Accessing
and Understanding SEC Data
Survey of Enacted Curriculum—http://www.seconline.org
© 2004 CCSSO All rights reserved.
SEC Collaborating Organizations
•Council of Chief State School
Officers www.SECsurvey.org
•Wisconsin Center for Education
Research www.SEConline.org
•Learning Point Associates/NCREL
www.ncrel.org/sec
•TERC Regional Alliance DEC
Project www.ra.terc.edu/DEC
S2
History
• The Survey of Enacted Curriculum
Collaborative Project was initiated in
1998 in mathematics and science.
• Over the past six years CCSSO and
partners have work with state agencies
to implement the SEC tools for data
collection, analysis, and reporting.
S3
• In 2003, working with state specialists,
Learning Point Associates and WCER,
the collaborative developed an English
Language Arts version of the survey.
• Surveys are available K-12 for
Mathematics, Science, and English
Language Arts and Reading
S4
Survey Formats
Printed
Web-based
S5
Results Formats
• Online graphic representations
• Printed graphic representations
• Raw data
S6
The Survey Returns Data to Make
Visible:
• Teachers’ Professional Setting
• Instructional Practice
• Content Alignment
S7
Survey Instruments – Teachers
• Teacher Characteristics
– Teacher Readiness
– Professional Development
Experiences
– Influences on Curriculum
– Teacher Beliefs
• Class Description
S8
Survey Instrument – Practice
• Instructional Practices (Math)
– Scales – A
– Use of Homework – E
– Classroom Activities– F
– Problem-Solving– G
– Small Group Work – H
– Hands-On – I
– Use of Assessments – J
S9
Survey Instrument – Alignment
• Subject Content
– Subject Specific Topics
– Time Spent on Topic
– Level of Cognitive Demand
S 10
Key Education Questions
• What progress have we made in
implementing mathematics state
standards?
• ANSWER: Content
Alignment Results of
Survey of Enacted
Curriculum
S 11
Key Education Questions (cont’d)
• What do we know about our
instructional program? How can we
create a baseline of data as an
evaluation of our MSP or Reading First
program?
• ANSWER: Instructional
Practices Results of the SEC
Charts F, G, H, I, J
S 12
Key Education Questions (cont’d)
• What do we need to know about our
instructional program to help us understand
why our performance is low?
• ANSWER:
– Alignment of Enacted Curriculum to
Standards and Assessments
– Charts L (PD in Content area),
– M (Influences on Instructional
Practices),
– N (Teacher Course-Taking),
– O & P (Teacher Readiness),
– Q (Beliefs)…
S 13
Key Education Questions (cont’d)
• Did our professional development
positively impact our instructional
program in our MSP or Reading First
program?
• ANSWER:
– Practice: Charts F,G, H, I, J, K…
– Beliefs
– Alignment
S 14
Alignment
Assessment
Standards
Curriculum
S 15
Surveys of Enacted Curriculum
The intended
curriculum:
State content
standards—
What students
should learn
A neutral content grid
The assessed
curriculum:
State (and other)
assessments—
tested learning
with cognitive demand
The enacted
curriculum:
What teachers
teach
The learned
curriculum:
Student outcomes
based on school
learning
Slide: Created By Learning Point Associates, 2003 S 16
The SEC Content Topics
Mathematics, Science, English Language Arts
The SEC provides a
neutral, researchbased language to
describe content of
English language
arts, mathematics,
and science.
Slide: Created By Learning Point Associates, 2003
S 17
The SEC Content Topics
Mathematics, Science, English Language Arts
Review the list of topics
presented for the
current Content Area.
For each topic in the list
that is taught to the
target class, select a
radio button
corresponding to 1,2, or
3 based on the
following definitions:
0 = Not covered
1 = Less than 1 lesson
2 = 1-5 lessons
3 = more than 5 lessons
S 18
Capturing the Cognitive Expectations
For each topic selected
from the previous
screen set the cognitive
expectations for
students for each of 5
categories of cognitive
demand, using the
following definitions
0 = No emphasis
1 = Slight emphasis
2 = Moderate emphasis
3 = Sustained emphasis
S 19
Expectations for Students in Mathematics
Memorize Facts/ Definitions/
Formulas
Recite basic mathematics facts
Recall mathematics terms & definitions
Recall formulas and computational
procedures
Perform Procedures
Use numbers to count, order, denote
Do computational procedures or
algorithms
Follow procedures/instructions
Solve equations/formulas/routine word
problems
Conjecture/ Generalize/ Prove
Determine the truth of a mathematical
pattern or proposition
Write formal or informal proofs
Recognize, generate or create patterns
Find a mathematical rule to generate a
pattern or number sequence
Make and investigate mathematical
conjectures
Identify faulty arguments or
misrepresentations of data
Reason inductively or deductively
Organize or display data
Read or produce graphs and tables
Execute geometric constructions
Demonstrate Understanding
of Mathematical Ideas
Communicate mathematical ideas
Use representations to model
mathematical ideas
Explain findings and results from data
analysis strategies
Develop/explain relationships between
concepts
Show or explain relationships between
models, diagrams, and/or other
representations
Solve Non-routine Problems/
Make Connections
Apply and adapt a variety of appropriate
strategies to solve non-routine problems
Apply mathematics in contexts outside of
mathematics
Analyze data, recognize patterns
Synthesize content and ideas from several
sources
S 20
Expectations for Students in English Language Arts and Reading
S 21
Expectations for Students in Science
Memorize Facts/ Definitions/ Formulas
Recite basic science f acts
Recall science terms and def initions
Recall scientif ic f ormula
Communicate Understanding of
Science Concepts
Explain concepts
Observe and explain teacher demonstrations
Explain procedures and methods of science
and inquiry
Organize and display data in tables or charts
Conduct Investigations/
Perform Procedures
Make observations
Collect and record data
Use appropriate tools
Make measurements, do computations
Execute procedures
Generate questions, make predictions
Plan and design experiments
Test ef fects of dif f erent variables
Analyze Information
Classif y and compare data
Analyze data, recognize patterns
Inf er f rom data
Draw conclusions
Apply Concepts/Make Connections
Use and integrate science concepts
Apply and adapt science information to
real-w orld situations
Build or revise theory
Apply science ideas outside the context of
science
Re s pons e Code s
Time on Topic
Re s pons e Code s
Expectations for Students
0 = None
(Not Covered)
0 = No e m phas is
(Not a perf ormance goal for this topic)
1 = Slight cove rage
(Less than one class/lesson)
1 = Slight e m phas is
(Less than 25% of time on this topic)
2 = M ode rate cove rage
(One to f ive classes/lessons)
2 = M ode rate e m phas is
(25% to 33% of time on this topic)
3 = Sus taine d cove rage
(More than f ive classes/lessons)
S 22
3 = Sus taine d e m phas is
(More than 33% of time on this topic)
S 23
Stop & Do: Investigate Curriculum
& Instruction
•
•
•
•
•
•
Content Alignment Results
Teacher Characteristics
Target Class Description
Instructional Practice
Professional Development
Teacher Beliefs
S 24
Task 1: Interpret Content Alignment
Results
1. Scan English Language Arts or Math
Content Chart.
2. Note the information that you readily
understand.
3. Highlight the pieces that need further
information to interpret.
4. Compare with the others in your
group - noting similarities and
differences.
S 25
Content Alignment: Online Vs.
S 26
How to Read the Chart
1
1
Specific content
2
Amount of instructional
time = colored of tiles.
7
2
5
3
Content areas –strands
(e.g. measurement).
4
The learning expectation (cognitive demand)
–horizontal axis.
Groups being compared.
3
6
5
4
6
Colored boxes facilitate
Comparison of data
Between two groups.
7
Alignment Re-centered:
0.5041 = the degree of
S 27
Alignment
between sides
Interpreting Coarse Grain Scales
Interval is 2.5
White
<2.5 %
<5.0 %
<7.5 %
>=7.5 %
=
=
=
=
=
Not Covered
Between 4 & 5 class periods
About 9 class periods
13 to 14 class periods
More than 14 class periods
S 28
Creating Comparisons at Topic Level
Click on the
white boxes at
the left of the
topic strand.
S 29
Sub-topic Level – Fine Grain
S 30
Topic Level: Fine Grain Scale
Conversions
Interval is .5
White
<0.5 %
<1.0 %
<1.5 %
>1.5 %
=
=
=
=
=
Not Covered
1 class periods
2 class periods
3 class periods
4 class periods
S 31
Stop & Do: Prepare to Investigate
Curriculum & Instruction
•
•
•
•
•
•
Content Alignment Results
Teacher Characteristics
Target Class Description
Instructional Practice
Professional Development
Teacher Beliefs
S 32
S 33
Floating Bar Charts - Details
• Two columns supports comparing
two cohort groups of result.
• Scale: percentage of class time
• Standard deviation and the mean
(represents roughly 2/3 of the respondents)
• (n): no results if n <3
• No difference between online and
printed survey format for results.
S 34
Task 2: Interpreting Floating Bar
Charts
1. Scan Chart F-1: Instructional
Activities by grade level.
2. Note the information that you
readily understand.
3. Highlight the pieces that need
further information to interpret.
4. Compare with the others in your
group - noting similarities and
differences in highlighted items.
S 35
S 36
3
1
4
S 37
2
How to Read the Chart
3
1 Two columns supports
comparing two cohort groups
of result.
2 Scale: percentage of class time
3 Standard deviation and the
mean (represents roughly 2/3
of the respondents)
4 (n): no results if n <3
5 No difference between online
and printed survey format for
results.
S 38
A teacher’s
own results.
S 39
Current SEC Disaggregations
• Grade Level
• Level of Achievement
• Amount of Professional
Development
• Percentage of minority students
• By class size
• By percentage of LEP students
• By percentage of Female students
S 40
To access the Report Generator
Go to: www.seconline.org
S 41
OHproject
10000
To complete a survey, Login as ‘Not Registered?’.
Click on the arrows by ‘Please Select’ to scroll to
your project name.
Click ‘Register’.
S 42
S 43
Click on the ‘Data Review’ tab.
S 44
Using the Administrator Report
Generator
S 45
Using the Administrator Report
Generator
• Select content area.
• Select the chart you want to use.
• Select groups (what cohorts do you want to
compare?).
• For example:
S 46
Processing
Your results are being processed.
Thank you for your patience
S 47
S 48
Budget Planning to Use the SEC
• Print Version of the Survey =
$20.00 per person.
• Online Version of the Survey =
$10.00 per person
• Membership in the SEC
Collaborative (State, District =
$15,000 per year) Includes use of surveys,
professional development and technical assistance +
opportunity to work with other SEC project leaders.
S 49
To Set-Up Your Group To Use the SEC
 Call Alissa Minor at WCER:
608 265-0447.
To Set-Up Professional Development
and/or Technical Assistance for Your
Group
Call CCSSO, 202 336-7044 or your
collaborator partner organization.
S 50
• You've carefully thought out all the
angles.
• You've done it a thousand times.
• It comes naturally to you.
• You know what you're doing, its what
you've been trained to do your whole
life.
• Nothing could possibly go wrong, right ?
S 51
Think Again!
S 52