Transcript Measuring knowledge for teaching mathematics

Developing Measures of
Mathematical Knowledge for Teaching
MSP Regional Conference
Boston, MA
March 30-31, 2006
Geoffrey Phelps, Heather Hill,
Deborah Loewenberg Ball, Hyman Bass
Learning Mathematics for Teaching
Study of Instructional Improvement
Consortium for Policy Research in Education
University of Michigan
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
1
Overview of today’s session
1. LMT/SII Measures Development
2. Some Sample Results
3. LMT/SII Measures and Dissemination
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
2
What is “Content Knowledge for Teaching”?
An Example From Subtraction
Subtract:
3002
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
783
3
Analyzing Student Errors
3002 - 783 = 4832
3002
-
783
2781
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
4
Analyzing Unusual Student Solutions
299 1 2
3002
-
3002
-
783
783
3-7-8-1
2219
2219
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
5
LMT/SII Measures Development
Why Would We Want to “Measure”
Teachers’ Content Knowledge for Teaching?
• To understand “what” constitutes mathematical
knowledge for teaching
• To understand the role of teachers’ content
knowledge in students’ performance
• To study and compare outcomes of professional
development and teacher education
• To inform design of teachers’ opportunities to
learn content knowledge
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
7
Measuring Teachers’ Mathematics
Knowledge: Background and History
• Research on teacher behavior
• Early research on student achievement
– Proxy measures for teacher knowledge
– Tests of basic skills
• 1985 on: “the missing paradigm” pedagogical
content knowledge or PCK
• 1990s: interview studies of teachers’
mathematics knowledge (MSU -- NCRTE)
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
8
Study of Instructional Improvement
• Study of three Comprehensive School Reforms;
teacher knowledge a key variable
• Instrument development goals:
– Develop measures of content knowledge teachers
use in teaching
• K-6 content for elementary school teachers
• Not just what they teach - specialized knowledge
– Develop measures that discriminate among teachers
(not criterion referenced)
– Non-ideological
• But we faced significant problems….
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
9
Problems As We Began This Work
• No way to measure teachers’ content
knowledge for teaching on a large scale
– Small number of items, many written by Ball,
Post, others appeared on every instrument
– Nothing known about the statistical qualities of
those items (difficulty, reliability)
– Studies relied on single items, yet single items
unlikely valid or reliable measures of teacher
knowledge
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
10
Early Decisions and Activity
• Survey-based measure of CKT-M
– 3000 teachers participating in SII
– Multiple choice
• Specified domain map
• 5 people + 5 lbs cheese + 5 weeks = 150
items (May 2001)
• Large-scale piloting, summer 2001
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
11
Early Decisions and Activity
Types of knowledge
Mathematical content
Content knowledge
Knowledge of
content
and students
Number
Operations
Patterns, functions,
and algebra
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
12
Early Analyses and Validity Checks
• Results from piloting
– We can measure teachers’ CKT-M
– Reliabilities of .70-.90
– Factor analysis shows distinct types of
knowledge
• Knowledge of content and students (KCS)
separate from CK
• Specialized content knowledge (SCK) vs.
common content knowledge (CCK)
Hill, H.C., Schilling, S.G., & Ball, D.L. (2004) Developing measures of teachers’
mathematics knowledge for teaching. Elementary School Journal 105, 11-30.
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
13
Reliabilities (1PL-IRT): Elementary
Knowledge of content
Knowledge of content
and students
Number and
operations (K-6)
.72-.81
.58-.67
Patterns, functions,
and algebra (K-6)
.70-.85
Geometry (3-8)
.85-.86
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
14
Reliabilities (1PL-IRT):
Middle School
Knowledge of content
Number and
operations (5-9)
.74-.75
Patterns, functions,
and algebra (5-9)
.86-.89
Geometry (3-8)
.84-.86
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
Knowledge of content
and students
15
Content Knowledge :
Number and Operations
• Common knowledge
– Number halfway between 1.1 and 1.11
• Specialized knowledge
– Representing mathematical ideas and operations
– Providing explanations for mathematical ideas and
procedures
– Appraising unusual student methods, claims, or
solutions
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
16
Representing Number Concepts
Mrs. Johnson thinks it is important to vary the whole when she
teaches fractions. For example, she might use five dollars to be the
whole, or ten students, or a single rectangle. On one particular day,
she uses as the whole a picture of two pizzas. What fraction of the
two pizzas is she illustrating below? (Mark ONE answer.)
a) 5/4
b) 5/3
c) 5/8
d) 1/4
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
17
Providing Mathematical Explanations:
Divisibility Rules
Ms. Harris was working with her class on divisibility rules. She told her
class that a number is divisible by 4 if and only if the last two digits of the
number are divisible by 4. One of her students asked her why the rule for
4 worked. She asked the other students if they could come up with a
reason, and several possible reasons were proposed. Which of the
following statements comes closest to explaining the reason for the
divisibility rule for 4? (Mark ONE answer.)
a) Four is an even number, and odd numbers are not divisible by even
numbers.
b) The number 100 is divisible by 4 (and also 1000, 10,000, etc.).
c) Every other even number is divisible by 4, for example, 24 and 28 but
not 26.
d) It only works when the sum of the last two digits is an even number.
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
18
Appraising Unusual Student Solutions
Which of these students is using a method that
could be used to multiply any two whole numbers?
Student A
35
x 25
1 25
+75
8 75
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
Student B
35
x2 5
1 75
+ 70 0
875
Student C
35
x 25
25
1 50
1 00
+ 6 00
875
19
Common vs. Specialized CK
• Appears in exploratory factor analyses on
2/7 forms; confirmatory on 3/7
• Individuals can be strong in common but
not specialized; vice versa
• Support from cognitive interviews of
mathematicians
• Suggests there is professional knowledge
for teaching
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
20
Ongoing Work
• Item and measures development
– Middle school national probability study
– Develop new measurement modules for data
analysis and for probability
• Validation efforts
– “Videotape” study
– Cognitive tracing studies
– Content validity checks
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
21
Some Sample Results
An Example:
Establishing a Relationship to Student
Growth
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
23
Links to Study of Instructional Improvement
Student Achievement Analysis
• SII CKT-M measure – 38 items
– SII: .89 IRT reliability
• Model: Student Terra Nova gains predicted by:
– Student descriptors (family SES, absence rate)
– Teacher characteristics (math methods/content,
content knowledge)
• Teacher content knowledge significant
– Small effect (LT 1/10 standard deviation)
– But student SES is also on same order of
magnitude
Hill, H.C., Rowan, B., & Ball, D.L. (2005) Effects of teachers'
mathematical knowledge for teaching on student achievement. American
Educational Research Journal 42, 371-406.
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
24
A Second Example:
Evaluating Teacher Professional
Development
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
25
Tracking Teacher Growth
• Items piloted in California’s Mathematics
Professional Development Institutes
(MPDI)
– Instructors: Mathematicians and mathematics
educators
– 40-120 hours of professional development
– Focus is squarely on mathematics content
– Summer 2001
– Pre/post assessment format (parallel forms)
Hill, H. C. & Ball, D. L. (2004) Learning mathematics for teaching: Results
from California’s Mathematics Professional Development Institutes. Journal
of Research in Mathematics Education 35, 330-351.
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
26
MPDI Teacher Growth (Year 1)
• For all institutes for
which we have data,
teachers gained .48
logits, or roughly ½
standard deviation
• Translates to 2-3 item
increase on
assessment
• Considered
substantial gain
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
1.2
1
0.8
Pre-test
Post-test
0.6
0.4
0.2
0
All institutes
27
Results from Sample Institutes
1.6
1.4
1.2
1
Pre
Post
0.8
0.6
0.4
0.2
0
MPDI I
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
MPDI II
MPDI III
MPDI IV
MPDI V
28
MPDI Evaluation: Other Findings
• Length of institute predicts teacher gains
– 120-hour institutes most effective, on average
– But some 40-hour institutes very effective
• Focus on mathematical analysis, proof, and
communication leads to higher gains
• Many questions remain
– Effects of content (e.g., mathematics vs. student
thinking)
– Treatment of content: common vs. specialized
– Effects of teacher motivation
– Long term learning from colleagues, curriculum,
practice
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
29
LMT/SII Measures and Dissemination
Current Item Pool
• Equated forms for elementary school:
– Number & operations / Content knowledge
(K-6)
– Number & operations/ Knowledge of content
and students (K-6)
– Patterns Functions & Algebra/ Content
knowledge (K-6)
– Geometry (3-8)
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
31
Current Item Pool
• Equated forms for middle school:
– Number & operations / Content knowledge
(5-9)
– Patterns Functions & Algebra/ Content
Knowledge (5-9)
– Geometry (3-8)
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
32
Item Workshops and
Dissemination
• Interested users attend a one-day workshop in
Ann Arbor
• We cover
– History of item development
– Analytic methods and validation studies
– How to use technical materials
• Users get
– Access to measures
– Support materials
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
33
Dates and Contact Information
• Learning Mathematics for Teaching
– http://sitemaker.umich.edu/lmt
• Dates for LMT Workshops
– May 19, 2006
– August 10, 2006
– Brenda Ely ([email protected])
• Geoffrey Phelps
– [email protected]
– 734-615-6076
http://www.sii.soe.umich.edu/
http://sitemaker.umich.edu/lm
t
34