Transcript Document

Technology Promoting Student
Excellence:
A preliminary analysis of the first year
of New Hampshire’s 1:1 laptop program
Damian Bebell
Lynch School of Education
Boston College
[email protected]
Full evaluation report and PowerPoint
slides will be
available for download at :
www.intasc.org
History of Educational Technology
Purported Benefits
• Motivate and Engage students
• Increase resources and information
• Exposure to technology (21st century skills)
• Improve teaching (lesson plans, communication)
• Movement towards student centered classrooms
• Streamline record keeping
• Special Needs/Accommodations
• Differentiate Instruction/learning
Trends in Educational Technology
• In the late 80’s a trend emerged that
technology was good for students – so that
they will be prepared for the technology
jobs/skills of the future
• Current thinking is that technology is a
vehicle for improved student learning of
traditional curriculum
Student : Computer Ratios
• Standard Metric of technology access
– National Student:Computer Ratios
•
•
•
•
125:1 = 1983
9:1 = 1995
6:1 = 1998
4:1 = 2003
Technology is shared
Source: Market Data Retrieval, 1999; Education Week, 2004
Use is sporadic
(lots of research)
Movement towards 1:1 technology
• Each teacher and student has full access to a
computer (usually laptop)
• Previous research suggests:
– Equity issues disappear
– Technology becomes the relied upon tool students use
for research, writing, and presentation
– Classroom management is simplified
– Students are more engaged enthusiastic
Current 1:1 laptop programs
• Maine Learning Technology Initiative (2002)
– All 7th and 8th grade students and teachers in 239
middle schools
– Apple iBooks, wireless classrooms,teacher training,
support, and professional development
– 1,000 flowers blossoming philosophy
• Henrico County, VA (2002)
– Apple iBooks in Grades 6th-12th
– 20,000+ students
– No systemic research or evaluation
• Massachusetts (3 Berkshire middle schools)
• Andover, MA (Toshiba laptops--parent purchase)
Current 1:1 laptop programs (2)
•Michigan
–Cross platforms
–Looking into less expensive technology (i.e. Palms)
•Sedgwick, KS (2002)
–Apple iBooks in middle school
•Texas
•Florida
•Georgia
•Vermont
New Hampshire Question:
• Would the initial positive findings from
Maine’s 1:1 laptop program generalize to six
New Hampshire middle schools?
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Photo taken from: http://www.state.nh.us/governor/ (1/28/04)
History of TPSE
Fastest roll out of a 1:1 technology program
ever!!!
History of TPSE (2)
Benson’s Philosophy:
"Technology must be heavily leveraged in a way to
enhance the classroom experience and excite the
student's passion for learning,”
In a perfect world, this would mean each of the
participating student would be excited each
morning in anticipation of coming to school and no
student would ever want to leave after school
ends.”
Source:: http://www.state.nh.us/governor/
History of TPSE (3)
1:1 laptops would…
• Improve educational teaching,
learning and achievement
• Use an interactive instructional
practice
• Bridge the significant digital divide
• Create a highly educated, technologysavvy workforce
Source: Benson PowerPoint 9/2/03
TPSE Timeline (3)
9/2/03:
Program announced
10/15/03: Competitive material submitted
10/03:
24 private organizations donate over
$1.2 million to fund the program (no
public funds)
11/3/03: Participating schools announced
12/03:
Installation of technology
» Teachers receive laptops
» Teacher training
January 5-6, 2004:
iBooks distributed to 7th grade students
Participants
6 New Hampshire middle schools selected
from approximately 20 applications
•Armand R. Dupont School (Allenstown, NH)
•Indian River Middle School (Canaan, NH)
•Haverhill Cooperative Middle School (N. Haverhill, NH)
•Paul School (Sanbornville, NH)
•Thornton Central School (Thornton, NH)
•Winnisquam Regional Middle School (Tilton, NH)
400+ students
40 teachers
Program Characteristics
•
•
•
•
•
•
•
Apple iBook laptops for teachers and students
Software
Digital cameras and digital video cameras
Wireless school wide networks
Printers
Teacher training
Technology support
Study Design/Methodology
Try and capture the initial impact/effect of the the laptop
program (Jan. 04)
Convince stakeholders that research and evaluation was an
important component of the initiative
PRE/POST Design
Pre
Survey
Post
Survey
Follow Up
Web Survey
Teacher
Survey
Jan. 04
May/June
04
Oct. 05
Student
Survey
Jan. 04
May/June
04
---
Study Design/Methodology
Spring 2004 Student Surveys (n=862 both pre and post)
1. Measures of technology use in school
2. Technology use at home
3. Across subject area
4. Personal comfort level with technology
Spring 2004 Teacher Surveys (n=47 both pre and post)
1. Measures of technology use in school
2. Measures of technology use beyond the classroom
3. Across subject area
4. Personal comfort level with technology
5. Demographic Information
6. Attitude toward technology
Study Design/Methodology
Fall 2004 Teacher Survey Follow Up (n=32)
Capture teachers perceptions about the specific impacts of
the 1:1 program on students, teaching and learning
•Collected Online
•http://corvus.bc.edu/nhteacherfollowup/NH_teacher_followup.cfm
•Survey items adapted from teacher survey measures created
for use in Maine
•Specific focus on how technology has impacted different
groups of students
•Traditional students
•High Achieving Students
•At-Risk or Low Achieving Students
Study Design/Methodology
No systematic examination of achievement…yet
–
–
–
–
–
–
First need to measure impact and level of us
No shared assessment in NH at the 7th grade
Need previous measures of student achievement
Individual student data
Difficulty in getting comparison groups to participate
Teachers perception of technology impact on
achievement
Today’s presentation focuses on
the immediate (1st six months)
impact of the laptop program
Student Level Results/Findings
Student survey response rate
Estimated #
of 7th grade
Students
Pre (1/04) Post (5/04)
Responses Responses
Dupont
63
61
58
Haverhill
71
73
47
Indian River
118
100
96
Paul
80
64
61
Thornton
28
27
28
Winnisquam
175
114
134
Students used technology in school substantially
more across all six 1:1 settings
How often do you use technology in your classroom?
(Q1A)
Dupont
Haverhill
Indian River
Pre
Post
Paul
Thornton
Wiinsquam
Total
1
Never
Total
15 2min
or less
Wiinsquam Thornton
3 min
15 -60
Paul
Indian
River
1-24 hrs
2+5 hrs
Haverhill
Dupont
Pre
2.28
1.67
3.93
2.17
1.93
2.4
3.22
Post
4.68
4.71
4.52
4.85
4.38
4.85
4.9
Students used technology in school substantially
more in English/Language Arts
How of ten do you use a computer in Reading/ELA? (Q3A)
Dupont
Ha v erhill
Indian Riv e r
Pre
Post
Paul
Thornton
Wiinsqua m
Tota l
1
Never
Total
Pre
Post
2.32
4.39
2
3
15 min
Wiinsquam
Paul
min
or less Thornton 15 -60
1.44
4.29
4.04
4.15
2.54
4.75
Indian
River
2.11
4.65
4
5
Haverhill
1-2
hrs
2.92
3.88
Dupont 2+ hrs
2.62
4.36
Students used technology in school substantially
more in Social Studies
How of ten do you use a computer in Social Studies?
(Q3B)
Dupont
Ha v erhill
Indian Riv e r
Pre
Post
Paul
Thornton
Wiinsqua m
Tota l
1
Never
Pre
Post
Total
1.74
4.12
2
Wiinsquam
15 min Thornton
or less
1.32
4.15
1.26
4.08
3
Paul
15 -60 min
1.19
4.68
Indian
River
1.97
3.36
4
Haverhill
Dupont
1.77
4.16
2.62
4.36
1-2 hrs
5
2+ hrs
Students used technology in school substantially
more in Math
How of ten do you use a computer in Math? (Q3C)
Dupont
Ha v erhill
Indian Riv e r
Pre
Post
Paul
Thornton
Wiinsqua m
Tota l
1
2
Never Total
Pre
Post
1.24
3.3
3
Wiinsquam
15 min Thornton
or less
1.07
3.02
1.89
3.19
Paul
Indian
River
1.08
3.54
1.27
4.38
15 -60 min
4
5
Haverhill
1-2 hrs
1.26
4.59
Dupont 2+ hrs
1.38
4.09
Students used technology in school substantially
more in Science
How of ten do you use a computer in Science? (Q3D)
Dupont
Ha v erhill
Indian Riv e r
Pre
Post
Paul
Thornton
Wiinsqua m
Tota l
1
2
Never Total
Pre
Post
1.68
4.11
3
Wiinsquam
15 min Thornton
or less
1.27
4.05
1.96
3.19
Paul
Indian
River
1.27
3.34
1.34
4.6
15 -60 min
4
5
Haverhill
1-2 hrs
2.31
4.24
Dupont 2+ hrs
2.6
4.16
Students used technology for wide variety of tasks
How often do you a computer in school to:
Sending and receiving
email
Pre Mean
take a test or quiz
Post Mean
work with
spreadhseets/databases
analyze data
create graphs or tables
solve problems
create Hyperstudio or
Powerpoint presentation
play computer games
write first drafts
edit papers using a
computer
find information on the
internet
1
Never
15 2min
or less
3 min
15 -60
1-24hrs
5 hrs
2+
Teachers use of technology as reported by the students
How often does your teacher use a computer for each
subject? (Q7)
Science
Math
Mean Pre
Mean Post
Social Studies
Reading/ELA
0
R
1
Never
15 min
2
or less
15 -603 min
1-24hrs
2+ 5hrs
Students use of technology at home
How of ten do you use your home computer to: (Q12)
create/maintain web sites
create/edit digital photos or movies
computer program
download/play mp3/music
write papers for school
Total Pre
Total Post
search the 'net for fun
email
chat/instant message
search the 'net for school
play games
Never
1
15 min
2
or less
15 -60
3 min
1-24hrs
2+5hrs
Students reported technology skills
P re
P os t
How well are you able to:
communicat e using email
record and analyze dat a
using a spreadhseet
creat e mult imedia
present at ions
write papers using a
comput er
f ind inf ormat ion on the
Int ernet
1
Not Well
2
3
4
Very Well
Pre/Post SY (03/04) Teacher Results
Teacher
survey
response rate
less sure than
student
survey
Typically Math, Science,
English/language Arts, or Social
Studies teachers completed the
survey, however around 10% of
respondents were not primary
classroom teachers
Teacher Pre/Post Beliefs and Confidence Measures
How important are compute rs in y our
teaching? (Q3)
Pre
1=Very Important
2=Somewhat Important
3=Not Very Important
Post
1
Series1
2
3
Post
Pre
1.68
2.44
How confident are you when using
computers? (Q4)
P re
1= Very Confident
2=Somewhat Confident
3=Not Very Confident
P os t
1
S eries 1
2
3
P os t
P re
1 .6 8
2 .0 4
Teachers Use of Technology
prepare or m ai nta in IE PS
create /m aintai n web sit es
use a com put er to m odel
re lat ionshi ps/ functi ons
acc ess studnets vi a c om pute r
creat W e bQuest or buil d t he web
into a l esson
use a com put er to hel p student be tte r
undert st and a concept
Pre
Pos t
use a com put er to prese nt info t o
cl ass
del ive r instructi on wit h computer
ada pt an acti vit y to st udent speci al
nee ds
com m uni cate wit h t eac hers, parents
and adm i nistrators via em ai l
perform resea rch and le sson
pl anni ng usi ng t he web
create t est , quiz or assi gnm ent
m ake handouts using computer
1
2
make
handouts
create tes t, perform
communic
quiz or
res earch
ate with
as signmen and les son teachers,
t
planning
parents
deliver
ins truction
with
computer
us e a
us e a
3.86
4.36
4.14
4.09
Pre
3.84
3.68
Post
4.5
4.5
4.5
4
4
adapt an
to
Once or activity
student
special
twice a2.68year 2.08
3.08
us ing
Never
computer
R
3
computertimes
computer
Several
to present
to help
info to
student
per
1.84
1.76year 1.71
creat
access
WebQuest studnets
or build
via
the web
computer
1.6
3.36
5
us e a
computer
Several times
to model
relationshi
per1.6month
1.52
3.23
3.36
Several times
per
1.28 week
create/mai prepare or
ntain web maintain
sites
IEPS
1.4
1.86
1.57
How often do teachers assign students to create the following products
using technology?
videos or movies
web pages, etc.
multimedia projects
Pre
stories or books
Post
graphs or charts
pictures/art work
reports and term
papers
1
2
Never
Pre
reports and term
papers
1.88
Post
3.45
3
4
Once orgraphs or charts Several
timesmultimedia projects
Several
times
pictures/art work
stories or books
web pages, etc.
twice
a year 1.56
per1.48
year
per month
1.88
1.46
1.16
2.68
2.82
1.95
3.09
1.57
5
Several times
videos or movies
1.16 per week
2.5
Also observed changes in teachers beliefs towards learning and
technology?
Overwhelming agreement among participating teachers that :
Students work harder when using computers
Technology allows students to create better looking finished
products
Students are more willing to write second drafts when using
a computer
Students develop a deeper understanding of the
material when using technology
Increase in the belief that students
interact with each other more
while working with computers
Teacher Survey Follow Ups (Oct. 04)
32 teachers across the six schools responded to an
online follow up survey (n=32 in all cases)
My goals for students have changed
4 0 .0
3 7 .5
3 5 .0
3 0 .0
2 5 .0
2 5 .0
2 0 .0
1 8 .8
1 5 .6
1 5 .0
1 0 .0
5 .0
3 .1
0 .0
St rongly A gree
A gree
Neutral
Disagree
St rongly
Disagree
Teacher Survey Follow Ups (Oct. 04)
My role in the classroom has changed
7 0 .0
6 2 .5
6 0 .0
5 0 .0
4 0 .0
3 0 .0
2 0 .0
1 2 .5
9 .4
1 0 .0
1 2 .5
3 .1
0 .0
St rongly A gree
A gree
Neutral
Disagree
St rongly
Disagree
The school climate has changed
7 0 .0
6 5 .6
6 0 .0
5 0 .0
4 0 .0
3 0 .0
2 0 .0
1 8 .8
1 5 .6
1 0 .0
0 .0
St rongly A gree
A gree
Neutral
0 .0
0 .0
Disagree
St rongly
Disagree
Teacher Survey Follow Ups (Oct. 04)
Student acheivement has improved
6 0 .0
5 0 .0
5 0 .0
4 0 .0
3 1 .3
3 0 .0
2 0 .0
1 5 .6
1 0 .0
3 .1
0 .0
0 .0
St rongly A gree
A gree
Neutral
Disagree
St rongly
Disagree
Teacher Survey Follow Ups (Oct. 04)
My beliefs about teaching and learning have
changed
6 0 .0
5 6 .3
5 0 .0
4 0 .0
3 0 .0
1 8 .8
2 0 .0
1 0 .0
1 5 .6
6 .3
3 .1
0 .0
St rongly A gree
A gree
Neutral
Disagree
St rongly
Disagree
My understanding of how people learn has
changed
4 5 .0
4 0 .6
4 0 .0
3 7 .5
3 5 .0
3 0 .0
2 5 .0
2 0 .0
1 5 .0
1 0 .0
1 2 .5
6 .3
5 .0
3 .1
0 .0
St rongly A gree
A gree
Neutral
Disagree
St rongly
Disagree
Teacher Survey Follow Ups (Oct. 04)
My computer skills have improved
6 0 .0
5 6 .3
5 0 .0
4 3 .7
4 0 .0
3 0 .0
2 0 .0
1 0 .0
0 .0
St rongly A gree
A gree
0 .0
0 .0
0 .0
Neutral
Disagree
St rongly
Disagree
Teacher Survey Follow Ups (Oct. 04)
I have had adequate professional development
opportunities
6 0 .0
5 0 .0
5 0 .0
4 0 .0
3 0 .0
2 5 .0
2 0 .0
1 0 .0
1 5 .6
9 .4
0 .0
0 .0
St rongly A gree
A gree
Neutral
Disagree
St rongly
Disagree
The delivery of curriculum in my classes has
changed
7 0 .0
6 5 .6
6 0 .0
5 0 .0
4 0 .0
3 0 .0
2 1 .9
2 0 .0
1 2 .5
1 0 .0
0 .0
St rongly A gree
A gree
Neutral
0 .0
0 .0
Disagree
St rongly
Disagree
Teacher Survey Follow Ups (Oct. 04)
The school has developed effective policies and
procedures for the laptop program
6 0 .0
5 0 .0
5 0 .0
4 0 .0
3 4 .4
3 0 .0
2 0 .0
9 .4
1 0 .0
6 .3
0 .0
0 .0
St rongly A gree
A gree
Neutral
Disagree
St rongly
Disagree
6 0 .0
Students have greater freedom of choice with
regard to their individual learning style
5 3 .1
5 0 .0
4 0 .0
3 0 .0
2 1 .9
2 1 .9
2 0 .0
1 0 .0
3 .1
0 .0
St rongly A gree
A gree
Neutral
Disagree
0 .0
St rongly
Disagree
Teacher Survey Follow Ups (Oct. 04)
My ability to assess and evaluate student work
has improved
4 5 .0
4 0 .6
4 0 .0
3 5 .0
3 0 .0
2 5 .0
2 1 .9
1 8 .8
2 0 .0
1 5 .6
1 5 .0
1 0 .0
5 .0
3 .1
0 .0
St rongly A gree
A gree
Neutral
Disagree
St rongly
Disagree
Teacher Survey Follow Ups (Oct. 04)
Participation in class
100
9 3 .8
90
8 4 .4
80
70
6 2 .5
60
50
40
3 4 .4
30
20
10
1 2 .5
6 .3
3 .1
0
0
3 .1
Tradit ional students
A t-risk/low achieving
student s
High achieving
student s
Declined
3.1
0
3.1
No Ef f ect
12.5
6.3
34.4
Improved
84.4
93.8
62.5
Teacher Survey Follow Ups (Oct. 04)
Motivation
100
9 3 .8
90
8 1 .3
8 1 .3
80
70
60
50
40
30
10
1 5 .6
1 5 .6
20
3 .1
3 .1
0
6 .3
3 .1
Tradit ional students
A t-risk/low achieving
student s
High achieving
student s
Declined
3.1
6.3
3.1
No Ef f ect
3.1
15.6
15.6
Improved
93.8
81.3
81.3
Teacher Survey Follow Ups (Oct. 04)
Engagement/interest level
120
100
9 6 .9
9 0 .6
8 1 .3
80
60
40
1 5 .6
20
3 .1
6 .3
0
0
3 .1
3 .1
Tradit ional students
A t-risk/low achieving
student s
High achieving
student s
Declined
3.1
0
3.1
No Ef f ect
6.3
3.1
15.6
Improved
90.6
96.9
81.3
Teacher Survey Follow Ups (Oct. 04)
Ability to work independently
80
7 1 .9
7 1 .9
70
60
5 3 .1
50
4 3 .8
40
30
2 8 .1
25
20
10
3 .1
3 .1
0
0
Tradit ional students
A t-risk/low achieving
student s
High achieving
student s
Declined
3.1
0
3.1
No Ef f ect
25
28.1
43.8
Improved
71.9
71.9
53.1
Teacher Survey Follow Ups (Oct. 04)
Ability to retain content material
70
5 9 .4
60
4 6 .9
50
50
50
4 6 .9
4 0 .6
40
30
20
10
3 .1
3 .1
0
0
Tradit ional students
A t-risk/low achieving
student s
High achieving
student s
Declined
3.1
0
3.1
No Ef f ect
46.9
40.6
50
Improved
50
59.4
46.9
Teacher Survey Follow Ups (Oct. 04)
Quality of work
90
80
7 8 .1
75
70
6 2 .5
60
50
40
3 4 .4
30
2 1 .9
2 1 .9
20
10
3 .1
3 .1
0
0
Tradit ional students
A t-risk/low achieving
student s
High achieving
student s
Declined
3.1
0
3.1
No Ef f ect
21.9
21.9
34.4
Improved
75
78.1
62.5
Teacher Survey Follow Ups (Oct. 04)
Interaction with teacher
100
8 7 .5
90
7 8 .1
80
70
5 9 .4
60
50
40
3 4 .4
30
1 5 .6
20
10
1 2 .5
6 .3
6 .3
0
0
Tradit ional students
A t-risk/low achieving
student s
High achieving
student s
Declined
6.3
0
6.3
No Ef f ect
15.6
12.5
34.4
Improved
78.1
87.5
59.4
Teacher Survey Follow Ups (Oct. 04)
Interaction with other students
100
8 7 .5
90
8 4 .4
80
75
70
60
50
40
30
2 1 .9
1 5 .6
20
10
9 .4
3 .1
3 .1
0
0
Tradit ional students
A t-risk/low achieving
student s
High achieving
student s
Declined
3.1
0
3.1
No Ef f ect
9.4
15.6
21.9
Improved
87.5
84.4
75
Teacher Survey Follow Ups (Oct. 04)
Quality of student writing
70
6 5 .6
6 5 .6
6 2 .5
60
50
40
3 4 .4
3 4 .4
3 1 .3
30
20
10
3 .1
3 .1
0
0
Tradit ional students
A t-risk/low achieving
student s
High achieving
student s
Declined
3.1
0
3.1
No Ef f ect
34.4
34.4
31.3
Improved
62.5
65.6
65.6
The Rise and Fall of 1:1 Computing
•The cost of going 1:1 is great
•$40 million in Maine,
•$1.2+ million in NH
•Over $25 million in VA
•There is a climate of great pressure on
schools to demonstrate that money spent is
directly and positively impacting students
•Current definition of impacting students is
increased performance as measured by a
standardized test (accountability at all levels,
AYP, etc)
The Rise and Fall of 1:1 Computing (2)
It is estimated that the measurable
impact of educational technology
investments can take 4+ years
Stakeholders expect to see measurable
differences in student scores given
their investments (typically not very
patient)
Rise and Fall of 1:1 Computing
Research/Evaluation is costly (time and money)
Methodological Challenges
Good accurate measures of technology “use” (STEP 1)
Valid measurement of student achievement
Issues with paper based tests for high-tech students
(Russell, 1999; 2001; 2002)
Measures of prior achievement
IDEAL QUESTION:
What kind of technology use leads to what kind of
achievement gains (for x kind of student)?
QuickTime™ and a
TIFF (Uncompressed) decompressor
are needed to see this picture.
Laptop flop; Maine results
should give pause to NH
MAINE'S two-year-old experiment with laptop
computers in the classroom has flopped as measured
by student test scores. Now maybe this nonsensical
trend will fizzle out and we can get back to spending
time and money on educating students instead of
buying them expensive tools that don't help them
learn the basics. Maine has laid out $37.2 million on a
four-year experiment that provides laptop computers
to all students in grades 7 and 8…
Manchester Union Leader August 12, 2004 Page A16