Transcript Modeling the Determinants of Success in Managerial Accounting

The UIUC SCALE Experiments
By Dan Stone (with
help from Lanny
Arvan & Burks Oakley)
Univ. of Illinois at
Urbana-Champaign,
1988 – 2000
Univ. of Kentucky,
beginning 8/2000 
Presentation Outline
Thanks to …!
Motivation
Introduction to SCALE & ALN
SCALE Research Design
/ Evaluation
SCALE Results
The Bottom Line: What
creates successful ALN?
Thanks to:
Bob Jensen, Lanny Arvan (UIUC), Burks
Oakley (UIUC) for their enthusiasm,
risk-taking, leadership & innovation
More thanks & copyright disclosure:
 portions of this presentation
are taken from presentations by
Lanny Arvan & Burks Oakley
 portions of this presentation
are taken from the SCALE & CET
websites
 See citations at end of presentation
Presentation Outline
Thanks to …!
Motivation
Introduction to SCALE & ALN
SCALE Research Design
/ Evaluation
SCALE Results
The Bottom Line: What
creates successful ALN?
Motivation
Huge investments in asynchronous
learning network (ALN) programs (e.g.,
Penn State World Campus,
Stanford-Online,
Duke Global Exec
MBA)
SCALE project:

over 80 on-line courses
Motivation
Do the SCALE results suggest that
ALNs can improve:
educational effectiveness?
 efficiency?
 access to higher education?

What makes for successful
on-line courses & programs?
Motivation
My interest in SCALE & ALN:
Teach accounting systems &
technology classes

Use pedagogy (technology) to
teach some course content (technology)
Adopted computer aided instruction module in
1991, adopted Powerpoint in 1992
In the first group of adopters of First-Class at UIUC
(1994)
On-line grading & email software for 6 years
On-line, web-based course materials for 3 years
Motivation
My interest in SCALE & ALN:
Lingering doubts … if I weren’t teaching
“technology as content” would “technology as
pedagogy” be worth the trouble?



“Recent surveys of the instructional use of
information technology in higher education clearly
indicate that there have been no significant gains
in either productivity improvement or
pedagogical enhancement” (Kenneth Green,
1997, “The Campus Computing Project”).
Clark 1991, “Media do not influence learning.”
“ALN teaching is arduous.”
Presentation Outline
Thanks to …!
Motivation
Introduction to SCALE & ALN
SCALE Research Design
/ Evaluation
SCALE Results
The Bottom Line: What
creates successful ALN?
Introduction to SCALE & ALN
Sloan Center for Asynchronous Learning
Environments (SCALE) (1995-1999)




Origin: $2.1 million grant from Alfred P. Sloan
Foundation + UI matching
Goal: Restructure undergraduate courses to
integrate asynchronous learning network (ALN)
methods
20 major projects in 1995, now > 80
projects
9 “efficiency” projects – designed to
lower instructional costs per student
Introduction to SCALE & ALN
SCALE replaced by UIUC funded:
Center for Educational Technologies
(CET)

Lanny Arvan, Director
UI-Online

Led by Sylvia Manning
(Chancellor)
Introduction to SCALE & ALN
What is ALN?
In-class learning is “synchronous.”
 Contrast synchronous in-class learning
with “Site independent
learning” (distance education)
–courses delivered largely or
entirely online

Adapted from Lanny Arvan
presentation (the Scale Evaluation)
ALN Goals & Technologies
Enhanced learning opportunities



Interactive course materials,
e.g., PCs & WWW
Simulations, multimedia,
visualization, e.g., streaming video,
Java, Virtual reality (e.g., digital
video cases – “WWHP A CD Case
study”
(http://www.cba.uiuc.edu/lab/BrownBags/WWHP.pps)
Real-time homework, quizzes & grade information, e.g.,
on-line, interactive testing, on-line grades Adapted from Burks Oakley
presentation (“E-Learning”)
ALN Goals & Technologies
Improve access to people & information
Subject matter experts (faculty, TA’s)
 Other students (peer-peer & group
interactions)
 E.g., Conferencing, email
 E.g., Course management
software (i.e., CourseInfo,
WebCT, Mallard).

Adapted from Burks Oakley
presentation (“E-Learning”)
ALN Example Applications
Econ 300 (Intermediate
Microecon) – Lanny Arvan



SCALE efficiency project
A required (disliked) course for all
UIUC business majors
ALN goals:
More feedback & learning opportunities (Mallard on-line
testing)
 Increase interactions (on-line) with faculty & peers (First
Class)
 Disincentives for cram-for-test-&-nothing-else mentality
(homework & quizzes count)
Adapted from Burks Oakley

presentation (“E-Learning”)
ALN Example –
Econ 300 First Class Menu
Presentation Outline
Thanks to …!
Motivation
Introduction to SCALE & ALN
SCALE Research Design
/ Evaluation
SCALE Results
The Bottom Line: What
creates successful ALN?
SCALE Research Design
/ Evaluation
Separate University-sponsored evaluation
team & unit
Research design: Case & field studies
Even when available, ALN vs. nonALN
control groups are problematic due to:



Student self-selection into courses & programs
Instructor self-selection into using technology
Differing evaluative methods & processes (e.g.,
differing rules for on-line vs. traditional exams,
quizzes) (I.e., mixing “media” & “method” – Clark)
SCALE Research Design
/ Evaluation
Most extensive evaluation is of 1st year
classes (19 courses)
An impression  evaluation team
sought evidence supporting increased
efficiency & effectiveness (after
spending $3M – what else
could UIUC do?)
SCALE Research Design
/ Evaluation
Data Source
Sample Size
Pre-course Student Surveys
1,438 Students in 16 Courses
Student Group Interviews
10 Groups
Teaching Assistant Group Interviews
2 Groups
End-of-Semester Student Surveys
1,118 Students in 17 Courses
Pre-Course Faculty Interviews
26
End-of-Semester Faculty Surveys
23
End-of-Semester Faculty Interviews
6
Computer Conference Monitoring
5 Courses
Adapted f from SCALE website
Presentation Outline
Thanks to …!
Motivation
Introduction to SCALE & ALN
SCALE Research Design
/ Evaluation
SCALE Results
The Bottom Line: What
creates successful ALN?
SCALE Evaluation Results
ALN...
Increased communications


Surveys: 51% of students reported an
increase in communication with instructor &
43% with other students.
40% of students reported an increase in the quality of
their interaction with the instructor.
Improved access to information –



"Information when you want/need it."
"Personal control of information,"
"quick-response from peers & students."
Adapted f from SCALE website
SCALE Evaluation Results
ALN...
 Enhanced learning environment





“A new and exciting way to learn," “More prepared for class,"
“Time to learn out of class," "work at own pace."
"On-line homework was a great experience," “On-line
quizzes were "a good way to study for exams."
70% would take another course using computer
conferencing.
75% rated overall experience with computer conferencing
good, very good, or excellent.
60% reported increase in learning due to computer
Adapted f from SCALE website
conferencing.
SCALE Evaluation
Results
ALN...
Improved computing & technology knowledge


"Increased my knowledge of and confidence with
computers." “I’m less
apprehensive about
computers"
70% indicated increased familiarity with
computers.
Adapted from SCALE website
SCALE Evaluation Results
Success of ALN depended on...
Student training


75% found conferencing software
easy or somewhat easy to use.
Interviews: Many students wanted better training.
E.g., "Require a mandatory class on system use."
Student effort & commitment



60% used computer conferencing  1 time week.
Females used conferencing slightly more than did
males (not stat significant)
“Class member comments helped me learn."
Adapted from SCALE website
SCALE Evaluation Results
Success of ALN depended on...
Incentives, I.e., what’s in it for the student?
 Students: "Must give incentives to use the software."
 “Don’t just tack ALN on to an existing course."
Accessible computers
 Lack of computer access = most frequent complaint
 "It was hard to find an available computer."
 "The [modem] lines were always busy."
 "Get more computers or get rid of
[ALN component]."
Adapted from SCALE website
SCALE Evaluation Results
Success of ALN depended on...
System availability
& support
"I couldn't login,
so I stopped
using it."
 "Too many
system crashes"

Adapted from SCALE website
SCALE Efficiency Project Results
Did ALN improve course efficiency?
From whose perspective? (instructor vs.
university)

Instructor time omitted as a cost … despite
acknowledgement that “ALN teaching is arduous.”
(instructor interview). Increased instructor-student
contact time in ALN classes.
Never for small or writing-intensive classes
For each of the 3 examples that I use:

No difference in exam performance for ALN vs.
traditional sections
Adapted from SCALE website
SCALE Efficiency Project Results
Did ALN improve course efficiency?
Introductory Statistics (100)
Development cost funded by grant
 On-going cost of ALN initiatives = $1,000
peer tutor per semester
 ALN class sizes 50 to 100% larger
than traditional sections
(e.g., 154 vs. 101 students)

Adapted from SCALE website
SCALE Efficiency Project Results
Did ALN improve course efficiency?
Econ 300
Section sizes increased from 60 to 180
 Development cost = ~ $56,224

Introductory Spanish (210)
1 time development cost =
~ $15,000
 Class section sizes doubled
with ALN (19 to 38)

Adapted from SCALE website
An ALN success story (LEEP)
UIUC School of Library & Info. Science
Online version of on-campus
MS degree (not a SCALE project) –
 Library Education Experimental
Program
(LEEP) - accredited by the ALA
Began with 31 students in Fall 1996
(including students in Alaska & Vermont)
Currently 125 students enrolled (~ 1/2 in-state)
Adapted from Burks Oakley
presentation (“E-Learning”)
An ALN success story (LEEP)
As of 1999, 96 students completed MS degree
(from 37 states, rural Illinois, Virgin Islands,
France, Colombia, Japan, & Thailand)
All faculty teach  1 online course each
academic year
To serve demand for LEEP, GSLS College:
 added 4.5 FTE faculty
 budget has grown > $750,000
 New building in progress!
Adapted from Burks Oakley
presentation (“E-Learning”)
SCALE Efficiency Project Results
Do the SCALE results suggest that
ALNs can improve access to higher
education?

Yes!!!! LEEP program & SCALE efficiency
projects
Presentation Outline
Thanks to …!
Motivation
Introduction to SCALE & ALN
SCALE Research Design
/ Evaluation
Results
The Bottom Line: What
creates successful ALN?
The Bottom Line: What makes
for a successful ALN?
Support from the top for technological
innovation (e.g., LEEP)
 Incentives for participation among
faculty & departments
Pedagogically sound content
 Commitment to quality pedagogy
& content
Enthusiastic & committed faculty & support staff
 Are faculty early (15%), middle (70%), or late (15%)
adopters? - Incentives differ by group
 Helpful: wired, computer literate students & faculty
Adapted from Burks Oakley
presentation (“E-Learning”)
The Bottom Line: What makes
for a successful ALN?
Thoughtful consideration of copyright and
ownership issues (faculty vs. University,
publishers vs. University, faculty vs. students)

Who owns what? Accessibility vs. ownership (see
recent Chronicle of Education articles)
Support & maintenance



Development / programming
Access to student computing is
essential (& often problematic)
Supported, mature products
Adapted from Burks Oakley
presentation (“E-Learning”)
The Bottom Line: What makes
for a successful ALN?
Marketing to current &
potential students, current &
potential faculty, University administrators

What’s in it for the students?
faculty? administrators?
What’s the goal?

Improved class availability? Technology as
content? Richer content?
Adapted from Burks Oakley
presentation (“E-Learning”)
References & resources
Clark, Richard E., 1991, “When researchers swim upstream:
reflections on an unpopular argument about learning from media,”
Educational Technology (February), 34-40.
Scale website: http://w3.scale.uiuc.edu/scale/index.html
UI on-line: http://www.online.uillinois.edu/
Web-board tutorial:
http://w3.scale.uiuc.edu/scale/docs/using_webboard/index.html
LEEP program: http://alexia.lis.uiuc.edu/gslis/degrees/leep.html
UIUC Center for Educational Technologies (CET):
http://www.cet.uiuc.edu/
Webboard: http://webboard.oreilly.com/
WebCT: http://www.cet.uiuc.edu/webct/index.asp
Dan Stone
Email:
[email protected]
WWW:
http://www.cba.uiuc.ed
u/d-stone1/dstone1.html
Phone: 257-3788