Transcript www.epic.columbia.edu

EPIC Online Publishing
Use and Costs
Evaluation Program
Overview
• Overview of Evaluation Program
• Research update
Purpose of EPIC
Evaluation Program
• To investigate how online electronic resources
affect different aspects of the scholarly
communication process.
a) general perspective
b) through lens of EPIC projects
How Does the Shift to Electronic
Resources Affect the Following Groups?
• Publishers
• Administrators/Information Technology
• Librarians
• Faculty
• Students
Research Approach
• Interviews
• Focus Groups
• Surveys
• Weblog analysis
Info gathering
Survey development
Research Projects to Date
• Librarian Focus Group
• Librarian Online Survey
• Faculty Interviews
• Faculty Online Survey
• Student Interviews
• Student Online Survey
• http://epic.columbia.edu/eval/
Librarian Online Survey
Methodology
• a 15 minute online survey of librarians was conducted
in Winter, 2002
• e-mail invitations were sent to librarians at higher
education institutions across the country
• All were librarians at institutions that are either CIAO
or Earthscape subscribers, or have been identified as
being potential subscribers
• 1007 respondents (response rate: 32%)
Objectives
• Learn what librarians perceive as the benefits
and disadvantages of electronic resources.
• Gain insights into how electronic resources
affect the role of the librarian.
• Gain insights into how electronic resources
affect the role of print resources.
Conclusions from Librarian
Survey
Electronic Resources:
• Allow libraries to serve patrons more effectively
• Put a strain on some libraries
• Do not replace print resources
Faculty Survey
Methods
• a 25 minute online survey of faculty and graduate
students was conducted in Winter 2003
• e-mail invitations were sent to scholars at higher
education institutions across the country
• scholars were involved in the disciplines of
International Affairs or Earth Sciences
• 845 respondents (response rate 14%)
Objectives
• Gain insights into how academics are using electronic
resources in their academic work
• Gain insights into the perceived benefits and
disadvantages of electronic resources
• Learn how academics perceive electronic resources as
affecting undergraduate students’ scholarship
• Examine differences based on stage of career, type of
discipline, and type of institution
Respondent Profile
Gender & Age
N=845
72.1%
Male
Gender
Female
27.9%
21-30
13.5%
31-40
Age
28.0%
41-50
25.9%
51-60
21.3%
61+
9.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Position & Type of Institution
N=845
30.5%
Full Professor
21.2%
Associate Professor
18.8%
Assistant Professor
Level of
Appointment
Research position in an academic
institution
3.8%
Adjunct
Professor/Instructor/Lecturer/Other
5.9%
Graduate Student/Post-doctoral
Fellow
19.8%
67.7%
Research University
Type of
Institution
20.4%
Master's College
11.8%
Baccalaureate College
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Academic Responsibilities and Field of Study
N=845
Earth/Environmental Science
63.6%
Field of
Study
International Affairs
36.4%
Solely as a researcher
5.2%
More as a researcher than as
a teacher
Academic
Responsibiliti
19.3%
About equally as a researcher
and as a teacher
40.7%
More as a teacher than as a
researcher
32.5%
Solely as a teacher
2.2%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Electronic Resources and
Academics’ Work Habits
Use of and Proficiency with Electronic Resources
N=845
Every day
Use of electronic
resources
62.2%
A few times a week
29.3%
7.6%
A few times a month
Once a month
0.6%
Less than once a month
0.2%
0.0%
Never
Extremely proficient (5)
Proficiency with
electronic resources
Mean
3.86
20.1%
Very Proficient
48.3%
Somewhat proficient
29.3%
Not very proficient
Not at all proficient (1)
2.2%
0.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Electronic Resources’ Affect on Work Habits
N=845
Somewhat agree
Neither agree nor disagree
Somewhat disagree
Strongly disagree (1)
Does not apply
I am increasingly
dependent on electronic
resources
8.6%
46.0%
I sometimes find that I settle
for materials that are
available online rather than
make a trip to the library to
get what I really would like
34.0%
8.4%
6.6%
1.
1%
I use the physical library
less than I would if
electronic resources were
not available to me
4.27
4.13
31.2%
16.7%
20.9%
20.7%
0.
7%
2.88
9.7%
2.30
3.9%
0%
18.7%
10%
14.9%
20%
30%
27.9%
40%
50%
0.
5%
I have a tendency to go no
further than electronic
resources in search for
information
38.7%
MEAN
3.
9%
47.8%
0.
0%
2.
8%
2.
0%
Strongly agree (5)
34.1%
60%
70%
80%
90%
100%
Work Habits and The Library
N=845
I do most of my work outside the physical library
because I can readily access the information I need
remotely
59.6%
I do most of my library work outside the physical
library because I can adjust my information needs to
suit my preference for working from my home/office
22.8%
I do most of my library work inside the physical
library
17.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90% 100%
Perceived
Benefits and Disadvantages
of Using
Electronic Resources
Benefits of Electronic Resources--Convenience
N=845
Strongly agree (5)
Somewhat agree
Neither agree nor disagree
Somewhat disagree
Strongly disagree (1)
Does not apply
1.
3
0. %
1%
MEAN
4.38
26.3%
3%
4.20
9.7%
1.
4%
51.5%
3.
5.
Possible for me to
work from home or
other location off
campus/work site
9%
7.1% 3.9%
3%
30.4%
3.
Save time
57.2%
0%
Slide 1 of 2
10%
20%
36.9%
30%
40%
50%
60%
9.0%
70%
80%
90%
7.8%
0.
44.7%
1%
4.16
More information with
less effort
100%
Benefits of Electronic Resources—Improved Availability
N=845
Somewhat agree
Neither agree nor disagree
Somewhat disagree
Strongly disagree (1)
Does not apply
33.0%
5%
0.
18.1%
3.97
3.
7%
0.
14.8%
6.0%
4.03
7%
41.5%
10.2%
0%
10%
21.7%
20%
19.9%
30%
40%
31.6%
50%
60%
70%
4.1%
15.1%
80%
90%
4.01
3.
17.8%
5%
Greater access to older
scholarly information
44.3%
1.
29.6%
6%
0.
Items become available
quicker electronically than
in print
Slide 2 of 2
0%
Some materials are more
readily available
electronically than in print
34.0%
4.21
9%
33.6%
6.
6.
Increased access to
pictures/graphics
0.
4.
10.3%
1%
43.1%
2%
41.2%
2.
Greater access to current
scholarly information
MEAN
5%
5%
Strongly agree (5)
100%
2.80
Disadvantages of Electronic Resources
N=845
Strongly agree (5)
Somewhat agree
Neither agree nor disagree
Somewhat disagree
Strongly disagree (1)
Does not apply
I have difficulty judging the
quality of content
11.5%
The amount of information
available can be
overwhelming for me
10.9%
0%
37.4%
20.1%
37.3%
10%
20%
30%
18.5%
17.9%
29.5%
22.0%
40%
50%
13.7%
19.6%
20.9%
60%
70%
4.4%
9.0%
12.8%
16.3%
80%
90%
0.
2%
14.8%
20.7%
3.57
3.31
0.
9%
Too much information is
available
41.8%
3.15
0.
4%
18.7%
0.
7%
MEAN
Increases the need to
separate out the "reliable"
information from the
"unreliable" information.
2.98
100%
Benefits and Disadvantages by Discipline
N=845
4.49
Save time
4.32
3.70
Increased access to pictures/graphics
4.10
International Affairs
Earth/Environmental Science
4.18
Some materials are more readily available
electronically than in print
3.95
3.46
Too much information is available
3.22
3
3.2
3.4
3.6
3.8
All of the differences in means by discipline on this chart are statistically significant.
4
4.2
4.4
4.6
4.8
5
Benefits and Disadvantages by Seniority
N=845
4.29
Save time
4.50
4.03
More information with less effort
4.32
Some materials are more readily available
electronically
Associate/Full
Professor
4.10
3.94
Graduate
Student/Post
Doctoral
Fellow/Assistant
Professor
3.65
Increases the need to separate out the "reliable"
information from the "unreliable" information.
3.43
3.24
I have difficulty judging the quality of content
3.00
2.50
3.00
3.50
All of the differences in means by seniority on this chart are statistically significant.
4.00
4.50
5.00
Teaching with Electronic
Resources
Use of Electronic Resources for Teaching
N=419
I direct students to primary or supplementary materials found online
90.0%
I retrieve data online for classroom use
85.7%
I go online to get current events information to help demonstrate
ideas
79.0%
I go online to get images that help demonstrate ideas
72.1%
I go online to research lectures that I will be covering in class
67.3%
I place course materials online (e.g. reserve readings)
65.2%
I created a course website for one or more of my courses
57.0%
I use online classroom programs (e.g. Blackboard)
35.3%
I use online materials in place of textbooks
27.4%
I teach extension courses online
8.6%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90% 100%
Use of Electronic Resources for Teaching by Discipline
N=419
I direct students to primary
or supplementary
materials found online
96.8%
85.9%
I go online to get current
events information to help
demonstrate ideas
89.2%
72.9%
International Affairs
Earth/Environmental
Science
I go online to get images
that help demonstrate
ideas
54.8%
82.4%
42.7%
I use online classroom
programs
30.9%
0%
10%
20%
30%
40%
50%
60%
All of the differences in means by discipline on this chart are statistically significant.
70%
80%
90%
100%
Electronic Resources & Students—Benefits
N=419
Strongly agree (5)
Somewhat agree
Neither agree nor disagree
Somewhat disagree
Strongly disagree (1)
Does not apply
Students have increased
access to current
information because of
electronic resources
62.3%
4.57
10.3%
0%
10%
39.6%
26.5%
20%
23.4%
30%
40%
50%
9.8%
27.0%
60%
70%
4.5%
9.5%
80%
90%
3.
3%
43.4%
2.
1%
4.24
Students have increased
access to older
information because of
electronic resources
Slide 1 of 2
32.5%
0.
5%
Students are now able to
do projects they wouldn't
have been able to in the
past because some types
of information are more
readily available online
0.
7%
3.
3
1. %
0%
0.
2%
MEAN
100%
3.01
Electronic Resources & Students—Problems
N=419
Somewhat agree
Neither agree nor disagree
Somewhat disagree
Strongly disagree (1)
Does not apply
MEAN
Students have difficulty evaluating the quality of content
of some online information
28.6%
4.8%
0.5%
4.59
Students have a tendency to go no further than electronic
resources in searching for information
33.2%
4.47
51.3%
33.4%
Students' use of electronic resources has resulted in them
often not learning how to use the physical library
50.8%
33.4%
4.38
37.7%
10% 20% 30% 40%
16.9%
4.
5%
3.82
28.2%
0%
Slide 2 of 2
4.32
2.
9%
8.4%
9.
8%
Students spend less time thinking or deliberating over
material because the whole research process is sped up
by electronic resources
2.
1%
4.
3%
1.
0%
Electronic resources facilitate plagiarism among students
3.
3%
10
.0
%
1.
4%
0.
5%
58.0%
1.
2%
4.
1%
3.
1%
0.
5%
64.0%
1.
9%
0.
2%
Strongly agree (5)
50% 60% 70% 80% 90% 100%
Class Time Spent Instructing Students
How to Evaluate Online Information
N=419
Overall
69.5%
Baccalaureate Colleges
74.0%
Master's
Colleges/Universities
73.7%
64.9%
Research Universities
International Affairs
77.7%
Earth/Environmental
Science
64.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Availability of Courses Instructing Students
How to Evaluate Electronic Resources
N=419
My institution has a required course
3.6%
My institution has an optional course
6.4%
There is no formal mechanism set up to train
students how to evaluate online information
50.8%
Other
6.7%
I don't know/Not sure
32.5%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
The Effect of Electronic Resources on Overall
Quality of Students’ Learning Experience
N=419
Very positively (5)
22.9%
61.3%
Somewhat positively
MEAN
4.00
Neither positively nor
negatively
9.5%
Somewhat negatively
5.3%
Very negatively (1)
1.0%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Satisfaction with Electronic
Resources
Importance of and Satisfaction with ER
3.75
Full text available
2.85
3.70
Ability to download materials
3.02
3.66
Search capabilities of database
2.89
Importance
3.65
Ease of navigation
2.87
Satisfaction
3.67
Ability to print materials
3.17
3.66
Currency of information
3.14
3.50
Availability of older/archived information
2.63
2.00
2.50
3.00
3.50
4.00
Importance of and Satisfaction with ER
3.42
Speed of site
3.06
3.30
Reputation of provider
3.19
The ability to tell if a
resource on the site is
peer reviewed
3.27
2.66
Importance
Satisfaction
3.07
Text is in preserved
format
3.03
3.01
Availability of high quality
images/graphics
2.86
2.91
No required login
2.88
2.00
2.50
3.00
3.50
4.00
Important and
Satisfied
•
•
•
•
•
Ability to download
Ability to print
Currency of info
Speed of sites
Reputation of
providers
• Text in PDF or other
standard format
Important and
Not Satisfied
•
•
•
•
Full text available
Search capabilities
Ease of Navigation
Availability of
older/archived
information
• Ability to tell if peer
reviewed
Other suggestions
•
Reduce price
• Standardize navigation and search procedures across
databases
• Synthesize databases so that they are accessible
through a single portal
• Faster turn-around time/More frequent updates
• Clearly identify or label sources
Conclusions
• Electronic resources have become the main tool for scholars’
information gathering.
• Electronic resources are seen as providing convenience more
so than increased availability of information.
• The main problem with electronic resources is having to
separate out reliable from unreliable information.
• Most faculty believe students have an enhanced learning
experience because of the information available through
electronic resources.
• Future efforts could focus on providing increased access to
older and full text material, and improving search and navigation
capabilities.
Christina Norman, Ph.D.
Research Director, EPIC
Columbia University
211 International Affairs Building
420 West 118th Street, Mail Code 3301
New York, NY 10027
[email protected]
EPIC evaluation web site:
http://epic.columbia.edu/eval/