Transcript COURSE WRAP UP - Simmons College

Conclusion
1
Metrics Revisited
Inputs
 Outputs
 Quality and Satisfaction

 Interrelated.
Rely heavily on subjective responses
and self-reporting

Outcomes
 Impacts
on community- Our effects
2
Outcomes Assessment



Measuring change as a direct result of interaction
with the library
Most often addressed in academic and school libraries
Types of change:
 Knowledge
 Behavior
 Awareness/
Percpetions
 Attitudes/ Opinions
3
Assessment Defined

The ongoing process of:
 Establishing
clear, measurable expected outcomes of
student learning
 Ensuring that students have sufficient opportunities to
achieve those outcomes
 Systematically gathering, analyzing, and interpreting
evidence to determine how well student learning matches
our expectations
 Using the resulting information to understand and improve
student learning [at both the course and program] level
Suskie (2004, p. 3)
4
NEASC STANDARD SEVEN:
Library and Information Resources
1: The institution makes available the library and information resources necessary for
the fulfillment of its mission and purposes. These resources support the academic
and research program and the intellectual and cultural development of students,
faculty, and staff. Library and information resources may include the holdings and
necessary services and equipment of libraries, media centers, computer centers,
language laboratories, museums, and any other repositories of information or
technological systems required for the support of institutional offerings. Clear and
disseminated policies govern access, usage, and maintenance of the library,
information resources, and services. The institution ensures that students use these
resources as an integral part of their education.
2: Through the institution's ownership or guaranteed access, sufficient collections,
information technology systems, and services are readily accessible to students
wherever programs are located or however they are delivered. These collections,
systems, and services are sufficient in quality, level, diversity, quantity, and
currency to support and enrich the institution's academic offerings. The institution
provides facilities adequate to house the collections and equipment so as to foster
an atmosphere conducive to inquiry, study, and learning among students, faculty,
and staff. --- Satisfaction?
5
NEASC STANDARD SEVEN (continued)

3: The institution provides sufficient and consistent financial support for the
effective maintenance and improvement of the institution's library, information
resources, and services. It makes provision for their proper maintenance,
preservation, currency, and security. It allocates resources for scholarly support
services compatible with its instructional and research programs and the needs of
faculty and students.

4: Professionally qualified and numerically adequate staff administer the
institution's library, information resources, and services. The institution provides
appropriate orientation and training for use of these resources, as well as instruction
in basic information literacy.

5: The institution participates in the exchange of resources and services with other
institutions and within networks as necessary to support and supplement its
educational programs. It provides appropriate support for distance learning students
and faculty, such as on-line reference service and contractual access to relevant offcampus library resources.

6: The institution regularly and systematically evaluates the adequacy and
utilization of its library, information resources, and services and uses the results of
the data to improve and increase the effectiveness of these services (satisfaction,
service quality?)
6
NEASC STANDARD Four:
Programs and Instruction

Compliance = outcomes







student learning outcomes
student outcomes
Undergraduate Degree Programs
Graduate Degree Programs
Scholarship and Research
Instruction
Admissions and Retention

http://community.pmc.edu/neasc/standards/Default.htm
7
More on Accreditation

Regional Accreditation- 3 use “information literacy” 3 use
“equivalent language”




NCC makes the point that today’s students must be prepared to be
“knowledge workers” in the sense that they must not just master certain
information, but must be able to comprehend, synthesize and apply that
information. More than just technologically literate, these workers will
be valued for their “capacity to sift and winnow massive amounts of
information in order to discover or create new or better understandings”
Support for collaboration
Professional and Discipline-Specific Accreditors also paying
attention to information literacy
Important to fields focused on Evidenced-Based Practice
8
Student Outcomes versus
Student Learning Outcomes

Student outcomes are aggregate
statistics on groups of students (e.g.,
graduation rates, retention rates,
transfer rates, course and program
completion rates, and job placement:
employment rates for a graduating
class)



Such outcomes are institutional
outcomes and are used to compare
institutional performance
They do not measure changes in
students themselves due to their college
experience
these outcomes are outputs and reflect
what the institution has accomplished;
they do not reflect what (or how much)
students learned

Student learning outcomes —
development of students:
demonstrable acquisition of specific
knowledge and skills

How well do students



Transfer and apply concepts,
principles, ways of knowing, and
problem solving across their major
program of study?
Integrate their core curriculum,
general studies, or liberal studies into
their major program or field of study?
Develop understanding, behaviors,
attitudes, values, and dispositions that
the institution asserts it develops?*
* Maki (2004)
9
Student Learning Outcomes

What should students learn?


How well are they learning it?


Across courses
What evidence do we gather to answer these
questions?


How does the content of one course relate to another?
Beyond graded assignments, course evaluations
How do we use that evidence to improve learning

Looking a courses as comprising a program
10
Purposes of Assessment

Impacting student
learning

Working
collaboratively: thinking
beyond the set of
courses “I” teach

Demonstrating
accountability and
accreditation—
 Meeting
institutional
mission
 Demonstrating
institutional
effectiveness
11
Institutional Effectiveness



Defined as how well an institution achieves its
mission and major institutional goals
“Since student learning is the heart of most
institutional missions, the assessment of student
learning is a major component of the assessment of
institutional effectiveness.”
However, institutional effectiveness does examine
other aspects such as scholarship and research,
community service, etc.
 Suskie, 9-10
12
Student Learning Outcomes
Conceptual
Leadership
Critical Thinking
Creative Thinking
Problem Solving
Information Literacy
Global Citizen
Values (moral, etc.)
Foundations and Skills
for Lifelong Learning
Skills
Oral/written communication
Foreign language communication
Technological sophistication
Quantitative reasoning ability
Other
Do “satisfaction” and “service quality”
impact performance?
13
Direct Methods





Embedded course
assessment (performance on
assignments, etc.; minute
paper)
Portfolio assessment
Performance (internships,
practicum, student teaching)
Professional jurors or
evaluators
Capstone course/experience






Experimental research
designs), with pre- and posttesting
Use of standardized tests
Think-aloud protocol
Directed conversation
Videotape/audiotape
evaluation
Analysis of
theses/dissertations/
senior papers (content
analysis, interviews, or oral
defense)
14
Indirect Methods




Surveys (self-reporting)
and self-assessments
Curriculum and syllabus
evaluation
Exit interviews
Observation

Other
15
A culture of

Assessment
Evidence

Evaluation
16
Have They Learned Anything?
17
So What?
Or Why Evaluation Research?
18
Evaluation Research Addresses
Questions Such As




What improvements in a program, service, or policy
might result in continuous quality improvement and
better accountability?
How well does a program, service, or policy reach its
target population and meet the group’s information
needs and expectations?
Is the program, service, or policy being implemented
in the ways envisioned?
Is it effective?
19
Utility
What are the findings good for?
 What decision in the real world do the findings
impact?
 What insights will the study provide?
 Can the study findings be dismissed with “so
what?”

20
Evaluation (for a Purpose)


Feedback loop (basic
systems model)
Decision making




Accountability
Summary judgment
(summative evaluation)
Continuous [quality]
improvement
Benchmarking and Best
Practices
21
Evaluation Research
Is also a political and managerial activity, an
input into the complex mosaic from which we
can make policy decisions and allocations for
the planning, design, implementation, and
continuance of programs to better the human
 In this sense evaluation research is part of
social policy and public administration

22
Evaluation Research--Steps
Reflective inquiry
Reliability
Validity
Procedures
Conduct study
Analyze data
Prepare/deliver report
Communication
skills
Setting up action plan
23
What are important areas for “evaluation
research”?
______________
 ______________
 ______________

etc.
24
Evaluation of Information Services
Systems Analysis
Other Research Courses*
*Strongly recommended for
anyone planning a career in
academic libraries,
especially university libraries
Courses in Individual Concentrations
25
Scholarly Communication
Key Issues
Time lag between authorship, peer review,
publication, and dissemination
 Information overload

26
Future of Publishing
Our Core Interests
Competitive market
 Easy distribution and reuse
 Innovative applications of technology
 Quality assurance
 Permanent preservation (“archiving”)

27
Electronic Scholarly Publishing
Some Models








Digital library model
Electronic book model
Electronic collection model
Retrospective model
Preprint model
Peer review lite model
Commercial publishing model
Academic server model
28
Electronic Scholarly Publishing
Some Models (continued)
Prestigious publishing model
 University publishing cooperative
 Public domain model
 Government server model

29