Assessment and the Academic Librarian

Presentation to Iowa Private Academic Libraries Conference March 16, 2005 Curtis J. Taylor

Outline of Presentation

      What is Assessment?

 Definition    Themes of Assessment 9 Principles of Good Practice in Assessing Student Learning On-Going Phases of Assessment Direct vs Indirect Assessment Why use Assessment Methods?

 Accountability vs Improving Student Learning What Types of Assessment may be Useful and Appropriate in an Academic Library?

Examples of Assessment Instruments and Methods for an Academic Library Discussion

What is Assessment?

   Assessment is an ongoing process aimed at understanding and improving student learning.

Assessment involves:   making our expectations explicit and public; setting appropriate criteria and high standards for learning quality;   systematically gathering, analyzing and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance.

Assessment helps us create a shared academic culture dedicated to assuring and improving the quality of higher education. (Thomas A. Angelo,

AAHE Bulletin,

November 1995, p.7, http://www.aahe.org/assessment/assess_faq.htm#define , accessed March 14, 2005)

Themes of Assessment

     Assessment should focus on improving

student learning

; The focus of assessment should not be limited to the classroom, but include the wide range of processes that influence learning; Assessment is a process embedded within larger systems; Assessment should focus collective attention and create linkages and enhance coherence within and across the curriculum; and Tension between assessment for improvement and assessment for accountability must be managed. (Thomas A. Angelo,

AAHE Bulletin,

April 1995, p.11)

9 Principles of Good Practice in Assessing Student Learning

1.

Values

The assessment of student learning begins with educational values.

2.

Integrated

Assessment is most effective when it reflects an understanding of learning as multidimensional, integrated, and revealed in performance over time.

3. Clear Purposes Assessment works best when the programs it seeks to improve have clear, explicitly stated purposes.

4.

Outcomes and Experiences

Assessment requires attention to outcomes but also and equally to the experiences that lead to those outcomes.

5. Ongoing Assessment works best when it is ongoing not episodic.

9 Principles of Good Practice in Assessing Student Learning

6.

Broad Representation

7.

Assessment fosters wider improvement when representatives from across the educational community are involved.

Use

8.

Assessment makes a difference when it begins with issues of use and illuminates questions that people really care about.

Change-Oriented

Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.

9.

Accountability

Through assessment, educators meet responsibilities to students and to the public.

Authors:

Theodore J. Marchese; Kay M. McClenney; Marcia Mentkowski; Margaret A. Miller; E. Thomas Moran; Barbara D. Wright ( Alexander W. Astin; Trudy W. Banta; K. Patricia Cross; Elaine El-Khawas; Peter T. Ewell; Pat Hutchings; http://www.aahe.org/assessment/principl.htm

, accessed March 14, 2005)

http://www.cod.edu/Dept/Outcomes/AssessmentBook.pdf

, accessed March 14, 2005

Indirect Assessment vs. Direct Assessment



Indirect Assessment

  Surveys and Questionnaires (What did you learn? How satisfied are you? Was it useful?) Independent Reviewers (This is what we observed…) 

Direct Assessment

 Pre-Test/Post Test (Test…Learn/Experience…Test)   Focus Groups Information Literacy measures  Course Embedded (Ongoing, tied to goals/objectives)  Portfolios (critical examination of work)

Why use Assessment Methods?

  Accountability  Increasingly, we are called to account.

  It is appropriate to count books and journals. However, counting books is not enough to measure student learning outcomes.

Improving Student Learning  What is your mission? Why do you exist? How do you connect to the greater institutional mission? How do your efforts impact student learning?

 What are your goals/objectives? What specific things do you attempt to do each and every day? Why do you do them? Could they be done more effectively?

 How do you know when you have been successful? What measurements do you use to assess whether you are meeting your goals/objectives?

Assessment for Accountability

  Often done for accreditation (HLC) or college guides (e.g. US News and World Report)  What do we have?

  Do we need these resources/services? Is more always better?

How often are they used? Could the resources be put to better use?

   What do we need?

  Must, Ought, Could system What are students and faculty requesting? Are there new constituencies that we could serve within our mission?

What are we missing?

  What new services would be used if we had them?

What could we eliminate if we really needed to?

What roadblocks prohibit maximum usage of resources or services?

 Do we need to change our system?

Data are often collected for the sake of collecting data. Unused data have no value.

Taking the next step….

   Move from accountability to improving student learning.

Others on campus are doing it too      Assessment Committees Program Review Academic Program Prioritization Admissions Committees and Retention Councils Student Portfolio Development Collaboration and Broad representation   On campus collaboration: use the data that are already collected Inter-campus collaboration: share data and results across campuses

Improving Student Learning

 New Measures Initiative of the ARL         User Satisfaction Market Penetration Ease and Breadth of Access Library Impact on Teaching and Learning Library Impact on Research Cost Effectiveness of Library Operations and Services Library Facilities and Space Organizational Capacity Blixrud, Julia C. "Mainstreaming New Measures."

ARL

, no. 230/231 (October/December 2003): 1-8. .

Improving Student Learning

 

National Survey of Student Engagement (NSSE) Two new experimental items were added in 2004

 Using online resources or tools (e.g. Internet, email, course or library websites, search engines) for academic purposes  Using a computer offline for academic purposes  National frequencies and descriptive statistics; Carnegie classification frequencies and descriptive statistics; and institutional frequencies and descriptive statistics

Improving Student Learning

 College Student Survey (CSS)  Additional Questions may be added to the survey instrument. In 2004 Dordt College added the following questions:  In your experience at Dordt during the current school year, about how often have you done each of the following  Used an index or database to find material on some topic  Asked a librarian for help in finding information on some topic  To what extent has your experience at Dordt contributed to your knowledge, skills, and academic development in the following areas  Finding and refining research topics      Finding information in the library (print or electronic Critically evaluating information sources you found Understanding and avoiding plagiarism Providing correct citations for all sources used Presenting information effectively in writing, online, or oral presentation

Improving Student Learning

 College Student Experiences Questionnaire (CSEQ) Library Experiences Scale 

In your experience at this institution during the current school year, about how often have you:

 Used the library to study         Found something interesting while browsing Asked a librarian/staff member for help Read assigned material not texts Used index or database to find material Wrote bibliography for a term paper Gone back to read basic reference Made a judgment about quality of information Frequencies available by gender, class, race, and Carnegie classification ( http://www.indiana.edu/~cseq/pdf/CRLarticle_Kuh_Gonyea.pdf

, accessed March 14, 2005)

Reflection Items

   Where does my library need assistance?

    Accountability measures?

Linkages to Student Learning?

Developing direct or indirect measures?

Creating a local instrument? Selecting a national instrument?

Where can I find the assistance I need?

   Someone on my campus that is already doing this?

Someone in this room?

A peer review team from a professional organization?

What resources are available on my campus?

  CSEQ? NSSE? CSS?

Local surveys of students? Faculty? Alumni?