Transcript Slide 1
New Characteristics of Excellence Information Literacy & Student Learning: The Big Picture & Regional Portraits Historically Black Colleges & Universities Library Alliance, Savannah, GA Dr. Terrence Mech King’s College, Wilkes-Barre, PA October 31, 2006 Change & Evolution “The Major test of a modern U.S. university is how wisely and how quickly it is able to adjust to important possibilities.” Clarke Kerr Change & Fear Loss of power & prestige Loss of resources Loss of autonomy Intrusion into personal & professional domains Change in definition of success Change in reward system Fear of technology Fear of having to relearn Physics of Change An object at rest will remain at rest unless acted upon by a greater outside force. If the pressure to change is not greater than the resistance to change, little will happen--stasis. Accreditation Today Self-regulation and peer review Voluntary responsibility for own improvement Strengthen the quality & integrity of higher education, making it worthy of public confidence Shift away from input standards Emphasis on outcomes Southern Association of Colleges and Schools Principles of Accreditation: Foundations for Quality Enhancement (2001) CR 2.12 Quality Enhancement Plan CS 3.5 Educational Programs: Undergraduate Programs CS 3.8 Library and Other Learning Resources Southern Association of Colleges and Schools CR 2.12 Quality Enhancement Plan The institution has developed an Acceptable Quality Enhancement Plan and demonstrates that the plan is part of an ongoing planning and evaluation process. Southern Association of Colleges and Schools CS 3.5.1 Educational Programs The institution identifies collegelevel competencies within the general education core and provides evidence that graduates have attained those competencies. CS 3.5.1 Educational Programs Relevant Questions for Consideration What are the specific college-level competencies within the general education program? What evidence is available to show that students have attained these competencies? Southern Association of Colleges and Schools CS 3.8.2 Library and Other Learning Resources The institution ensures that users have access to regular and timely instruction in the use of the library and other learning/information resources. CS 3.8.2 Library and Other Learning Resources Relevant Questions for Consideration What is the objective and type of assistance available to users? What delivery mechanisms exist for instruction and assistance to library users and how are they assessed? How is the effectiveness of learning resource programs determined? Middle States Association of Colleges & Schools Characteristics of Excellence in Higher Education (2006) Standard 11: Educational Offerings Standard 12: General Education Standard 14: Assessment of Student Learning Standard 11 Educational Offerings The institution’s educational offerings display academic content, rigor, and coherence that are appropriate to its higher education mission. The institution identifies student learning goals and objectives, including knowledge and skills, for its educational offerings. Standard 11 Fundamental Elements Adequate learning resources, facilities & staff Syllabi incorporate expected learning outcomes Programs foster a coherent learning experience Collaboration between librarians & faculty to foster students’ information skills Programs promote students’ use of information and learning resources Standard 12 General Education The institution’s curricula are designed so that students acquire and demonstrate college-level proficiency in general education and essential skills, including at least oral and written communication, scientific and quantitative reasoning, critical analysis and reasoning, and technological competency. Standard 12 Fundamental Elements General education enhances students’ intellectual growth Requirements clearly & accurately stated Skills & abilities developed in general education are applied in the major Assessment of general education outcomes Assessment results used for improvement Standard 14 Assessment of Student Learning Assessment of student learning demonstrates that, at graduation, or other appropriate points, the institution’s students have knowledge, skills, and competencies consistent with institutional and appropriate higher education goals. North Central Association of Colleges and Schools The Handbook of Accreditation (2003) Criterion Three: Student Learning and Effective Teaching Criterion Four: Acquisition, Discovery, and Application of Knowledge North Central Association of Colleges and Schools Criterion Three: Student Learning & Effective Teaching The organization provides evidence of student learning and teaching effectiveness that demonstrates it is fulfilling its educational mission. Student Learning & Effective Teaching 3a. Organization’s goals for student learning outcomes are clearly stated for each educational program. 3b. Organization values and supports effective teaching. 3c. Organization creates effective learning environments. 3d. Organization’s learning resources support student learning and effective teaching North Central Association of Colleges and Schools Criterion Four: Acquisition, Discovery, and Application of Knowledge The Organization promotes a life of learning for its faculty, administration, staff, and students by fostering and supporting inquiry, creativity, practice, and social responsibility in ways consistent with its mission. Acquisition, Discovery, and Application of Knowledge 4a. Organization demonstrates, through the actions of its board, administrators, students, faculty, and staff, that it values a life of learning. 4b. Organization demonstrates that acquisition of a breadth of knowledge and skills and the exercise of intellectual inquiry are integral to its educational programs. Acquisition, Discovery, and Application of Knowledge 4c. Organization assesses the usefulness of its curricula to students who will live and work in a global, diverse, and technological society. 4d. Organization provides support to ensure that faculty, students, and staff acquire, discover, and apply knowledge responsibly. Align your Outcomes MissionObjective/GoalOutcomes Institutional outcomes Program outcomes Course outcomes Class outcomes Today’s High Education Articulation & assessment of outcomes Student centered approach to course & curriculum development Emphasis on student understanding & capabilities rather than fact-learning Emphasis on undergraduate research Preparing students to apply their knowledge in non-academic settings (NEA 2001) Questions? Rising to the Challenge 40% of HS graduates are not prepared: 39% of college students & HS graduates report having gaps in the skills and abilities expected today. 35% of college students & 39% of HS graduates have large gaps in at least one crucial skill; 86% of both groups have gaps. College instructors---42% of their students are not adequately prepared. Employers---39% of HS graduates are not prepared for their current job & 45% are unprepared for advancement. Most Grads Cite Gaps In At Least One Skill 35% of college students report large gaps in at least one area, 86% report some gaps in at least one area. College students Non-students Oral communication/ public speaking Science 13% 16% Doing research 10% 13% Reading/understanding complicated materials 44% 11% 14% Mathematics Quality of writing that is expected 45% 46% 12% large gaps/struggling 15% large gaps/struggling 9% 10% 5% 9% 51% 42% 41% 40% 45% 35% 38% 29% 33% Employers/Instructors Dissatisfied With High Schools’ Skills Prep College instructors Employers Reading/understanding complicated materials 25% very dissatisfied Quality of writing that is expected 22% very dissatisfied Doing research 24% very dissatisfied 70% 41% 62% 40% 59% 29% Mathematics 52% 20% very dissatisfied 32% Oral communication/ public speaking 40% 34% 36% Science 24% Employers/Instructors Dissatisfied With High Schools’ Skills Prep College instructors Thinking analytically Employers 29% very dissatisfied 66% 42% Work and study habits Applying what is learned in school to solving problems 22% very dissatisfied 16% very dissatisfied 50% 17% very dissatisfied 55% 39% 20% Computer skills 65% 17% How do you know your students & graduates have the information skills they will need? Why Information literacy ? Builds on general education skills Reading Critical thinking & reasoning abilities Written & oral communication skills It is actively student focused It is assessable Strong information skills have a positive effect on student academic success Information Literacy “To be information literate, a person must be able to recognize when information is needed and have the ability to locate, evaluate, and use effectively the needed information.” American Library Association Presidential Committee on Information Literacy: Final Report, 1989 Information Literacy’s Common Themes Recognizes a need for information Engages in information seeking behavior Explores, accesses and locates materials Interacts with the information to formulate hypotheses Synthesizes, interprets. organizes, applies and communicates the information Evaluates the results (Eisenberg & Brown, 1992) “Access to computers, computer applications, web-enhanced courses, and information resources such as books, journals, and databases does not translate into information literate students.” Developing Research & Communication Skills, p.6 Information literacy is not about libraries. Information literacy is about teaching and learning. From Teacher Centered to Learner Centered Cover the material VS. using the material Cover topics VS. mastering learning objectives Listening/reading VS. constructing knowledge through integration From Teacher Centered to Learner Centered Lecture VS. active learning Present VS. engage Grades VS. classroom assessment Teaching content VS teaching students Sage on the stage VS. designer of learning environments Metacognitive Approaches “How can students gain the skill to manage their own thinking, change unproductive search strategies, monitor their time and attention, know what they know and what they need to learn?” Diane Halpern The Recursive Nature of Learning Determine the Information Needed Access Information Evaluate Information Use Information Understanding Model of the Information Search Process Tasks Initiation Selection Exploration Formulation Collection Presentation ----------------------------------------------------------------------------------------------------------------------------------------------- Feelings uncertainly optimism confusion clarity (affective) frustration doubt Thoughts (cognitive) sense of satisfaction or direction/ disappointment confidence vague-------------------------------------→focused -------------------------------------------→ increased interest Actions seeking relevant information--------------→seeking pertinent information (physical) exploring documenting Carol Collier Kuhlthau, Information Search Process, Rutgers University Cognitive development Ways of Knowing. Baxter Magolda (1992) Domains Absolute Knowing Transitional Knowing Independent Knowing Contextual Knowing Nature of Knowledg e ● Is certain or absolute ● Is partially certain and partially uncertain ● Is uncertain – everyone has own beliefs ● Is contextual; judge on basis of evidence in context Role of learner ● Obtains knowledge from instructor ● Understands knowledge ● Thinks for self ● Shares views with others ● Creates own perspective ● Exchanges and compares perspectives ● Thinks through problems ● Integrates and applies knowledge Role of instructor ● Communicate ● Uses methods knowledge aimed at appropriately understanding ● Ensures that ● Employs students methods that understand help apply knowledge knowledge ● Promotes independent thinking ● Promotes exchange of opinions ● Promotes application of knowledge in context ● Promotes evaluative discussion of perspectives ● Student and teacher critique each other Absolute Transitional Independent Contextural Figure 2.1. Ways of Knowing by Year. Baxter Magolds (1992) p. 70-72 90% 83% 80% 80% 70% 68% 57% 60% 53% 50% 46% 40% 32% 31% 30% 16% 20% 12% 11% 10% 1% 5% 1% 2% 2% 0% Freshman Sophomore Junior Senior Year 5 Why Information Literacy ? More time & energy students invest in activities related to desired outcomes, the more likely they are to achieve those outcomes Educationally effective institutions design experiences that channel students’ energies to purposeful activities Questions? Information literacy is a curriculum issue. The Three Curricula One in the catalog One professors teach One students experience Where in the Curriculum Where do students learn & enhance their information skills? Institution, program, course levels Separate course, blocked courses, integrated General education & major Where & how do you assess it? How do you document that your students are information literate? An Information Literacy Curriculum is: University-wide Inquiry and resource based Makes effective use of instructional technologies and communication tools Learner centered Integrated with learning outcomes in general education and the disciplines Higher Education’s Dark secrets Despite our rhetoric about teaching higher order skills—critical thinking & problem solving many faculty focus on the acquisition of knowledge (Cashin & Downey, 1995) Students learn what the professors emphasize A week has 168 hours Full-time students spend about 16 hours a week in class That means 90% of the time students are NOT in class When, where, and how does most the most effective learning take place? Eder Change Assignments & Syllabi Start with learning outcomes Communicate learning outcomes Create assignments that work Create a sequences of assignments Progressive in sophistication Active & problem-based There is more teaching going on around here than learning and you ought to do something about that. Graduating Senior King’s College, 1968 Assessment “Assessment is an ongoing process aimed at understanding and improving student learning.” Tom Angelo, AAHE Bulletin, 1996 Assessment is not about us, it is about student learning and institutional effectiveness. National Survey of Student Engagement NSSE--a “leading tool for assessing the quality of the undergraduate experience.” Indiana University Center for Postsecondary Research and Planning www.iub.edu/~nsse Academic Challenge Number & size of written projects Course work emphasizes Synthesizing and organizing ideas, information, or experience Making judgments about the value of information, arguments, or methods Applying concepts to practical problems or new situations Connecting the Dots Engagement has positive effects on grades and persistence for students from different racial and ethnic backgrounds, even after controlling for key pre-college variables. Engagement has compensatory effects on first-year grades and persistence to the second year of college for historically underserved students. p.3 Direct Evidence of Student Learning Student assignments Standardized tests Course embedded assessments Portfolios of students’ work Capstone experiences Student performances & exhibits Other observations of student behavior Direct Evidence of Student Learning Rubrics & exemplars Concept maps Juried/peer review of student projects Performance on a case study or problem Locally devised tests Commercially produced tests Project SAILS Kent State University Libraries ACRL competencies 12 skill sets 40-50 multiple-choice items per test, 35 minutes Subject-specific modules Communication Studies, Education, Biology, History Project SAILS Proven validity & reliability .80 Assesses at the institutional level External and internal benchmarking Web & paper format Price-$3.00 per completed test, cap of $2,000 per administration Educational Testing Services “ICT proficiency is the ability to use digital technology, communication tools, and /or networks appropriately to solve information problems in order to function in an information society. This includes the ability to use technology as a tool to research, organize, evaluate, and communicate information and the possession of a fundamental understanding of the ethical / legal issues surrounding the access and use of information.” ICT Literacy Assessment Two versions—Core & Advanced Core--high school seniors, 1 & 2 year college students Advanced--rising college juniors Uses ACRL Competencies 7 skill sets Scenario-based tasks (75 minutes): 14 short 4-minute tasks & 1 medium 15-minute task ICT Literacy Assessment Web format Assesses at the student level Price-$33.00 per test www.ets.org/ictliteracy Information Literacy Test (ILT) James Madison University Uses ACRL Competencies 1, 2, 3, 5 Assesses at student level 12 skill sets 60 items, reliability of .88 Computerized, multiple-choice test Price-$10.00 (1-100 tests), $7.00 (101-500) $5.00 (501 or more) www.jmu.edu/assessment/resources/ instruments_ILT.htm South Dakota Regental Information Literacy Exam Uses ACRL Competencies 25 multiple-choice questions Reliability & validity Assess at the student level WebCT administered Low cost Dr. Carol Leibiger, Information Literacy Coordinator, USD [email protected] 605-677-6089 Information Literacy Assessment King’s College Uses ACRL Competencies 25 multiple-choice questions Assess at the student level Reliability & validity No cost Dr. Terrence Mech Library Director, King’s College [email protected] 570.208.5943 Information Skills of Freshmen from Three Private Colleges: Mean Scores/Percentage Fall 2005 N Composite Scores Std 1 Std 2 Std 3 Std 4 Std 5 Freshmen 1147 48.31 51.02 46.57 51.87 43.49 48.58 College A1 254 44.77b 42.52b 44.49 42.99b 43.62 50.24a College B2 730 50.45a 54.16a 48.00 56.05a 44.68a 49.37a College C3 163 44.20b 50.18a 43.44 46.99b 37.91b 42.45b Class abRefers to comparisons within column where the MEAN scores of group a are significantly (P< .05) higher than the MEAN scores of group b. Carnegie Classification of Institutions of Higher Education (2001) 1Master’s Colleges and Universities II 2Doctoral/Research Universities – Extensive 3Master’s Colleges and Universities I Information Skills of Freshmen from Three Private Colleges: Mean Scores/Percentage Fall 2005 Class Freshmen College A1 College B2 College C3 N 1147 254 730 163 Composite Scores 48.31 44.77b 50.45a 44.20b Knowledge Application 48.54 40.75b 52.32a 43.76b 48.09 48.49 48.74a 44.60b abRefers to comparisons within column where the MEAN scores of group a are significantly (P< .05) higher than the MEAN scores of group b Carnegie Classification of Institutions of Higher Education (2001) 1Master’s Colleges and Universities II 2Doctoral/Research Universities – Extensive 3Master’s Colleges and Universities I Seniors’ Information Skills by Standard: Mean Scores/Percentage Correct - Fall 2005* Major N Composite Scores Std 1 Std 2 Std 3 Std 4 Std 5 Seniors 166 58.82 63.25 56.87 60.24 57.47 56.27 Major A 7 33.71b 40.00b 28.57b 25.71b 34.29b 40.00 Major B 3 38.67d 46.67 40.00 40.00 40.00 26.67 Major C 48 57.08a 58.75 57.08a 57.50a 55.83 56.25 Major D 29 57.38a 66.21a 50.34 59.31 57.24 53.79 Major E 15 62.40a 69.33a 57.33 64.00a 62.67 58.67 Major F 4 63.00a 65.00 45.00 70.00 80.00a 55.00 Major G 17 63.06a 64.71 63.53a 75.29a 51.76 60.00 Major H 11 63.27a 67.27 67.27a 65.45a 63.64 52.73 Major I 32 64.13a,c 68.75a 64.38a 61.88a 62.50 63.13 ab, cd refers to comparisons within column where the MEAN scores of group a, c are significantly (P< .05) higher than the MEAN scores of group b, d. *Please note that in some cases the small group size and number of questions per standard may preclude more meaningful statistical comparisons. Seniors’ Information Skills by Standard: Mean Scores/Percentage Correct - Fall 2005* Major N Composite Scores Knowledge Application Seniors 166 58.82 62.44 55.46 Major A 7 33.71b 36.90b 30.77b Major B 3 38.67d 44.44 33.33 Major C 48 57.08a 58.68a 55.61a Major D 29 57.38a 60.34a 54.64a Major E 15 62.40a 68.33a 56.92a Major F 4 63.00a 66.67 59.62 Major G 17 63.06a 70.10a 56.56a Major H 11 63.27a 67.42a 59.44a Major I 32 64.13a,c 68.23a 60.34a ab, cd Refers to comparisons within column where the MEAN scores of group a, c are significantly (P< .05) higher than the MEAN scores of group b, d. *Please note that in some cases the small group size and number of questions per standard may preclude more meaningful statistical comparisons. Mass Comm 115 & 493 Students’ Information Skills: Mean Scores/Percentage Correct Fall 2006 Group N Composite Scores Std. 1 Std. 2 Std. 3 Std. 4 Std. 5 Knowledge Application Comm 115 28 45.57 55.71 42.14 47.86 35.00 47.14 44.34 46.70 Comm 493 31 55.87* 70.32* 52.26* 53.55 49.68* 53.55 62.90* 49.37 *Refers to comparisons within the column where the MEAN score of one group is significantly (p<.05) higher than the MEAN score of the other group. OSMOSIS Collaboration: Is based on a common premise Is a difficult challenge in contemporary higher education Is necessary to create studentcentered approaches Is a cultural issue Barriers to Collaboration Graduate schools prepare specialist Departments hire specialist Most scholarship is conducted alone Promotion & tenure favor individual achievements 15/35 In any organization: 15% leaders, opinion shapers (Includes dreamers) 35% willing followers, early adopters --------------------------------------------- 35% reluctant followers, late adopters 15% curmudgeons Eder 15% Curmudgeons 5% already busy and productive Leave them alone 5% reasoned skeptics Take them to lunch 5% just plain mean Leave them alone Whose Job is It? Everybody Somebody Anybody Nobody Questions?