Transcript This Is A Title Slide For Template Design

NOTICE
 This presentation and its contents are the jointly held property of the
three coauthors and The Board of Regents of The University of Texas
System.
 Please do not cite any of the materials presented herein without
appropriately referencing the work per the information on the next slide.
 My colleagues and I welcome your feedback; send your e-mail to
[email protected].
 If you plan to use any part of this presentation, please let us know.
Thank you.
-David Silva, Araya Maurice, David Purkiss
 To move through the presentation, press the “enter” key.
The Development and Use of UT Arlington’s
Active Learning Assessment Tool (ALAT)
Araya Maurice*, David Purkiss*, and David J. Silva^
*Office of Institutional Research, Planning and Effectiveness / ^Office of the Provost
Southern Association of Institutional Research Annual Conference
October, 20th 2008
Active Learning at UT Arlington
 …puts the student at the center of the learning process, making
him/her a partner in discovery, not a passive receiver of information.
 …requires students to interact with and integrate course material by
reading, writing, discussing, problem-solving, investigating,
reflecting, and engaging in such higher order thinking tasks ...
 …draws upon such teaching strategies as question-and-answer
sessions, short in-class writing exercises, student research,
internships and community service, team learning, and problembased learning.
3
What is the ALAT?
 “Active Learning Assessment Tool”
 Closely modeled after the Active Learning Inventory Tool (ALIT)
developed at Northeastern University (School of Pharmacy)
 “[A]n ‘inventory tool’ to characterize the use of Active Learning
techniques by faculty” in Pharmacy classes (Devlin et al.)
 Tremendous thanks to colleagues at Northeastern University, including
Dr. Richard Porter
4
Comparing ALIT and ALAT
Northeastern
ALIT
UT Arlington
ALAT
Impetus
Accreditation Council for
Pharmacy Education (ACPE)
and American Association of
Colleges of Pharmacy (AACP)
Quality Enhancement Plan (QEP)
as formulated for reaffirmation by
the Southern Association of
Colleges and Schools (SACS)
Scope
School of Pharmacy
(with plans to generalize)
University-wide
Personnel Multiple trained faculty
observers
Single trained student or staff
observer with periodic second
observer (staff) to verify / calibrate
Coding
Scheme
17 activity types
5 levels of complexity
6 interactive domains
22 activity types
3 levels of complexity
5
Active Learning Inventory Tool
Northeastern University
6
“Active Learning: Pathways to Higher
Order Thinking at UT Arlington”
12 faculty-designed / committee-selected pilot projects
 Time Frame = 3 years (2007-’08, ‘08-’09, ‘09-’10)
 Differing Course Formats
 Large lecture class
 Capstone
 Seminar
 Multiple Disciplines
 Architecture, Business, Education, Engineering (3), History, Honors, Library /
English, Mathematics, Nursing, Political Science
 Varying Interventions
 Personal Response Systems (“clickers”)
 Blogging / On-Line Community-Building
 Fieldwork
 On-line Problem Solving (with immediate feedback)
 Design & Production Projects
7
QEP Measurements
Student Level
 Academic Background
 Demographic Information
 Self-Reported Engagement + Attendance
 Performance
 Critical Thinking Test (CTT)
 Evaluations from a Criterion-Based Course Rubric
Course Level
 Aggregated student-level data
 Course Descriptives including indications of “control” vs. “active learning”
 IDEA
 Focus Group Transcripts
 Active Learning Assessments (ALAT)
8
UT Arlington ALAT: First Iteration
1.
2.
3.
4.
5.
6.
7.
8.


22.
23.
24.
25.
26.
Student offers unsolicited comment.
Student responds to question.
Student asks question.
…
…
…
…
…
.
.
…
Students teach each other in small groups.
Students draw diagrams which interrelate various concepts.
Student exercise in problem solving…
Student problem solve in small groups.
9
UT Arlington ALAT: First Iteration
SPEAK
WRITE
SOLVE PROBLEM
ORGANIZE
PLAY
“CLICKER”
QUIZ
PRESENTATION
10
UT Arlington ALAT: Second Iteration
11
The Deming Cycle
12
The Assessment Cycle
13
UT Arlington ALAT: Third Iteration
14
UT Arlington ALAT: Current Version (2008-09)
15
ALAT Coding, Complexity Level and
Duration Categories
Activity (ALAT Code)
Complexity Level
No Active Learning Activities
0
Active Learning Activities: 1, 5, 12
1
Active Learning Activities: 4, 8, 11, 13, 15, 17
2
Active Learning Activities: 2, 6, 9
3
Active Learning Activities: 3, 7, 10, 14, 16
4
Duration
Category
Active Learning Activity Lasted Less Than 1 Minute
1
Active Learning Activity Lasted Between 1 and 5
Minutes
2
Active Learning Activity Lasted More Than 5 Minutes
3
16
Data Entry: Active Learning Info System
17
ALAT Reliability Issues
Inter-rater:
Performed for the first two semesters of use
Assessment Specialists performed concurrent ALAT
observations and then compared results between raters
ALAT Comparisons (visual): Good match on codes; some
variation on number of occurrences
Intra-rater:
Beginning the third semester of use
Shifted focus to ongoing training and norming of ALAT
observers to promote rater consistency
…Mathematical Model?
18
Data: Active Learning Indices for 2007-08
Stratum
Course
Introductory
Seminar
Capstone
Active
Control
Engineering (Mixed)
0.84
0.72
History
0.68
0.61
Electrical Engineering
0.12
0.10
Political Science
0.18
0.33
Honors (Sophomore)
2.16 (F07), 1.09 (S08)
Honors (Senior)
1.42 (F07), 0.90 (S08)
Mathematics
Nursing
0.45 (F07)
1.57 (F07), 1.83 (S08)
Utility
Measurement of Classroom Activity for IR purposes
Providing instructional faculty with feedback
 Comparing perception with reality
 Realigning as appropriate
Creating an inventory of praxis for chairs and deans
Providing information to central administration on
 Range of teaching praxis campus-wide
 Generalizations as stratified by college, course level, course format
 Directions for future professional development needs
(specific to teaching)
20
Caveats
Measures only what happens in the classroom
Makes no direct claims regarding learning per se
21
Questions / Comments
Araya Maurice, Ph.D.
Assistant Director for Assessment of Student Success
[email protected]
David Purkiss, M.A.
Undergraduate Research and Assessment Specialist
[email protected]
David J. Silva, Ph.D.
Vice Provost for Academic Affairs
[email protected]
22