Transcript TracDat Overview - Ferris State University

TracDat Overview
ROBBIE TEAHEN
ROBERT VON DER OSTEN
KIM WILBER
One Approach to Outcomes
http://www.youtube.com/watch?v=DRBW8eJGTVs
Purpose of Assessment
Simply: To advance the quality of student learning
through careful elaboration of intended learning,
meaningful measures of students’ learning
achievements, and systematic collection of data that
informs instructional and other improvements – at
the level of courses, programs, colleges, and
institutions. Assessment involves going beyond the
evaluation of individual student performance
(Teahen, 2008).
The Learning Design Cycle
Determine learning
Needs
Assess
performance
Analyze learner
needs
Curriculum
Development
Specify learning
outcomes
Plan learning
activities
Session Outcomes
 As a result of participating in today’s session, you will:








Demonstrate your ability to log in to the system
Change your password
Describe the meaning of relevant terms, such as “reporting” or
“assessment” units and “learning outcomes”
Create an appropriate learning outcome
Enter assessment unit (e.g. program) information (if unit admin)
Create at least one learning outcome entry
Know the tentative implementation plans/timelines – pending
further refinement
Be prepared to continue to enter target outcomes/goals, assessment
plans, and your results on an ongoing basis
Goals of the Implementation
 Create a repository for conserving assessment




information
Stimulate cross-discipline sharing of learning and
institutional effectiveness outcomes
Focus unit-level efforts on specifying and monitoring
assessment outcomes
Encourage course-, program-, and institutional-level
learning enhancements
Streamline reporting requirements for academic
units
Topics to be Included Today
 User roles
 Access
 Changing the password
 Entering data – outcomes, plans, and results
 Attaching documents
 Linking assessment unit outcomes to reporting unit
standards or plans
 Producing reports
 Resources (Documents, “power users,” and
Oversight team, Kim)
Roles and Security
 Roles:
 User
 Reports-Only
 Program-level admin
 Ferris Administrator (Kim Wilber)
 Access and Security levels and permissions:
 Form (see Academic Affairs assessment website)
 Approvals by Deans
Attempting to Allay Common Concerns
 Comparing instructor performance . . .
 Outcomes and measures are expected to be highly unique, so it will not be
possible to compare “performance” across programs or disciplines
 Instructor Evaluation
 Instructors will continue to be evaluated by existing methods. If they want to use
this information to demonstrate their effectiveness, it’s their option.
 What Information is Reported
 Inclusion is decided at the appropriate level. . . What the program personnel put
in is what they decide to enter. Most reporting will be at the program level, and
course-level reported is expected to evolve at the
 This Too Shall Pass. . .
 Assessment of student learning has been required since 1995. Expectations have
heightened.
 I Don’t Have Time (It’s too much work) . . .
 Assessment is integral to the work of a professional educator. It is a part of the
cycle of instructional design and the continuous improvement approach of
“plan, do, check, act”
Assessment Unit Information
 Show/Tell/Do (using the Demo Site)
 Unit Name
 Mission
 Sites Offered
 Accreditation Body
 Date of next accreditation visit
 Advisory Board?
 Next program review data
Assessment Plan Information
 Show/Tell/Do (using Demo)
 Outcome name
 Outcome statement
 Assessment Method
 Criterion for Success
 Related Goals
 Additional assessment methods
Outcome Checklist
 describe one of the major skills that is an intended outcome for a course or
program
 represent a skill that a competent individual would use outside the context
of the course
 begin with an action verb describing what the learner will be able to do
upon completion of this course/program
 are measurable and observable
 require application of skill, knowledge, or attitude/value – at a level of
application or above on Bloom’s taxonomy
 present a clear, concise, and precise statement describing the action
 specify a single performance/outcome, not a combination
 describe learner performance, not the instructor's activities, learning plans,
or instructional strategies
Sample Program-Level Outcome and Measure(s)
 Outcome: Technology Use
 Outcome Statement: Learners will demonstrate their use
of common functions associated with software relevant to the
discipline (e.g. MS Office, SPSS, CAD, etc.)
 Measure(s):
1.
2.
3.
Capstone project assignments will incorporate the utilization of
common software applications associated with the field. Rubrics will
be provided for each that address learner performance in technology
use.
Exams in the second-year major course will incorporate timed tests
utilizing identified software to produce documents appropriate to
meet external performance requirements. Standards of the profession
will be utilized to assess learners’ performance.
Throughout the program, individual course requirements will
incorporate and report on technology-use performance by students, as
appropriate
Components to Enter in Plan (Program)
 Outcome Name
 Outcome
 Measures
 Outcome Type
 Status (active or not)
 Star
Don’t Forget!
SAVE CHANGES (button at bottom)
“Means of Assessment”
 Relating to a particular outcome, specify:
 Method Category
list still being refined
 Offer your suggestions



Method (Description)
Criterion


What will success look like for this program or course?
Schedule

When will assessment methods be implemented? Frequency?
Sample Results and Action Plan
 Learners will demonstrate their use of common functions associated with
software relevant to the discipline (e.g. MS Office, SPSS, CAD, etc.)
 Measure(s):
1.
Capstone project assignments will incorporate the utilization of common software
applications associated with the field. Rubrics will be provided for each that address
learner performance in technology use.
Results:


Review of 32 capstone projects for students in the X program during the spring of 2008
revealed that 95% of the learners were able to perform all specified functions within MS
Word and Powerpoint, but just 62% could demonstrate their abilities to perform specified
functions within Access. Further, AutoCAD design capabilities were rated to be at an
average level of 3 on a scale of 1 to 5, with 10% of the soon-to-graduate students not
meeting minimum standards for the profession.
Action Plan:


Faculty within the major will meet in August 2008 to examine the curriculum to
determine where and how Access and CAD are introduced and reinforced and
development supplemental units to assist students to more adequately achieve intended
outcomes. Faculty meetings will address this performance concern and monitor
curricular changes and end-of-year performance of students in spring 2009. Results from
the Spring 2009 will be reported and determination made about whether additional
curricular reform is required.
Linking Documents and to Goals
 Attach documents. . .
 Examples:
Rubrics
 Standards
 Assignments

 Link to standards or criteria
 Such as accreditation standards
 Program outcomes
 General education outcomes
Tentative Timelines
 Enter Unit information immediately (mission,




accreditor, etc.)
Enter Program Outcomes and Measures before end
of semester
Enter at least one program outcome by end of spring
Enter multi-section and general education course
assessment plans during spring 2009
Enter outcomes as available for course-level
assessments
Regarding the Learning Paradigm:
"The result of this paradigm shift is a
college where faculty are the designers
of powerful learning environments,
where curriculum design is based on an
analysis of what a student needs to know
to function in a complex world rather
than on what the teacher knows how to
teach, where the college is judged, not
on the quality of the entering class, but
on the quality of aggregate learning
growth possessed by its graduates,
where compartmentalized departments
are replaced by cross-disciplinary
cooperatives, and where every employee
has a role to play and a contribution to
make in maintaining a learner-centered
environment (p. 5).”
-Bill Flynn, Palomar College, 1998