Transcript Enterprise-Based E-Learning Planning and Evaluation

Enterprise-Based E-Learning
Planning and Evaluation in a
Large-Scale Operation
© Copyright Nicola Martinez and Susan Oaks 2003. This work is the
intellectual property of the authors. Permission is granted for this
material to be shared for non-commercial, educational purposes,
provided that this copyright statement appears on the reproduced
materials and notice is given that the copying is by permission of the
authors. To disseminate otherwise or to republish requires written
permission from the authors.
Presenters
 Susan Oaks, Ph.D.,
Academic Area Coordinator, Writing, Literature and
Communications; Chair, Curriculum Committee
 Nicola Martinez, MA,
Coordinator of Instructional Design and Curriculum
Development
Center for Distance Learning
SUNY Empire State College
Saratoga Springs, New York
Objectives




Demonstrate an integrated planning & tracking
database for course development, review, and
evaluation
Discuss context/issues leading to database
Discuss collaborative approach—and multiple
uses of database—by faculty, instructional
designers, term prep staff
Focus on quality assurance and applicability to
other institutions
Who We Are:
Empire State College
Locations around
New York State
The Center for
Distance Learning
(CDL) serves adult
students around the
world
Mission Statement
Empire State College is dedicated to:
 enabling motivated adults
 regardless of geography or life circumstance
 to manage and master a rigorous academic program
 and earn a degree
Areas of Study
Areas of Study:
The Arts | Business, Management & Economics |
Community & Human Services | Communications,
Humanities & Cultural Studies | Educational
Studies | Historical Studies | Human Development
| Labor Studies | Science, Math & Technology |
Social Theory, Structure & Change
Center for Distance Learning
offers asynchronous online courses in five, 15week semesters per year
leading enroller in the SUNY Learning Network
partner with several institutions such as eArmyU &
Navy, public & private organizations, labor unions
Center for Distance Learning
Course Statistics
approximately 5,000 students annually
200 distinct online courses
more than 350 course sections
over 6000 enrollments per term
46 courses developed for fall web delivery
50+ courses in development for Jan 04
existing courses scheduled for cyclical delivery
Context: Rapid Online
Growth




Increase in f/t faculty from 8 to 15 (and growing)
within past two years
Hire two Coordinators of Instructional Design and
Curriculum Development to shepherd faculty
developers through course development process
and coordinate instructional support
Move from online business degrees to online
degrees in many areas of study
Move all existing courses online (over 200)
Courses in development
Some courses in development this term:
Artistic Expression in a Multicultural America
Introduction to Visual Literacy
Dance Across World Cultures
Food and Drink in Cultural Context
Reflective Learning
Biology of the Brain
Cognitive Psychology
Minds and Machines
Urban Studies
Chemistry in Context
Previous Course
Development System
 Elected Curriculum committee (4 faculty) + 1
instructional designer
 All faculty review & comment on course proposals
 Curriculum committee review, vote, and respond
 E-mail records of issues/outcomes
Problems with
Previous System




Not representative of all broad disciplinary areas
Sparse review & feedback on proposals
E-mail records time consuming; inconsistent
depending on committee chair
Course development relates to other systems
(instructor training, book ordering, quality
evaluation); maintenance of multiple, discrete
systems
New System:
CourseTrak
A Collaboration Between:



The Center for Distance Learning
Dean: Meg Benke
The Center for Learning and Technology
Director: Evelyn Ting
The Office of Educational Technology
Vice President: Patricia Lefor
New System:
CourseTrak




Integrates information for all players: instructional
designers, faculty, support staff
Creates a public record from course proposal
through review and offerings
Fosters collaboration through its public nature
Facilitates evaluation & review
CourseTrak:
Faculty Perspective





Reminds course developer/Area Coordinator of
required information,objectives, content and
learning activities
Requires rationale for new courses in terms of
academic skills, student population, etc.
Invites other faculty collaboration
Facilitates quality review process
Enlists staff help with materials ordering,
scheduling, etc.—integrates systems
CourseTrak:
Instructional Design Perspective
For large scale development of pedagogically
and academically sound courses infused with:
Active learning
Media-rich content
Student interaction
Opportunities for Teaching, Social and Cognitive
Presence
Constraints
WHILE EXPERIENCING A RAPID VOLUME INCREASE
IN ENROLLMENTS, COURSES, AREA
COORDINATORS, AND ADJUNCT FACULTY.
Increased volume of courses developed
per term:
from 5-8 for Fall 02
to 46 for Fall 2003
to 50 + for Spring 2004
Our Approach





Develop a new standard for course information
documents and implement them in the courses
Create a new design for CDL web courses to increase
navigability and overall consistencies in look, feel, and
format
Enrich courses with multimedia components and other
graphic enhancements
Include library referenced materials in all courses, with at
least one library-based learning activity per course
Identify and implement best practices in the pursuit of
academic and pedagogical excellence in web course
development for adult learners taking distance learning
courses
A Consultative
Approach
Strategies for the development and achievement
of these objectives included consulting with:
 Curriculum Committee
 CDL Faculty
 The Curriculum Development Team
 The L-Guide Focus Group
 The Center for Learning and Technology Library
And Multimedia Instructional Design Staff.
Course Ratings
 We established a three-tiered categorization of
courses based on the inclusion of the above
recommendations and the application of a new
course design.
 All new and revised courses are reviewed an
assigned the following ratings:
Course Rating System
Rating Legend
Tier 3
Exemplary course
Tier 2
Acceptable (some criteria not met,
revisions needed)
Tier 1
Not Acceptable
NR
Not Rated
Course Rating System
Tier information and classification is generated as a
result of the information captured in The Empire
State College Online Course Evaluation Checklist.
All courses are considered unrated until they have
been reviewed with this instrument.
The evaluation piece associated with The Empire
State College Online Course Evaluation Checklist
happens at several levels.
Online Course
Evaluation Checklist
 Administered during and after a new course has been
developed, before delivery
 Before an old course is revised to identify specific areas
targeted for improvement
 After an old course has been revised to ensure that it is
complete and meets all baseline criteria for a Tier 3 course
 This will also be used, in the future, to evaluate instruction
The Course Resource
Needs Analysis
 is used to establish the resources, support, services, and
document creation needed for a course in development or
revision.
An instructional designer conducts the needs analysis in
consultation with the course developer (s).
The Course Resource
Needs Analysis
Information captured in the Course Resource Needs
Analysis Form is used to generate work requests for
 technical assistance
 library assistance
 multimedia learning object creation
 digital media research and implementation
 and instructional design/pedagogical support, among
other things.
Focus on Quality
Assurance
 Establish Baseline Criteria for Courses
 Evaluate and Review Courses
 Track Performance
 Capture Data
 Make Necessary Adjustments to Courses
 Create a Data Driven Analysis
 Develop a Report
 Make Recommendations
 Improve Process and Performance Using Results
Applicability to Other
Institutions
This is an example of an in-house, web
based system
 designed to meet our course planning, revision,
and evaluation needs
 and tailored to the culture and goals of our
institution
Applicability to Other
Institutions
Points to Consider in the Development
of such a system:
Level of Institutional Support
# of faculty developing courses
# of new faculty needing intense support
How to integrate the system with other relevant
systems
Access Control and Security Issues
Analyze your assumptions
Contact Information
[email protected]
[email protected]
Empire State College: www.esc.edu
Center for Distance Learning:
www.esc.edu/cdl