Outcomes of a Statewide Educational Interactive Video Program Evaluation

Mark Hawkes Dakota State University Annual Conference of the America Evaluation Association November 5, 2003

The Proliferation of Statewide Interactive Video Systems

Illinois Missouri Montana New Hampshire New Jersey Illinois Video Education Network (IVEN) Missouri Research and Education Network (MOREnet) Montana Educational Telecommunications Network METNET) Granite State Distance Learning Network Garden State Distance Learning Consortium

Ohio Oklahoma Oregon Texas West Virginia

I-TV Proliferation . . .

Ohio Telecommunity Initiative OneNet State Distance Education Program Texas Education Telecommunications Network (TETN) West Virginia Education Network

DDN Room—Dakota Valley

DDN Room—Sturgis

DDN Room—Madison, H.S.

DDN Room—Lemmon H.S.

DDN Room—Redfield H.S.

DDN Room—Pollock H.S.

DDN Room—Watertown H.S.

DDN Room—Watertown M.S.

DDN Room—Canton H.S.

DDN Room—Shannon Co. H.S.

DDN Room—DeSmet H.S.

DDN Room—Faith H.S.

Study Process

 Funded by the South Dakota Department of Education through SDADE, which is South Dakota's Federally Funded Star Schools Project.

 Major focus of evaluation centers on use and application of the Digital Dakota Network.

Evaluation Methods

   32 high schools and elementary schools visited statewide by evaluator pairs.

Data collection commenced in February and ended in May 2002, with site visit summaries constructed August through December 2002.

Qualitative methods, including interview, focus groups, observation, and hardware audit were used.

Summary Statistics of Study Sites Community Size Student Population Certified Teachers Student to computer ratio Available seating for DDN

Average

7,278

Range

339-59,607

State Ave.

1,833 896 64 2.41:1 20 131-4,000 12-249 .90-4.79:1 8-75 713.7

51.6

3:1 NA

DDN Instructional Uses

     Collaboration activities (research data comparisons, foreign language exchanges) Enrichment activities (Dr. Seuss Day, Mystery Reader, performances, Buffalo Round-up) Expert access (Smithsonian, EROS) High School course delivery College course delivery

DDN Logistical Uses

 Teacher professional development activities  Meetings between Educational service providers (LOFTI)  Regional consortia meetings between administrators or teachers  Community “Cracker Barrel” sessions  Athletic seeding and bracketing  Community organization use (mental health services)

Visual map of DDN use on instructional and logistical dimensions

Over-arching Evaluation Findings     South Dakota schools are filled with educational technologies and folks who can robustly use them The DDN system is an underutilized resource in most schools Conditions for optimal I-TV are consistent from site to site Barriers to use are generally local and not system-related

Barriers to be Addressed

       Lack of access Inconsistent bell schedule Attitude the interactive video is just for small schools Lack of incentives for teacher/instructor Difficulty of distance delivery courses Suspicion that new state administration will not support the DDN Belief that system replaces teaching peers and postions

Conditions for Exceptional Use

 Exceptional planning  Remote site supervision  Teacher incentives  Motivated students  Principals and superintendents with foresight  Activating and leveraging local partnerships

State Delivery Model

Regional Delivery Model

Regional or State Delivery: Criteria for Analysis

        Bell and school schedule compatibility Flexibility of interaction technologies used Duration of consortia affiliations Activity level of current consortia Access to remote instructional sites Teacher availability Demographics Distribution of operation costs (technical support, bridges, etc.)

Criteria . . .

    Nature and prevalence of professional peer group collaborations (professional development, special education, conferences, etc.) Coordination of extra-curricular activities (athletics, administrative, etc.) Availability of I-TV operation expertise Participation and support of community partners—or capacity to attract the support

Effects of Regional Delivery Model

 Community viability  Remote site contacts  Distributed expertise  Re-energized cooperatives  Tailored I-TV curriculum  Better interactions with remote supervisors

Role for a Statewide Course Vendor

 Course coordination and brokerage  Delivery of specialized courses

E. Rogers Stages of Innovation Adoption Model Adoption Trial Evaluation Interest Awareness

Report Prepared by Joanne Ustad in completion of the M.S. in Educational Technology, Dakota State University

Elementary Stages of Adoption

Trial 12% Adoption 0% Aw areness 24% Evaluation 32% Interest 32%

Middle School Stages of Adoption

Evaluation 10% Trial 5% Adoption 0% Interest 24% Aw areness 61%

High School Stages of Adoption

Trial 7% Adoption 5% Aw areness 31% Evaluation 26% Interest 31%

Limitations of the Study

   The self-selection process may bias and results toward the more progressive districts.

Schools for site visits were selected from among volunteers. The lack of randomization may skew toward technology-users.

Site selection could tend to result in the perception that schools are further along in the Interactive Video (DDN) adoption process than they really are.

Applying the Lens of Evaluation

 Delivery models  System utility – Dominated by course delivery – Applies traditional pedagogies – Abandons constructivist approaches  Contextual factors  Staying true to the notion of meaningful learning (theory-driven)?

More Information at:

www.homepages.dsu.edu/hawkesm Email: [email protected]