Orlando Leon

, Enterprise Architect Education, ITS

Beth Berrean

, Communications/Website Services Manager, ISU

Opinder Bawa

, Chief Technology Officer, ITS

Recommendations for a Campus Survey Tool

Overview

• • • •

Charge Analysis Current Progress Next Steps

Our Charge

• • •

Should we purchase and support Qualtrics as an enterprise solution for the Campus?

Should UCSF decide on a single solution for all survey needs?

Should UCSF rationalize availability and support of existing survey solutions currently within the Campus?

Analysis: Sample Use Cases Encountered

• •

Education

– – – – Student Learner Survey Evaluation Feedback Registration

Research

– – – – – Radiology Orthopedics Study mOst Survey Tool ATHENA Clinical Trial Qualification Surveys Research Questionnaires

Analysis: Additional Use Cases Encountered

• •

Administrative

– – – – Polls Voting Usability Satisfaction/Feedback Surveys Interview Candidate Feedback

Clinical

– – Nursing PI Surveys Patient Satisfaction

Analysis: Existing Survey Tools

Currently supported by Primary Use Costs

Checkbox RedCap Salesforce SurveyMonkey Zoomerang SOM ISU, SOM EdTech Cancer Center & ITS ARS SOM ISU Ad-hoc Nursing Surveys for education and some one-off surveys for SOM TCO $7600/year for unlimited users Clinical research or by individual PI’s who need to store ePHI Unknown, but it is an open-source product Project-based Athena, Nursing Performance Improvement Group Simple surveys Alumni outreach $780/user/year for highest level of support $599/user/year

Findings on Tools: Qualtrics

• • • • • • • • • • •

Able to collect ePHI SSO integration with MyAccess Mobile versions of surveys “out of the box” Integration with Salesforce objects Supports 48 languages ADA compliance-checking tool Panels Triggers Robust user/group/organization/library features Programmatic interface to administration tools Training and Support

Considerations

•

In conversations with stakeholders (likely survey users —SON, SOM MedEd, Nursing PI, SOM TAC), people asked us to consider:

– – – – – – Product Features External/Vendor Support Internal Product Ownership Integration with Existing Systems Total Cost of Ownership Expertise in Survey Design vs Knowledge of the Technology

Recommendations

• • • •

Continue support for RedCap research, Salesforce for research and clinical needs Replace Checkbox with Campus license for Qualtrics Create a strategic marketing plan to help move all qualified users into Qualtrics Create product oversight group for each part of the Campus that intends to use the tool (Schools, Centers, Departments, Administrative) as first line of support for tool

Cost Projection

• •

2011-2012 – est $48,000

– – – – $20,000 - Annual license cost $20,000 - 0.25 FTE @ $80k/year for brand administration, organization configuration and custom template creation $5,000 – 0.05 FTE @ $100k/year for MyAccess integration $6,000 – 0.10 FTE @ $60k/year for marketing and communications

2012-2013+ est $24,000*

– – $20,000 - Annual license cost $4,000 - 0.05 FTE @ $80k/year for brand administration

* The assumption is that each functional unit using Qualtrics will administer their own users and organization configuration, while the overall brand administrator administers the overall Campus/global configuration and setup of new functional units

Scope of License

The term of the license for the Qualtrics Software is one year, beginning January 16, 2012 and includes an unlimited number of surveys.

The Qualtrics license may be used by academic areas within University of California San Francisco for projects that are academic in nature. Academic projects may serve research (instructional or academic) and administrative purposes.

The license may not be used for purely corporate or "for-profit" purposes. Such corporate projects would include purely clinical and patient related surveys that are not related to academic research. Affiliated hospital and clinics may purchase Qualtrics access according to Qualtrics corporate pricing model. A 10% discount will be applied to any license purchased for corporate purposes.

11

Current Progress

• • • •

BAA reviewed and signed License negotiated, reviewed, and signed

– – License terms Features

Advisory and Oversight Committee

– SON, SOM/UME/GME, SOD, SOP, Graduate Division, Academic Senate, Memory and Aging, Center for Health Professions, CTSI, Library, SAA, ITS

Configuration and Deployment

– MyAccess/SSO, Wiki/FAQ, Branding, Marketing/Communications, Administration 12

Next Steps

• • • •

Review and Consultation with University Relations and UCSF Legal Finalization of division and general use policies Marketing and Communication to UCSF community Group Training Sessions

13

Findings on Tools: CheckBox (v4)

• • • •

User interface: OK No end user (survey creator) training program As currently implemented at UCSF,

– – – Not able to collect ePHI No support for multi-lingual surveys Allows importing of data (Medical Students since 2004) – Integrated with AD (cannot integrate with MyAccess)

Key Feature: Actively Used

– The SOM MedEd group uses this tool widely and SOM ISU also uses it. It is stable, and there is a upgrade path.

Findings on Tools: RedCap

• • • • •

Able to collect ePHI User interface: complex ARS offers classes As currently implemented at UCSF,

– – – – No support for multi-lingual surveys No native accessibility features Not integrated with MyAccess Mobile compatible?

Key Feature: Validated Instruments

– Data collection instruments and forms reviewed for research relevance, accuracy in coding and function by the RedCap Library Oversight Committee.

Findings on Tools: Salesforce Health Quality Surveys (HQS)

• • • •

Able to collect ePHI Integrated into mobile application development platform

•

Open-source survey definition language supported by Harvard, MIT, and the Boston Children's Hospital Key Feature: Integration with Salesforce

– Integrated with multiple systems (including multiple EHR systems) and the UCSF IDR

“Anything is possible.”

Findings on Tools: Others

•

SurveyMonkey

– – ADA compliant Free

Zoomerang

• •

SurveyGizmo

– – – Mobile versions built into platform Professional services available Integration with Salesforce

Next Steps

• •

Endorse recommendation Decide where within the enterprise the product should be managed

• • •

Negotiate a BAA and license agreements with Qualtrics Create a Product Oversight Committee Develop a marketing/communications plan for helping internal constituents identify survey needs and tools

19