Promoting the RCR Video Case Studies: Resources and Evaluation

Paul Braunschweiger, Ph.D.

Professor, Radiation Oncology CITI Co-Founder Council of Graduate Schools – Graduate Education in a Global Context , 12-3-08

Disclaimer and Disclosures

• Dan Vasgird Ph. D. UN-L.

– CITI Developer Group – RCR working group. – 2006 CGS Grant to develop video vignettes (trigger cases) for RCR education.

– 2007 Cases added to CITI RCR Program. • Disclaimer and Disclosure.

Talking Points

• RCR education paradigms.

• University of Nebraska – RCR videos • Application of RCR videos in the CITI Program.

• Distribution of the RCR videos.

– CITI Program.

– Global Context.

• Evaluation of the RCR video resources.

– Learner Feedback • Resources

Integrity in The Research Enterprise

• • • Starts with Institutional Commitment Promoted and nurtured by the IRB, CoIC, IBC, IACUC, Sponsored Programs, Research Administrators. Embraced by the investigators, staff and students as just:

“The right thing to do”

"Integrity" by Canneto, Columbus, OH

Why Does Integrity and Professionalism Matter?

• Risks to subjects, Risks to society.

• Federal inquiry / intervention/ audit / suspensions.

• Wasted resources $$$$ • Institutional embarrassment • Personal embarrassment • Loss of funding, Loss of livelihood • Legal actions, $$$$, Jail • Undermines the Public Trust – Research becomes politicized.

• CoI and Pharmaceutical Industry – Justification and / or vindication for groups with anti-research agenda. (PETA, ALF)

The Foundations for Trust are Based in the Documentation of Ethical Behavior and Education .

Mandated Ethics Training

• • • • • • Human Subjects Protections – GCP – IRB Professional .

– Basic for investigators, staff and students.

Lab Animal Welfare – Basics for Investigators, staff and students – IACUC Member – Model / species specific / procedure specific HIPPA (Privacy and Security) – Investigators – Administrators Biological Safety and Security – Basics for Investigators, staff and students The Responsible Conduct of Research – Basics for investigators, staff and students – Administrators.

Grants and Contracts, Sponsored Programs – Grants management – Financial responsibility – CoI

Institutional Approaches Research Ethics Education

• Bottom up Approach – Research Ethics is integrated in the academic and professional develop of students and fellows by the research faculty. • Top down Federal /institutional mandate.

– Animal Welfare – Human Subjects Protection – HIPAA

Delivering the Message

• • • • • • 1:1 training – Effective, labor and time intensive, costly Small group training – Effective, labor intensive, less time intensive, costly, – Subcontracted Large group training, – less time intensive, Less effective, very costly, may not be convenient.

External consultants – Can be effective, very costly.

Web based training.

– Convenient, effective, inexpensive, timely, assessment. – Provides knowledge base for advanced training.

– Lacks direct interaction with instructor, most programs are very textual, problems with presentation, formatting, ISPs and the internet. ~15% would prefer something else.

Combined programmatic approach

Web based training

Assuming a similar time commitment, would you prefer another vehicle for this mandated instruction?

90% 80% 70% 60% 50% 40% 30% 20% 10% 0% web Vid class Alternatives book audio 80% 70% 60% 50% 40% 30% 20% 10% 0% Web Vid Class Alternatives book audio 80% 70% 60% 50% 40% 30% 20% 10% 0% Web Class Vid Alternatives book audio HSR Lab Animal Welfare RCR

CITI Learner Satisfaction Survey results 1-1-08

Video Presentation Paradigms

• Virtual Video Libraries – Reference Resource – UNL, SU, ORI, AAAS etc.

• Optional exercises in a web based tutorial.

– NIU • Query-Video Presentation-Query (QVQ) – UAB • 2-3 minute video clips of exemplars providing expert opinions on various aspects of the RCR.

– Columbia University • Cases and videos embedded in the RCR foundation text.

– CITI Program

University of Nebraska RCR video vignettes

• CGS Funded Project – Grad. Faculty and Students – 3 topics of greatest interest.

– Student actors – Primary target at UN-L about 100 grad students/year.

– Live and online evaluations – http://www.unl.edu/gradstudies/facstaff/rcr/

http://www.unl.edu/gradstudies/facstaff/rcr/

Integration of the UN-L RCR Vids into the CITI RCR Courses.

CITI Program RCR Courses

Biomedical Sciences Social & Behavioral Sciences Physical Sciences Arts & Humanities Administrators Supported by a resource development contract from DHHS / ORI

The Responsible Conduct of Research

• Research Misconduct – Historical perspectives – Whistle Blowing • Data Acquisition and Management • Responsible Authorship • Responsible Peer Review • Mentoring • Conflicts of Interest • Responsible Collaborative Research • Human Subjects Protection • Lab Animal Welfare

RCR Program Paradigm

• Public Access, subscriber access • Customized courses.

• Text, cases, videos and quizzes • Helpdesk ( 305 243-7970 ) for students and trainers • Completion Certificates • Institution specific training materials easily incorporated to the curriculum. • Training data storage and retrieval system • Voluntary Learner Satisfaction Survey

•

Research Misconduct

– Discipline Specific Title – Links – Introductory Video Scenarios – Exemplar video presentations – Imbedded Case Studies – Quiz

Who should take advantage of RCR education?

I believe that it is very important that the entire research team and not just the PI, should have a clear understanding of the Responsible Conduct of Research / human subjects protections.

60% 60% RCR Learner Satisfaction Survey 50% 40% 30% 20%

7% 84%

10% 0% 1 2 3 4 5 6 7 8 9

Strongly disagree Strongly agree

50% 40% 30% 20% 10% HSR Learner Satisfaction Survey

11% 78%

0% 1 2 3 4 5 6 7 8 9

Strongly disagree Strongly agree

CITI Learner Satisfaction Survey results 1-1-08

What is The CITI Program

www.citiprogram.org

• 8 web based courses: – Human subjects Protection • Basic courses – Biomedical • Basic course – SBR • Refresher Courses - Biomedical and SBR • Good Clinical Practice – Lab Animal Welfare – Health Information Privacy and Security – The RCR • 5 discipline specific course.

• RCR for engineers.

• RCR for the humanities.

CITI Support Staff

CITI Program Milestones

• 10-2000 10 Participating Institutions.

• 2006 New SBR Course added.

• • • • 2008 ~950 Participating Institutions ~22,000 Learners per month. 40-60% students >337,000 Learners since 1-1-2008.

>960,000 Learners have completed a course since 9-1-2000.

• • Global Context International Initiatives Multi-language Course Site. 11-2007 – Spanish – Portuguese – Chinese, Japanese, Russian,

Mongolian, French, Thai

CITI Worldwide Footprint

World wide n = 950 International n = 47sites / 35countries

Pan American Bioethics Initiative

CITI and U of M Ethics Programs • Fogarty International / NIH Funded R25 grant – P. Braunschweiger, K. Goodman and S. Litewka.

• Research Ethics Education and Training Program development in Latin America, Brazil, and the Caribbean. • Decision markers, policy architects in government, academia and industry.

• Charge to the Fellows: – Develop robust research ethics education program for their organization, implement, promote. – PABI Workshops, ethics committee capacity building, CITI Platform.

– Fellows recruit and mentor peers.

RCR Program Evaluation

From: The School of Athens, Raphaelo. 1509

User Satisfaction Survey Program Evaluation

• Voluntary, Anonymous. IRB Approved.

• Topics addressed – Demographic Information.

• Role, age, gender, country, language – Usability • Registration, navigation, quizzes, Internet, Presentation, Time – Knowledge base.

– Confidence in the new knowledge base – Willingness to apply the new knowledge – Intent to promote the RCR – Overall rating – Other

Evaluation • RCR Course Usage • Case study usage • Video vignette usage • Learner feedback

RCR Course Usage Certificates Awarded

Year 2008* 2007 Public Access 2,290 Participating Institutions 14,726 Total 17,016 2,600 3,669 6,269 Total 4,890 18,395 23,285 *December 1, 2008

70% 60% 50% 40% 30% 20% 10% 0%

How Many Introductory Videos Did You Review

0 At least 1 2 or 3 3

The 3 minute video case studies were:

80% 70% 60% 50% 40% 30% 20% 10% 0% too short too long about right

The Introductory video case studies were helpful to frame the content before I started reading. 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% No Yes

After completing the CITI RCR Course, I have a better understanding of the ethical issues central to the research enterprise so that I can conduct my work with the highest level of integrity.

35% 30% 25% 20% 15% 10% 5% 0% 1 2 3 4 5 6 7 8 9 Strongly disagree Strongly agree

Now that I have completed the CITI RCR Course, I am more confident in my ability to advise a student or a colleague on an issue involving the Responsible Conduct of Research.

25% 12% 66% 20% 15% 10% 5% 0% 1 2 3 4 5 6 7 8 9 Strongly disagree Strongly agree

Having completed the CITI Course in the Responsible Conduct of Research and gained a better understanding of how damaging research misconduct can be to students, to the institutions and to other investigators, I am now more likely to report a clear act of research misconduct to an appropriate person at my organization.

30% 25% 20% 15% 10% 5% 0% 1 2 3 4 5 6 7 8 9 Strongly disagree Strongly agree

New CITI RCR Videos

• Peer Review http://medweb02.med.miami.edu/CITIvideo/2peer.wmv

• Mentoring http://medweb02.med.miami.edu/CITIvideo/3mentor.wmv

• Data Management http://medweb02.med.miami.edu/CITIvideo/4data.wmv

• Collaborative Research http://medweb02.med.miami.edu/CITIvideo/1collab.wmv

Summary

• The foundation of research integrity is research ethics education.

– Early, often, multiple formats • Video – Evaluate your education programs. Are they working? How can you improve them?

– Listen to the feedback from your learners.

“New Integrity” by Artibella Avanti

Summary

• Web based educational approaches to ethics education: – Can increase the ethics knowledge base of a wide audience.

– Can prepare the learner for advanced instruction and debate.

– Can provide the opportunity to customize the message to multiple audiences.

– Should be a tool in a larger institutional RCR Program.

– Web based education is not for everyone. Alternatives.

• The Responsible Conduct of Research is beyond simply being compliant with Federal regulations and Institutional Policies; It is just the “right thing to do” .

RCR Video Resources

RCR Video Resources

ORI / DHHS www.ori/dhhs.gov

• • • •

General

–

17 Video Vignettes on Research Ethics and Academic Integrity. Syracuse University http://gradschpdprograms.syr.edu/resources/videos.php

Data acquisition and management – NIU, http://www.niu.edu/rcrportal/datamanagement/dmmain.html

– CITI, www.citiprogram.org

– AAAS http://www.aaas.org/spp/video/thevideos.htm

Conflicts of Interest – *Cleveland State University (flash) – UN-L http://www.unl.edu/gradstudies/facstaff/rcr/ – AAAS http://www.aaas.org/spp/video/thevideos.htm

Research Misconduct, Whistle Blowing – UN-L http://www.unl.edu/gradstudies/facstaff/rcr/ – UAB, http://www.uab.edu/researchintegrityandimages/default.html

– CITI www.citiprogram.org

– AAAS http://www.aaas.org/spp/video/thevideos.htm

RCR Video Resources

ORI / DHHS www.ori/dhhs.gov

• • • • Mentoring – UAB, http://www.uab.edu/researchintegrityandimages/default.html

– CITI, www.citiprogram.org

– NIU Flash simulation http://www.niu.edu/rcrportal/mentoring/overview/overview.html

Collaborative Research – CITI www.citiprogram.org

– NIU (Flash) http://www.niu.edu/rcrportal/cfront.html

Authorship – UAB, http://www.uab.edu/researchintegrityandimages/default.html

– UNL http://www.unl.edu/gradstudies/facstaff/rcr/ – CITI www.citiprogram.org

– AAAS http://www.aaas.org/spp/video/thevideos.htm

Peer Review – CITI www.citiprogram.org

RCR Video Resources

ORI / DHHS www.ori/dhhs.gov

• • Lab Animal welfare – AALAS,

*Wake Forest University, IACUC Inspection Virtual Walkthrough

–

OLAW. http://grants.nih.gov/grants/olaw/TrainingVideos.htm

– Nonhuman Primates, Laboratory Dog, Survival Rodent Surgery, Basic Biomethodology for Laboratory Mice.

Human Subjects – HSRA -

Protecting Human Subjects Training.

–

Children’s Hospital Boston, video

–

Case studies http://ori.hhs.gov/education/products/ucla/default.htm

• Other – Journal editor’s tool http://www.uab.edu/researchintegrityandimages/default.html

– The role of the RIO , Michigan State University http://ori.hhs.gov/rio/riovideo/ – RCREC http://rcrec.org/

RCR Web Resources Tutorials and courses

Human subjects research tutorials

• PRIM&R - IRB 101/ 201 CD • Dunn and Chadwick Book • NIH/NCI – new version • CITI Program – Biomedical – Social and Behavioral

Good Clinical Practice

– ACRP – SOCRA – GeneEd $$$/ learner – Infonetica GCP http://www.gcptraining.org.uk/ $$$$/learner • Multiple languages – Kaplan $$$$/learner – ClinfoSource https://www.clinfosource.com/index.asp

learner $$$$$$ / –

CITI $ /institution or organization

Lab Animal Welfare

–

AALAS, LAWTE

•

Basic courses investigators, staff, students

• •

IACUC members Species / model / procedure specific modules

– –

OLAW , Webinars

• • •

IACUC Members IACUC Coordinators IO CITI

•

Basic courses investigators, staff, students

• •

IACUC members Species / model / procedure specific modules

Health Insurance Portability and Accountability Act (HIPAA) • Investigator , Administrator – http://myhipaatraining.com/ – –

www.HRClassRoom.com/HIPAA

http://www.training-hipaa.net/

– Health Information Privacy and Security (HIPS) • CITI Program, www.citiprogram.org

The Responsible Conduct of Research

• Biological Safety – Basics for Investigators, staff and students • UN Medical Center http://www.unmc.edu/dept/ibc/index.cfm?L1_ID=7&CONREF=11 • The Responsible Conduct of Research – Basics for investigators, staff and students. (ORI funded) • UC Davis http://ethics.ucsd.edu/courses/integrity/index.html

• Columbia University http://ccnmtl.columbia.edu/projects/rcr/ • NIU http://www.niu.edu/rcrportal/ • University of Montana http://www.umt.edu/research/ethics/research_ethics.html

• UNH http://www.unh.edu/rcr/index.html

• Boston College http://www.bc.edu/research/rcradmin/ • ORI website http://ori.hhs.gov/ • CITI RCR Program www.citiprogram.org

Research Administration

Professional Societies • – – – – – NCURA • Sponsored Programs Grants management Financial Administration Contracts Management Clinical trials management – – – – – – SRAI Management & Operations Research Integrity Financial Management Research Law Sponsored Prog. Administration Sponsors & Agencies Boston College / ORI – Administrators and the RCR www.CITIProgram.org

CITI Program Resources.

Responsible Conduct of Research.

Public Access at www.citiprogram.org

I plan to take a more active role in

promoting ethical research practices

by : • • I intend to do little more than I am currently doing.

Attending more ethics related seminars • • Becoming a mentor to a student or young faculty member Encouraging my institution to adopt a formal RCR education requirement for students and faculty • Taking an advanced research ethics course • Adding a research ethics module to a course I teach 33% 32% 27% 16% 13% 11% n=3896

RCR for Engineers • Coming Winter 2009

Adapted for CITI from materials developed by Steve Erickson Ph.D. and Karen M.T. Muskavitch, Ph.D “Administrators and the Responsible Conduct of Research” funded by ORI/DHHS http://www.bc.edu/research/rcradmin/default.html

CITI RCR Course Design

•

Publication and Authorship

– Discipline Specific Title and hyperlinks – Introductory Video Scenarios – Imbedded Case Studies – Quiz

CITI – Program

11-2008

Participating Institutions and Organizations (~950) CITI Developer Group (~60)

RCR L.A.W.

Founders

CITI Executive Advisory Committee CITI Editorial Board (15)

Intl.

HSRP GCP HIPS www.citiprogram.org

CITI Administration – Office of Research Education, University of Miami