Transcript Title
Going Against Goliath 23rd May 2010 Katrina Kurtz, MLIS Carrie Iwema, PhD, MLS Ansuman Chattopadhyay, PhD Health Sciences Library System University of Pittsburgh http://www.hsls.pitt.edu/guides/genetics Photo credit: http://www.uncabaret.com/node?page=5 Bioinformatics Tools Answer biological questions Made by informatics researchers Freely available to the public Published in scholarly journals http://www.hsls.pitt.edu/guides/genetics Bioinformatics Tools Examples: Protein structure prediction Sequence alignment Databases of genes associated with disease Protein interactions http://www.hsls.pitt.edu/guides/genetics Who Uses Bioinformatics Tools? Biologists: Faculty researchers Graduate students Undergraduate students Doctors who do biomedical research Biomedical informatics researchers http://www.hsls.pitt.edu/guides/genetics Online Bioinformatics Resources Collection Manually curated Searchable collection of bioinformatics tools Freely available to the public 2696 resources http://www.hsls.pitt.edu/guides/genetics/obrc/ http://www.hsls.pitt.edu/guides/genetics http://www.hsls.pitt.edu/guides/genetics http://www.hsls.pitt.edu/guides/genetics http://www.hsls.pitt.edu/guides/genetics Common Criticisms… Why bother? Everyone uses Google. Google is taking over. http://www.hsls.pitt.edu/guides/genetics Is Google Actually Better? Large amount of data Popularity ranking Biologists are not librarians http://www.hsls.pitt.edu/guides/genetics Research Project Hypothesis OBRC is better than Google at Locating bioinformatics tools Quickly Easily Successfully Leaving users happier with: Results Overall experience http://www.hsls.pitt.edu/guides/genetics Research Project 10 biology graduate students Three tasks on both OBRC and Google Recorded search as they narrated Search experience survey http://www.hsls.pitt.edu/guides/genetics Research Project Measured: Time spent on each search Number of search queries Number of sites visited Success rate Users’ ratings http://www.hsls.pitt.edu/guides/genetics Task 1: Protein Interaction You are a pharmacologist interested in drugs that target proteins. Where would you find reliable information about how drugs interact with proteins, specifically information about the strength of the binding between the protein and ligand? http://www.hsls.pitt.edu/guides/genetics Task 2: Structure Prediction Find a tool that performs protein structure prediction. Determine from the search engine how often it has been cited. http://www.hsls.pitt.edu/guides/genetics Task 3: Ovarian Cancer Where would you look for information about all the published gene mutations implicated in ovarian cancer, in one place? http://www.hsls.pitt.edu/guides/genetics Is it faster? Average Task Completion Time 350 300 Time (s) 250 200 Google OBRC 150 100 50 0 Protein Task 1 Interaction Structure Task 2 Prediction Ovarian Task 3 Cancer http://www.hsls.pitt.edu/guides/genetics Average Is it easier to find results? Average Number of Search Queries 4 Number of Queries 3 Google OBRC 2 1 0 Protein Task 1 Interaction Structure Task 2 Prediction Ovarian Task 3 Cancer http://www.hsls.pitt.edu/guides/genetics Average Is it easier to find results? Number of Sites Visited Before Success Number of Sites Visited 4 3 Google OBRC 2 1 0 Protein Task 1 Interaction Structure Task 2 Prediction Ovarian Task 3 Cancer http://www.hsls.pitt.edu/guides/genetics Average Is it successful? Rate of Success Percentage of Successful Searches 100% 80% 60% Google OBRC 40% 20% 0% Task 1 Protein Interaction Task 2 Structure Prediction Task 3 Ovarian Cancer http://www.hsls.pitt.edu/guides/genetics Average Do users like it? Survey Results 7 6 Rating 5 Google OBRC 4 3 2 1 Appropriateness of Results Ease of Use Overall Experience http://www.hsls.pitt.edu/guides/genetics What do they say about Google? “I was just frustrated [by Google].” “[Google is] time consuming unless we know what we are searching for.” “Google was difficult to navigate with all the information on cancer.” “I realized that Google is not an effective search tool.” http://www.hsls.pitt.edu/guides/genetics What do they say about OBRC? “Wow, this is really direct!” “OBRC is an excellent resource, especially if you don’t know exactly what you’re searching for.” “[OBRC] was much easier to locate relevant information.” “Gives good results, but I don’t like the interface.” http://www.hsls.pitt.edu/guides/genetics What did we learn about OBRC? Time lag Scopus and Google Scholar Spell check Long search queries Unaware it exists http://www.hsls.pitt.edu/guides/genetics Limitations Small sample size Potential bias from tasks Previous experience with Google http://www.hsls.pitt.edu/guides/genetics Future Work Larger sample size Other user groups User-chosen task http://www.hsls.pitt.edu/guides/genetics Conclusions… Searches using OBRC are: Faster Easier More successful Users are more satisfied. Users plan to use OBRC in the future. http://www.hsls.pitt.edu/guides/genetics Conclusions… …libraries can and should compete! http://www.hsls.pitt.edu/guides/genetics This study is IRB approved. Funding provided by: NLM Grant 3 T15 LM007059-23S1 Thanks to my collaborators, Dr. Carrie Iwema and Dr. Ansuman Chattopadhyay Special thanks to Nancy Tannery and Dr. Rebecca Crowley http://www.hsls.pitt.edu/guides/genetics Thank you! Any questions? Katrina Kurtz [email protected] 412-624-3508 Carrie Iwema [email protected] 412-648-6887 Ansuman Chattopadhyay [email protected] 412-648-1297 http://www.hsls.pitt.edu/guides/genetics