Transcript How to Read a Research Paper

How to Read
a
Research Paper
Tamer Elsayed
Faculty of Engineering
Computer Science and Engineering Dept.
May 16th 2013
A Typical Researcher …
 Will likely spend
hundreds of hours every year reading
papers 
 A “good” graduate student
/researcher should read
(in average) a paper a day!
References
 S. Keshav,
How to Read a Paper, ACM SIGCOMM
Computer Communication Review, 2007
 Philip W. L. Fong, Reading a Computer Science Research
Paper, SIGCSE 2009
 Amanda Stent, How to Read a Computer Science
Research Paper, Technical Report.
 Mihai Pop, How to Read a Scientific
Paper, a
Presentation at University of Maryland, College Park.
Why
reading research papers?
Why?
 You
were asked to 
 For a literature survey of a new
field/problem
 Be up-to-date on current research in the field
 Allows you to replicate/extend the results
 Provides you with useful data
 Gives you “pre-digested” thoughts
 To decide whether (and where) to publish
 Teaches you how to write
 Review for a conference or a class
 ….
Roadmap
√ Why reading research papers?

Types of (computing) research papers

Three-pass approach for reading a paper

Doing a literature review

How can I remember the papers I have read?
Types of (Computing)
Research Papers
From venue, peer-review, and content perspectives …
Venue?
Conference
Papers
Most recent, “hot off the presses”, information
Technical
Reports
Expands on the information in a conference paper.
Journal Papers
(or Articles)
Expand and combine results from several
conference papers
Book
Chapters
Expand a conference paper or journal article
PhD
Dissertations
Frequently revised into either journal papers or
book chapters
Posters
Extended abstracts, short, good for discussions
Workshop
Papers
Very hot/new topics, work in-progress, ideas,
preliminary results
Peer-Review?

Conference papers and journal papers are “peer-reviewed”

have been examined by other computer scientists (3-5) prior to
publication.

“Double-blind” review

Technical reports are typically not peer-reviewed, but are still
excellent sources of detailed information .

Posters and workshop papers are peer-reviewed (less number
of reviewers).

MSc thesis or PhD dissertation?
Content?
Theoretical
• Describe/prove a theory
• Describe new algorithms
Engineering
• Describe an implementation of an
algorithm, or part or all of a computer
system or application
Empirical
Survey
• Describe an experiment designed to
test some hypothesis
• Review current results in a field of
research
A Three-pass approach
for reading a research paper
Each pass accomplishes specific goals and builds upon the
previous pass …
Three-Pass Approach
The First Pass
 A quick scan to get a bird’s-eye view of the paper.
 Should take about 5-10 minutes.
 Decide
1.
2.
3.
4.
whether you need to do any more passes.
Carefully read the title, abstract, and introduction
Read the section and sub-section headings, but ignore
everything else
Read the conclusion
Glance over the references, mentally ticking off the
ones you’ve already read
1
At the End of this Pass …
You should be able to answer the five Cs:
1. Category: What type of paper is this? A measurement
paper? An analysis of an existing system? A description
of a research prototype?
2. Context: What is the problem (space)? Which other
papers is it related to?
3. Correctness: Do the assumptions appear to be valid?
4. Contributions: What are the main contributions?
5. Clarity: Is the paper well written?
More passes?
1
The Second Pass
 Should take up to 1 hour.
 Read with greater care, but ignore
details such as proofs.
 Identify key points, or make comments in the margins.
 Look carefully at the figures, diagrams and other
illustrations in the paper.
 Remember to mark relevant unread references for
further reading

this is a good way to learn more about the background of the
paper.
1 2
After this Pass …
 Sometimes, you won’t understand it even at the end
of
the second pass 




subject is new to you, with unfamiliar terminology and acronyms.
proof or experimental technique that you don’t understand.
poorly written: unsubstantiated assertions and numerous forward
references.
it could just be that it’s late at night and you’re tired!
 You
a)
b)
c)
can now choose to:
set the paper aside, hoping you don’t need to understand the
material to be successful in your career,
return to the paper later, perhaps after reading background
material or
persevere and go on to the third pass.
1 2
The Third Pass
 To
fully understand the paper (e.g., reviewing)
 Can take about 4-5 hours for beginners and about 1
hour for an experienced reader.
 Requires great attention to detail.
 Attempt to virtually re-implement the paper: making
the same assumptions as the authors, re-create the
work.
 You should identify and challenge every assumption in
every statement.
 During this pass, you should also jot down ideas for
future work.
1 2 3
At the End of this Pass …
 Should be able to reconstruct the entire structure of the
paper from memory.
 Should be able to identify its strong and weak points.
 Should be able to pinpoint implicit assumptions,
missing citations to relevant work, and potential issues
with experimental or analytical techniques.
1 2 3
Doing a Literature Survey
Doing a Literature Survey
 What is it?
 Requires you to read tens of papers,
unfamiliar field.
 What papers should you read?
 3-pass approach to help.
perhaps in an
First Step
 Use an academic search engine
and some well-chosen
keywords to find 3 to 5 recent papers in the area.

(e.g., Google Scholar, MS Academic Search, CiteSeer, or
ACM digital library)
 Do one pass
on each paper to get a sense of the work,
then read their related work sections.
 You will find a summary of the recent work

and perhaps, if you are lucky, a recent survey paper.
 If you can find

such a survey, you are done.
Read the survey, congratulating yourself on your good luck.
1
Second Step
 Find shared citations and repeated authors in the
bibliography.

These are the key papers and researchers in that area.
 Download the
key papers and set them aside.
 Then
go to the websites of the key researchers and see
where they’ve published recently.

That will help you identify the top conferences in that
field
1 2
Third Step
 Go to the website
for these top conferences and look
through their recent proceedings.
 A quick scan will usually identify recent high-quality
related work.
 These papers, along with the ones you set aside earlier,
constitute the first version of your survey.
 Make two passes through these papers.
 If they all cite a key paper that you did not find earlier,
obtain and read it, iterating as necessary.
1 2 3
How Can I Remember the
Papers I Have Read?
Remembering Read-Papers
 Strongly recommend that you make an electronic file
for your own bibliography.
 A BibTeX file is a good idea.
 After reading, if you think it’s worth remembering, write
an entry for that paper in your bibliography file.
 You should note: authors’ names, paper title, venue,
date of publication, and page numbers (if possible).
 Add a 2-3 sentence description of the paper in which
you summarize for yourself the problem addressed by
the paper, the solution proposed or results learned, and
main contributions.
Other Resources
 Reading scientific papers (at Purdue)
http://www.lib.purdue.edu/phys/inst/scipaper.html
 BibTeX
http://www.bibtex.org/
 Mendeley
http://www.mendeley.com/
 Zotero – firefox extension reference manager
http://www.zotero.org/
 Comparison of reference manager software tools
available
http://en.wikipedia.org/wiki/Comparison_of_reference_
management_software
This talk is
one of a series …
How to Do Research









Research methodology
How to read a research paper
How to review a paper
How to write a research paper
How to write a lit review
How to write theses/dissertations
How to be a good graduate student
How to give a talk
How to present a poster
Workshop next Semester …
Stay Tuned 
Acknowledgement
To my colleagues :
 Maram Hasanain
 Latifa
AlMarri
 Rana Malhas