Transcript Slide 1

Research Methods:
Introduction
James Gain
[email protected]
What is Computer Science?
Origins
mathematics,
engineering, and
commercial practice.
Evolved into
theoretical,
experimental and
design (or user) orientated aspects.
balance and synthesize these aspects
Research Methods
Research Tools
Theory
Abstraction (= experimentation)
Design
Theory
(Maths)
CS
Experimentation
Research Methods
Design
What is Research?
New Stuff?
What sort of stuff?
Only for academics?
What is good research?
Who makes a good researcher?
Research Methods
Introduction to CS Research
A research lifecycle
Flavours of research
Ethics in research
Pitfalls
Research Methods
Getting started
Research proposals
Literature reviews
Presentations
MSc & PhD work
The Research Lifecycle
Problem
Identified
Research
Activities
Solution
Adopted
Research Activities:
Literature Search (survey previous work)
Do the Work (elaborate ideas and get results)
Write Up (plan and write a draft)
3! = 6 orderings
Research Methods
Is Research a Linear
Progression?
Progress?
Circular?
Evolutionary?
Research Methods
Orders: The Classics
LIT  WORK  WRITE
Don’t know when to stop the literature
search
Can inhibit innovation
WORK  LIT  WRITE
Might get a nasty shock (someone else
beat you to it)
CONCLUSION: safe options
Research Methods
Orders: quick and dirty
WORK  WRITE  [LIT]
Gamble that the referees (examiners)
know less than you
WRITE  WORK  [LIT]
Suits speculative work
Sometimes used to drag the referees in
as unwitting collaborators
CONCLUSION: living on the edge
Research Methods
Orders: paperchasers
WRITE  LIT  WORK
Writing serves as a plan of work focused
to produce a single publication
LIT  WRITE  WORK
Good for an idea out of your normal line
of research
CONCLUSION: unconventional
Research Methods
Flavours of Research
Theoretical
Develop new theories
Engineering
Develop better mechanisms to improve current
practice
Experimental
Evaluate a theory/mechanism – usually via
implementation and testing
Can include human factors
Some research projects cover the spectrum
Research Methods
Research Outputs
Theoretical
Theorems
Models
Analyses of existing research
Engineering
System architectures & prototypes
Code libraries
Knowledge bases
Ontologies
Hardware specifications
Research Methods
Experimental Research Outputs
Human factors
Surveys (questionnaires, interviews)
Experimental results (lab tests, field studies, case
studies)
Measurements and opinions (quantitative and
qualitative results)
System
Benchmarks
Test programs & measurements
Comparative analyses
Research Methods
Ethics in Research
A research lifecycle
Flavours of research
Ethics in research
Pitfalls
Getting started
Research proposals
Literature reviews
Presentations
Resource: “On being a Scientist:
responsible conduct in research”
MSc & PhD work
www.nap.edu/readingroom/books/obas
Research Methods
Avoid conflicts of interest
Science relies on professional judgement
which can be compromised by:
Financial conflicts (e.g., undisclosed shares in a
company with interest in the outcome of
research)
Social and personal beliefs (e.g., Einstein’s “God
does not play dice”)
Pressures of competition (e.g., reviewing a paper
with similar work)
Solutions:
Disclosure
Self knowledge
Peer review
Research Methods
Give credit where its due
Authorship (and order of authors):
Conventions can vary considerably
Best to decide upfront
Avoid “honorary” authors – must make a direct
and substantial contribution
Establishes accountability as well as credit
Acknowledgements:
The place to give credit for less substantial
assistance
Citations:
Part of the reward system – connected to
funding and reputation
Research Methods
Case Study: Pulsars
Credit is a sensitive issue when researchers
are of different seniority:
In 1967 Jocelyn Bell, a 24-year-old graduate
student, discovered pulsars
Supervised by Anthony Hewish she was in charge
of operating and analyzing data from a 4.5 acre
radiotelescope
One day Bell noticed "a bit of scruff" on the data
chart
Together Bell and Hewish analyzed the signal
and found several similar examples elsewhere in
the sky
With 3 others they published a paper
announcing the discovery
Research Methods
Judgement on Pulsars?
Hewish got a Nobel Prize,
Bell did not
Against:
Bell’s recognition of the
signal was the crucial act
of discovery
For:
Bell didn’t deserve a Nobel
Prize for doing what is
expected of a graduate
student in a project
conceived and set up by
others
Research Methods
Shut Down the Paper Mill
The publish or perish paper mill:
Research careers seem to depend on quantity
of publications not quality
Consequences:
1. Haste and negligence
But progress relies on a trust in previous results
2. MPUs (minimum publishable units)
But this dilutes contribution and forces wading
through masses of literature
Research Methods
Case Study: nanotechnology
Jan Schön:
Worked in condensed matter physics and nanotechnology
Claimed he could replace silicone-based transistors with
organic dye molecules
In 2001, averaged 1 paper every 8 days
On track for a Nobel Prize
Found Out:
Results seemed suspiciously precise
A researcher spotted identical graphs in two separate
papers
Whole constructed data sets reused in different
experiments
Research Methods
Judgement on Nanotechnology
Outcome:
Schön was fired from his position at Bell
Labs after an internal investigation
Many of his papers were rescinded
He was banned from applying for funding
in Germany
Other Consequences:
For his co-authors?
For reviewers of his papers?
Research Methods
Photo Manipulation
Recently many journals (Cell Biology,
Science, Nature) have begun testing for
photo manipulation
The following manipulations are not allowed:
Splicing together different images to represent a
single experiment
Changing brightness and contrast of only a part
of the image
Any change that conceals information, even
when it is considered to be aspecific
Showing only a very small part of the photograph
so that additional information is not visible
Research Methods
Misconduct
Fabrication (making up results)
Falsification (modifying results)
Plagiarism (copying without credit)
Suppresion (not reporting negative results)
Other deviations from accepted research practice:
Covering up misconduct, misuse of research
funds, etc.
Consequences:
Harm to individuals, squandering of public funds,
attracts criticism of Science
But how can scientists expect to get away with it?
Research Methods
Case Study: Dealing with
misconduct
Francine is finishing her Ph.D. and Sylvia is a fellow
grad student. Both have the same supervisor.
Francine realizes there are problems with Sylvia’s
work, she:
Is rarely in the lab
Never shows anyone her code
Has performance results that seem too “clean” to be real
Also:
Francine needs a reference from her supervisor and Sylvia
is one of her favourites
Both Francine and her supervisor are using Sylvia’s results
for their own research
Research Methods
Judgement on Misconduct
Should Francine first try to talk with Sylvia,
with her supervisor, or with someone else
entirely?
Does she know enough to be able to raise
concerns?
Where else can Francine go for information
that could help her decide what to do?
Research Methods
Final Do’s and Don’t’s
Do:
Get ethics clearance if your research is
potentially hazardous to human subjects
respect:
• IP rights and confidentiality
• Patents
• The ACM code of ethics
Don’t:
Publish the same thing in more than one place
Inform the media of results before peer-reviewed
publication
Research Methods
Pitfalls
A research lifecycle
Flavours of research
Ethics in research
Pitfalls
Getting started
Research proposals
Literature reviews
Presentations
Source: “The Researcher’s Bible”
MSc & PhD work
Research Methods
Homepages.inf.ed.ac.uk/bundy/howtos/resbible.html
Solving the World
Easy to pick research goals that are
too ambitious
Especially in Artificial Intelligence
Instead:
Allow the main burden of scoping to fall
on your supervisor
Find out where the state of the art lies
Look to the future work section of papers
Can also redo bad work, properly
Research Methods
Manna from Heaven
Don’t expect inspiration to strike, staring at a
blank piece of paper
“Science is 99% perspiration and 1%
inspiration”
Instead:
Read the literature with a question in mind
Talk to people - your project partner and
supervisor - and explain your ideas
Tackle a simplified version of your problem
Write down your ideas in a working form
Research Methods
Boondoggling
The appearance of work without actual
productivity
Surprisingly seductive
Coding for its own sake
Writing Yet Another Programming Language
(YAPL)
Gathering unnecessary experimental data
Instead:
Make sure your programming and
experimentation contributes directly to the
research
Research Methods
Ivory Tower
Focus on your topic is good but don’t shut
out the rest of the world completely
Because it prevents cross-pollenisation of ideas
Instead:
Keep in touch with the state of the art in related
fields - attend colloquia and talk to other
students about their research
Set aside a part of the week for reading
abstracts and skimming papers
Research Methods
Misunderstood Genius
Easy to believe that no one understands
your ideas because you are a genius
More likely:
Love of jargon. CS is full of jargon. Try to rephrase
your ideas using ordinary English
If I can do it, it's trivial. Once you have seen the
solution to a problem it appears simple
Love of complexity. It’s not a virtue to make an
unnecessarily complicated program - it’s just a
nuisance to other people. Occam and Einstein
were right!
Research Methods
Starting Research
A research lifecycle
Flavours of research
Ethics in research
Pitfalls
Getting started
Research proposals
Literature reviews
Presentations
MSc & PhD work
Research Methods
Finding a Research Question
What problems to tackle?:
What matters to you
Anything messy or difficult
New technologies
New users
A paper you enjoyed
A paper you disliked
Example question: “How can the
new technology <T> be adapted to
run on Cell Phones/PDA’s?”
Research Methods
Research Fit
What research are you suited to?
what interests you?
what expertise exists around you?
What are your skills?
Don’t neglect methodology:
Look at similar research to decide what kind of
methodology is best for your research question
Make sure you are willing to apply that
methodology
Research Methods
It’s not easy so why do it ?
Satisfaction & thrill of being the first to
ever create/understand something
Famous Eureka moment
Privilege and recognition of adding to
human knowledge
Meet/work with passionate, deep
thinkers
Freedom – what to do, how, when,
with whom
Research Methods
Refining the Research Question
Choose an initial objective
Read the literature
Refine the objective:
Narrow it
Write it as a question
Describe it in a single sentence
Decide on the measure of success
Do a quick first prototype/experiment
Research Methods
Research Proposal: Preparation
Ask yourself:
Am I familiar with related research in this area
Do I have a good understanding of the steps
that will be involved in achieving these goals
Do I have the ability to successfully conduct
each of these steps
Am I sufficiently motivated and enthusiastic
about all the steps in this project
Am I convinced that the results of this research
will be useful to others
Research Methods
Research Proposal: Structure
Honours proposal structure:
Project Description
Related Work
Outcomes (system, questions tackled,
expected impact, key success factors)
Work Detail (timeline, resources required,
deliverables, milestones, work allocation)
References
More general proposals might include:
Budget, CV, Dissemination plan
Research Methods
Research Proposal: Evaluation
Have you answered these questions:
What you are planning to do ?
Why ?
What the difficulties are?
Is it feasible for you ?
Do you have a plan of how to do it ?
Have you done your homework ?
Research Methods
Literature Reviews
A research lifecycle
Flavours of research
Ethics in research
Pitfalls
Research Methods
Getting started
Research proposals
Literature reviews
Presentations
MSc & PhD work
What is a Literature Review?
As a process:
Reading, taking notes, organising, documenting
Start with general, broad, textbook works
Move toward specialised, recent papers
As a document:
Not just a string of article summaries
Rather a coherent discussion of previous related
work
Includes intro, conclusion, references
Don’t confuse the document and process
Research Methods
The Literature Review Process
Understand
the Field
Back Chain
to References
Research Methods
• Lecture Notes
• Text Books
Find a Survey
• Text Books
• Survey Papers
• Theses
Find Focused
Research
• Research Papers
Forward Chain
to Citations
Broad
Narrow
Resources
Experts: lecturers, supervisors, librarians
ACM Digital Library
http://portal.acm.org/dl.cfm
Includes most ACM pubs (but not IEEE)
UCT has a subscription
Google Scholar
http://scholar.google.com/
Good all-round resource
CiteSeer
http://citeseer.ist.psu.edu/
Digital library and search engine
Heavily linked meta-data allows chaining through citations
Research Methods
Critical Reading Required
Being able to read rapidly and critically is a vital skill
First Skim:
Abstract, section headings and figures
Then Dive into Detail:
May require reading references for a full understanding
Take Notes:
Complete citation
Main research question & conclusions
Research methodology
Key ideas or results relevant to your research
Future work
Gaps/problems
Research Methods
Presentations
A research lifecycle
Flavours of research
Ethics in research
Getting started
Research
proposals
Literature reviews
Presentations
MSc & PhD work
Research Methods
Communication of Ideas
Feedback is important:
To learn of new developments
To share responsibility
To get support and advice
To develop communication skills
Teamwork is important:
As a forum for feedback
To tackle larger problems
To learn interpersonal skills
Modern Science is advanced by sharing
ideas and working in teams
Research Methods
Presentations
Present your main idea & its significance
Omit complicated & old ideas
Structure:
Intro, context, body, some detail, conclude
Know your audiences’ background
Make sure that both non- & experts benefit
Don’t gloss over problems with your ideas
Anticipate questions
Research Methods
Slide Layout
Allow about 2-3 minutes per slide
Avoid too much text
Just cues
About 7-15 bullet points at most
Include graphs/charts/pictures
Avoid code/maths
Research Methods
Nervous ?
Prepare well, run through with your
supervisor
Time yourself
Contract stomach muscles & breathe out
hard
Speaking Skills:
A pause is better than an interjection
Speak more slowly than you think necessary
Repeat questions if they are inaudible
Research Methods
Postgraduate Work
A research lifecycle
Flavours of research
Ethics in research
Getting started
Research
proposals
Literature reviews
Presentations
MSc & PhD work
Research Methods
Postgraduate Research
Masters (MSc):
Must demonstrate that you are familiar
with the state of the art in the field and
are capable of working at that level
Doctorate (PhD):
A PhD additionally requires that you are
able to contribute new knowledge
Need not be a major contribution
Research Methods
Examiner’s Instructions
A Candidate must demonstrate that he/she:
Principles: Understands the nature, objectives and scientific
principles underlying the investigation
Literature: Is adequately acquainted with the relevant
literature
Methodology: Has mastered appropriate techniques and
analytical methods
Findings: Assesses the significance of findings in a thorough
and logically coherent manner
Independence: Shows evidence of critical and
independent thought
Writing: Reports on the study in an acceptable scientific
format that is satisfactory in both presentation and literary
style
Research Methods
Considering an MSc?
Choose your supervisor carefully
Talk to their current students
Be aware of their supervision style
Be prepared to change your topic if
necessary
But do this early rather than late
Organise finances upfront and prepare a
budget
Ask about Bursaries, Tutoring, etc.
Beware of part-time research. Consider the
opportunity cost
Research Methods