Transcript IS8004 – Seminar 12 Presenting and Reviewing Qualitative Research Presenting and Reviewing   These also may seem to be quite different topics, but actually they.

IS8004 – Seminar 12
Presenting and Reviewing Qualitative
Research
1
Presenting and Reviewing


These also may seem to be quite different topics,
but actually they are closely linked
In the previous class, we looked at planning and
writing



Getting these right is critical to good research
Presenting relates to how you present the
research at a conference
Reviewing is how others evaluate what you
present – written or spoken
2
Presenting




Presentation of a written paper means that you
have to ensure it is well written, well argued, well
justified and pleasant to read
You have to write for the audience – and meet
their expectations (more or less)
You have to use language that they understand –
not too much jargon
If you present well, then reviewers can at least
understand your message
3
Presenting at Conferences




This is both harder and easier
You have much less time – typically 10-20
minutes; perhaps an hour in a seminar.
So you have to miss out much of the detailed
content – and focus on what’s really interesting
– for the audience
You also have to answer questions – which may
be quite difficult
4
Presenting 1





Although much work went into the planning of the
study, you may only assign 1 slide to this!
You may cite a few key literature sources, and mention
the methods
What should excite the audience is the results –
especially if they are novel or radical in some way
This can be usefully discsussed at more length
Interesting future research opportunities can be
discussed
5
Presenting Layout (10 mins)







1. Title, authors, affiliations
2. Introduction and Background
3. Literature
5. Method
6. Discussion of results
9. Future research
No conclusion or references. They are in the full
paper.
6
Dealing with Questions

If you know the answer, that’s great.


But often you don’t.
So, either you make something up (risky).




Or you deflect the question, like a good politician!
You could say something that is related.
Or, you say “that’s a great question” and ask the questioner
what s/he thinks!
Or ask for input from the rest of the audience


Usually there is at least one person with a *lot* to say who will be
keen to help out!
It is unlikely that you are the first to be in this state – you can learn
from others’ prior experiences.
7
Dealing with Discussants




A discussant is a person who has the task of
reading and commenting on all 2, 3, 4 papers in
a session at a conference and saying something
intelligent about all of them.
It is not an easy task – and discussants
sometimes try to present their own research
They also ask you questions.
Instead of replying directly, you can try to
develop a “conversation” with the discussant
8
Time





Conference time is limited, so you must not
overrun your time allocation
If you do, you may be cut off with many slides
not yet presented
The audience then focuses on the next paper
and forgets you.
You must budget for time very carefully.
I suggest a maximum of 1 slide per minute
9
Presenting Qualitative Research



Qualitative research is more about words and observations, so
you need to use that kind of evidence in the presentation
It is nice to cite what people said as evidence to support your
findings
But make the cites short and precise



2-3 lines at most, ideally less.
You could include a diagram, photograph or other media to
make a point
Examples from your own observations are nice


You can easily expand on them if the audience is interested
They can be the entry to a conversation – with audience or discussant
10
Reviewing






After you submit, then you have to wait
Some reviewers are fast, others slow
Most reviewers are critical and some are really nasty
Editors have the job of managing the review process
You can’t appeal to a reviewer (usually blind) but you
can appeal to an editor
You are an author – and will also be a reviewer


You see both sides of the situation
When you are the reviewer, try to be constructive and helpful
11
Reviewing & Reviewers

How do reviewers think and work?
Evaluation criteria
 Rejection criteria


Reviewing is common.
What are the advantages and disadvantages?
 Is there potential for bias or favouritism?


Developing competence in responding to
reviews

So as to increase your chance of being accepted
12
Attributes of Good Reviewers



Competent and Constructive
Reasonable, Unbiased and Open Minded
Ethical in their behaviour
Conflicts of interest
 Respect for your creativity and copyright




Not too critical, nor too lenient
Persuasive in their arguments/comments
Diligent and timely
13
Attributes of Good Reviews

The paper should be summarised



To show that the reviewer understood the paper
The strengths and weaknesses of the paper should
be identified
All advice for improving the paper should be
actionable



If there are weaknesses, precisely what should the author
do about them?
Which references should the author read?
Respect for the fact that it is the author’s, not the
reviewer’s, paper.
14
Evaluation Criteria for Papers

Ethics
Did the researcher act ethically?
 Did the researcher act to protect the interests of data
subjects (individuals or organisations)?


Research methods
Are the research methods used appropriate given the
nature of the research problem?
 Are the data collection and analysis methods
appropriate?

15
Evaluation Criteria for Papers




Are the references correct, and up to date?
Is the presentation clear, concise and
grammatically correct?
Are the concepts and arguments well organised,
structured and defensible?
Are the findings/contributions appropriately
positioned with respect to the existing literature
in this area?
16
Rejection Criteria



The article is uninteresting and no one would want
to read it
The problem researched is trivial, irrelevant or not a
problem at all
The article is so poorly constructed that a
completely new start is required


Or the arguments are so weakly/subjectively developed
as to be meaningless
There are serious ethical concerns about the way the
research was conducted

E.g. plagiarism, use of deception, illegal/unethical
practices, failure to protect research subjects’ privacy
17
IT&People Criteria









Criteria for papers are not bound to a particular methodology, but rather to:
1. is the topic relevant?
2. is the treatment new or newsmaking?
3. is the lit review adequate to support the framework presented?
4. Is the framework clear?
5. Is the evidence, discussion, description clear and convincing and does it
match the framework?
6. Does the paper apply standards appropriate to method (if quantitative
then defensible, if descriptive then thorough enough to reflect academic
responsibility)
7. Is the paper well-written, well organized
If no to any of the above, what is needed? However if the paper is so bad it
tires you to even think about it, then just reject. It is OK to reject, we have a
lot of submissions. If promising theme but not well developed, make some
general statements for revise and resubmit; if pretty good, then thorough
critique to help authors.
18
How to Respond to Reviewers?

Make sure that you address everything that they
ask for


Even if you choose to dispute their view
Provide a detailed presentation of your changes
in a two-column format
Left column – Reviewer Comments
 Right column – Author revisions
 Revision notes may be 10-20 pages long!


Even longer than the paper itself.
19
What if…




The reviewer totally disagrees with your choice
of method – and rejects for this reason alone?
You can’t change the method now!
So you may have to go to the editor (AE, SE, E)
and ask for a different reviewer
Sometimes it is the editor who says this, so you
learn not to send this kind of article to this
journal again
20
Remember!

Most journals accept 8-15% of submissions



And therefore reject about 85-92%
Conferences may accept 30, 50 or higher %
It is easier to reject (find fault) than accept
(appreciate). Less experienced reviewers and
editors tend to reject more – so if you have the
chance to nominate a reviewer or editor, don’t
pick a PhD student or recent graduate!
21
Decisions

If the editor offers a review and resubmit (R&R)
decision, be very happy!



The R&R may be major, but you must do as much as
you can and explain your changes carefully.


If you can get past the first round of review and stay in the
process, that is 50% of the way to acceptance
Because 50% of papers don’t make it through.
Don’t fight with the reviewers – you will lose, even if you are
right!
To get 2, 3 even 4 R&Rs happens. Don’t give up!
22
Journals



Each journal has its own culture, its own values
You need to learn about this – it will save you
much time
MISQ, ISR, ISJ, CAIS, JAIS, JMIS, I&M, DSS
23
References


Davison, R.M., Vreede, G.J. de and Briggs, R.O.
(2005) On Peer Review Standards for the
Information Systems Literature, Communications
of the AIS, 16, 49, 967-980.
Davison, R.M. (2003) Discussants and the
Quality of Interaction at Conferences,
Communications of the AIS, 11, 7, 128-136.
24