Transcript Review Process - How to review Fausto Giunchiglia

Review Process - How to review
Fausto Giunchiglia
By Fausto Giunchiglia and Alessandro Tomasi
Index:
1. Review Form 1
2. Review Form 2
3. Answer to the Reviews
4. Review Process
1. Review Form 1
1. Review Form 1
REVIEWER CODE:
1) Title
2) Author(s)
3) Paper Summary [short description of the message
and maybe of how it has been developed. What if
more than one message or no message?]
1. Review Form 1
4) Type of Paper/Research described
[Chose one of the options below and explain]
- Research (with original results) of which kind
(Theory, expemerimental, ...)
- Application (case study, ...)
- Synthesis of recent advances
- Other. Please specify
1. Review Form 1
5) General Ratings
[Rate within Bad/Weak/Fair/Good/Excellent] [0/1/2/3/4/5]
Put the score and some text motivating your score about:
5a) Relevance (with respect to the reference community)
5b) Originality (incremental, new work, ...)
5c) Significance of the work (how big the gap from the state of the art)
3d) Technical soundness
3e) References
3f) Presentation
1. Review Form 1
6) Technical Soundness
[Chose one of the options below and some text motivating
the choice]
- Technically correct
- Minor errors (indicate them)
- Major errors (indicate them)
- Unsupported claims (provide a detailed explanation)
1. Review Form 1
7) Presentation
[Rate within Yes/Somewhat/No]
Put the score and some text motivating your score
7a) Are the title and abstract appropriate?
7b) Is the paper well-organized (discuss course and fine grained
structure)?
7c) Is the paper easy to read and understand?
7d) Are figures/tables/illustrations sufficient?
7e) Is the English acceptable?
7f) Is the paper free of typographical/grammatical errors?
7g) Is the reference section complete?
1. Review Form 1
8) General Recommendation
[Please, chose one of the options below]
- Very strong accept (beautiful paper!)
- Strong accept (excellent and important contribution)
- Weak accept (good paper, some new interesting ideas)
- Weak reject (marginal, weak content, would require a
major revision)
- Strong reject (unreadable, nothing new, premature,
contains major errors)
1. Review Form 1
9) Main Reason for your Decision
[For accept choices please indicate one of the options below]
- accept because of the originality (good ideas, sound presentation)
- accept because of the quality of the proposed synthesis (useful review
on recent advances)
- other
[For reject choices please indicate one of the options below]
- reject because it is not relevant for the conference
- reject because of the presentation (unreadable, unstructured)
- reject because the content is too premature for really making sense
- reject because of the lack of originality (results already known, or similar
overview already published)
- reject because of major errors
1. Review Form 1
10) Your Level of Expertise (Compared to Level of Others)
- I am an expert of the field and know the relevant literature
- I understand the problem, I know some of the state of the art
- I only have a superficial understandings of the issues
11) Does the paper qualify for the best paper award [Y/N]
12) Comments to the Author(s)
[Please, provide here a clear justification of your ratings,
in particular with regards to the overall recommendation]
1. Review Form 1
13) Additional Comments to the Author
(after circulation of reviews among reviewers) (Can be
empty, cannot change previous review)
14) Additional Comments as Answer to Author's
Answer and Modifications
(Can Be Very Short:
- Evaluation Of Author Answer
- Value Judgement
- Final Score (Possibly Changed))
2. Review Form 2
2. Review Form 2
REVIEWER CODE:
1) Title
2) Author(s)
2. Review Form 2
3) Main Message:
Relevance: How relevant is the paper to the workshop?
0: not relevant at all
1: rather not relevant
2: relevant
3: very relevant
Technical Quality: What is the technical quality of the paper?
0: really bad
1: bad
2: good
3: really good
2. Review Form 2
Presentation: What is the overall presentation of the paper?
0: really bad
1: bad
2: good
3: really good
Overall Ranking: What is your overall recommendation?
0: strong reject
1: reject
2: weak reject
3: weak accept
4: accept
5: strong accept
2. Review Form 2
Confidence: Reviewer's expertise in the area
0: I know little about this area
1: I know enough about this area
2: I have good expertise in this area
Why to accept? What are the most important reasons to
accept this paper? (1-3 sentences)
Why to not accept? What are the most important reasons
NOT to accept this paper? (1-3 sentences)
Comments: Detailed comments on the paper (primarily for
the authors)
3. Answer to the Reviews
3. Answer to the Reviews
<Brief introduction>
LIST OF {
<general comment quoted from reviews>
<your answer arguing how you have accordingly modified the paper>
}
Moving now to the more specific comments:
LIST OF {
<specific comment quoted from reviews>
<your answer arguing how you have accordingly modified the paper,
providing detail but not too much>
}
<Concluding sentence>
4. Review Process
4. Review Process
0) Abstract
(send it to [email protected] not later than a week before the presentation)
1) Presentations
(Fortunate situation where you may know of what the paper is about)
2) All Papers Submitted by June, 13th
3) Review Allocation (by Program Chair) by June, 20th
4) Reviews Due by July, 4th
5) Reviews circulated to Reviewers for additional comments
6) Reviews send to Author by July, 11th
7) Author sends back answer and modified paper
by July, 23rd
8) Reviewers provide final answer
9) Exam Pass/No Pass (of Authors and Reviewers)
by July, 29th
4. Notice!!!
In the paper you have to use the same version of
the stylefiles. It is available by downloading it from
http://www.springeronline.com/sgw/cda/frontpage/
0,11855,3-111-2-124365-0,00.html
Avoid using stylefiles acquired from other sources
as these may not be correct
How to do
presentations
23
Structure
1.
Presentation methods
2.
Attitude towards the audience
3.
How to select content
4.
How to structure
5.
Introduction
6.
Stage fright
7.
Main part
24
Structure
8.
Using pictures
9.
Visual aids
10. Interposed questions, interruptions
11. Final discussion
12. The end
13. Optical – acoustic factors
14. Ancillary conditions
25
1. Presentation methods
Rule No. 1:
Control effect
Rule No. 2:
Integrate audience
26
1. Presentation methods

Seminar

Lecture

Presentation with final discussion

Presentation with intermediate
discussion
27
Comparison of some
presentation methods
What remains
Time to
prepare / h
Open discussion
80%
4,0
Presentation with
Intermediate
discussion
60%
2,5
Presentation with
Final discussion
30%
1,7
Lecture
20%
1,0
28
2. Attitude towards the
audience

Who is the audience

What do they know

What are they interested in

What do they understand
29
2. Attitude towards the
audience






Contact with eye
Simple and understandable languages
Concentrate on the important
Credibility
Reply to objections
Avoid „techno“ language
30
3. How to select content
How much?
From where?
What is interesting?
What is the objective?
What should the
audience do?
Do not want too
much!
31
4. How to
structure –
catch words

Speak as you think –
follow the thinking

Use written concept
of catch words
32
5.
Introduction



Try to find common
understanding with the audience
Lead to the subject
Initially: create attention
33
5. Introduction
Say at the beginning
 The structure
 The length
 Invite to a dialogue
But at the beginning:
Who are you
34
attention
Hope that
it will end
soon
Length of presentation
35
6. Stage-fright
Afraid to speak



Is natural!
Everybody has it !
Is not forever!
36
Reduce Stage-fright

Be well prepared!

Learn important parts by hard!

Relax!

Look for a positive „point“

Do some contacts before presentation!
37
7. Main part
Introduction
Main part
Current situation
Potential solution
How to realise
End
39
8. Using pictures /
figures
Picture




Take from the context of the
audience
Has to increase the message
To explain the issue
Does not be an end on itself
40
8. Metaphorical
language





To be a
comparison
Should be
practial
Story
Citation
Joke
41
9. Visual aids





Black board
Projector
Film
Videobeam
Overhead-Display
42
Presentation programs

Grafical Layout
 Colours – Contrast – Background
 Clipart-files
 Fonts (size)

Produce hand outs
43
What we keep
10%
20%
30%
50%
70%
90%
Reading
Hearing
Seeing
Hearing & Seeing
Reporting
Do it yourself
44
10. Questions

Question
 Are welcome
 Should be answered immediatly
 Bit can also be delayed or
forwarded to somebody else

Interupptions
 ignore!
 Look at the person!
 Stopp speaking!
45
11. Final discussion





Announce already in the introduction
Write up issues of discussion
Keep sequence of questions
Repeat questions (if necessary)
Draw conclusions
46
12. End
 Avoid
 New questions which do not help in the conclusions
 Main
ideas
 Should be summarized
47
12. End
Main idea + review objective + how to realise
=
Good end
48
13. Acoustic factors

No „speaking smoking"

Speak loudly and slowly

You should vary volume and
speed

Make breaks

Try presentation beforehand
49
13. Optical factors
Behave natural!



With your body
With your hands
With your eyes
50
14. Other circumstanceas

Room size

Lighting

Ventilation

Sockets

Other means (Microfon)

Breaks
51
Reading - How to read
Fausto Giunchiglia
Literature:
Bruno Buchberger, Thinking Speaking Writing
By Fausto Giunchiglia and Alessandro Tomasi
Index:
1. The Role of Literature
2. What is literature
2.1 Classify the results
3. The Documentation of Literature
4. Steps in the Use of Literature
1. The Role of Literature
1. The Role of Literature
“Keep Re-Inventing the wheel”
For many problems:
• the solutions is already in the literature
• the literature has solutions in similar problems
1. The Role of Literature
It is necessary to know
how the literature is organized:
• within your research community
• outside your research community
[translate other concepts
in our community]
Necessary
Useful
but not
Necessary
1. The Role of Literature
There is an interleaving of:
1) Try to solve the problem yourself
2) Read the Literature
1. The Role of Literature
Interactive Process of Reading Papers:
Step 1
Quantity
Step 2
Step 3
…
Depth
1. The Role of Literature
How do I know when I can stop reading?
When I know the 90% of the papers
cited by relevant papers
1. The Role of Literature
To keep Scholarship:
• go to conference
• talk to people (best opportunity in conference)
• journals
• proceedings
• web
•…
2. What is Literature
2. What is Literature
1. Monographs / Books
Main Way
to do Career
2. Journal Articles
3. Articles in Collections
4. Papers in Proceedings of Conference
5. Papers in Proceedings of Workshop
6. Technical Reports
7. Grey Literature (Ex: Web Sites)
More important
than papers
2. What is Literature
Each research community has:
1-2 Top Journals
1-2 Top Conference
2.1 Classify the results
2.1 Classify the results
1. Monographs / Books:
Note:
the research is finished
Contents:
extensive/complete description of research
Originality:
None (2/3 years old)
Quality Control:
Very Good
2.1 Classify the results
2. Journal Articles:
Note:
Come regularly
Contents:
Very good, very crisp, specialized
Originality:
Full of two years ago (in CS is obsolete)
Quality Control:
Very High (Many shots)
2.1 Classify the results
3. Articles in Collections:
Note:
Not regularly
Contents:
Very good, very crisp, specialized
Originality:
Full of two years ago (in CS is obsolete)
Quality Control:
Lower
2.1 Classify the results
4. Conferences:
Note:
Is Refereed
Contents:
New results, almost complete, semi-final
Originality:
Good (6 months ago)
Quality Control:
Medium (only one shot: accept or reject)
2.1 Classify the results
5. Workshop
Note:
Is Not Refereed
Contents:
New results, almost complete, semi-final
Originality:
Good (6 months ago)
Quality Control:
Medium (only one shot: accept or reject)
2.1 Classify the results
6. Technical Reports
Note:
--Contents:
Detailed about a specific topic
Originality:
Maybe
Quality Control:
Low (No Refereeing Process)
2.1 Classify the results
7. Grey Literature (Ex: Web Sites)
Note:
The main way to publish our results
Contents:
Random
Originality:
Random
Quality Control:
Random
3. The Documentation of Literature
3. The Documentation of Literature
• Library
- author files
- keywords files
• Review Journals
- Computing Journal
- Artificial Intelligence Review
- Scientific Magazines
- Scientific Magazines for Spreading Communications
- Journal for Publish new Results
• Bibliographies
• Citation index
• Research index
3. The Documentation of Literature
Research Community are organized
into societies
Produce:
Organized Conferences, Journals, Magazines
Examples of Societies:
ACM, IEEE, ECCAI, VLDB, …
3. The Documentation of Literature
Citations Index:
• defines journals
• cites titles, authors, abstract of all paper
Example:
ISI, the most important for Science
3. The Documentation of Literature
Research Index:
• the new game
• is on line
• how many times you are cited in Internet
Example:
http://citeseer.nj.nec.com
4. Steps in the Use of Literature
4. Steps in the Use of Literature
1. Use CiteSeer (http://citeseer.nj.nec.com)
2. Ask to Colleagues, Advisor,
Friends, … by e-Mail
3. Consult References
4. Consult Review Journals
5. Ask Authors