Reviewing

Pieter.Hartel@utwente.nl

[Ros07] T. Roscoe. Writing reviews for systems conferences. Technical report, ETH Zürich, Mar 2007. http://www.inf.ethz.ch/personal/troscoe/pubs/review-writing.pdf

Cyber-crime Science2

Research is a creative process

1. Study the literature

2. Think of a research question

3. Find an answer

4. Evaluate the answer

5. Write paper using standard format

6. Submit to conference or journal

7. Several Reviewers read the paper

8. The PC or Editor decide verdict

9. Celebrate if accept, else ...

Cyber-crime Science3

Purpose of the review

Quality assurance Justification of accept/reject Feedback to the authors Communicate your thoughts to the rest of

the PC An opportunity to clarify your own thoughts

Cyber-crime Science4

Steps writing the review

Read the paper first, scribbling notes in the margin

Lookup references if needed Write the review

» Summarise the paper in your own words» What in your opinion is the contribution?» How well does the paper fit the conference?» Write helpful comments for the authors on the basis of

your scribbles (more...)

Be cautious and constructive

Cyber-crime Science5

Types of comments

Have the authors missed relevant related work?

Is the paper well written? Any technical flaws? Anything important that is missing? Anything especially cool? Would this lead to a good presentation? Potential for a best paper award? Evidence of plagiarism and fraud?

Cyber-crime Science6

Program Committee Meeting

Preparation» Reread the papers and your reviews» Be sure to be able to explain your standpoint» Try to understand the standpoint of other reviewers» Have a look at the papers you did not review

Meeting» Be fair» Be concise» Try to reach a consensus» Above all else be professional!

Cyber-crime Science

Mini-conference

Must attend, 14 Jan,13:30-17:30, WA 4 5 min pitch (slides via email) A3+ poster (bring on paper) Best paper awards Selected papers can be published

7

Cyber-crime Science8

Final mark

We will mark pitch and poster We will read each paper and set aside unfair

reviews Paper mark is a weighted average of the

overall evaluation and the reviewers confidence

Final mark based on all of the above

Cyber-crime Science9

Exercise

Write a review of “Overcoming the Insider”

[Wil09] R. Willison and M. Siponen. Overcoming the insider: reducing employee computer crime through situational crime prevention. Commun. ACM, 52(9):133-137, Sep 2009. http://dx.doi.org/10.1145/1562164.1562198

[Har09] P. H. Hartel. Review of: “Overcoming the insider: reducing employee computer crime through situational crime prevention" by Willison R., Siponen M". Computing Reviews, page CR137444, Nov 2009. http://www.computingreviews.com/review/review_review.cfm?review_id=137444