Transcript Distributed and Embedded Systems (DIES)

Research methods
Marianne Junger
[Fie09] A. Field. Discovering statistics using SPSS. Sage, London, 3rd edition, Jan
2009. http://www.uk.sagepub.com/field3e/samplech.htm.
Science
Some scientific questions are about logic,
like mathematics & philosophy (perhaps)
Some issues are about finding out how
things are in the real world
Empirical research
• Is used to study ‘real world’
• E.g., human behaviour & physics
2
Cyber-crime science
Many questions are ‘empirical questions’
What sort of math-teaching helps children
most to learn mathematics ?
Is crime increasing or decreasing?
Does drug use lead to crime?
Does technology ‘cause’ crime?
Does ICT cause crime?
These questions are a matter of correct
empirical research
3
Cyber-crime science
Why is empirical research important?
Many cognitive biases colour our judgment
Field of crime & Criminal Justice: ‘faith
triumphs over facts’ [Wal06a]
Examples of sensitive issues:
1. Does visible police patrol help to combat
crime?
2. Does ‘Scare straight’ help to reduce
crime among first time offenders?
4
[Wal06a] S. Walker. Sense and Nonsense About Crime and Drugs: A Policy Guide.
Cengage Learning, sixth edition, 2006.
Cyber-crime science
http://www.cengage.com/asiahed/instructor.do?product_isbn=9780534616540.
Systematic data collection to avoid bias
We want to find the ‘correct’ answer
• Independent of – our own - beliefs
Aim: finding regularities- patterns in
behaviour
For aggregations – groups –not individuals
5
Cyber-crime science
Research cycle [Fie09]
6
Cyber-crime science
Correct procedures & methods to avoid errors
1. Think about causality
2. Explicit formulation of your research question
3. Units of analysis
4. Identifying main concepts
5. Determining how to measure them
6. Choosing a research design
Separate issues but linked within your study
7
Cyber-crime science
1. Causality: three conditions
 Time order: cause precedes effect.
• Do drugs lead to crime?
• Does crime lead to unemployment?
Association between cause and effect
No third variable causes both ‘cause’ and
effect = spuriousness
• For instance: association between crime
& unemployment
8
Cyber-crime science
Spuriousness
 Association between ‘unemployment-age
30’ and ‘crime-age 30’ is spurious: it
disappears after controlling for IQ-age 12
AGE 12:
Low IQ
Poor school results
Age 30
Unemployed
Age 30
Crime
9
Cyber-crime science
2. Explicit formulation of your research
question
Some questions are hard to answer
• Was Julius Cesar a happy person? =
subjective
• Could we have avoided WWII? = history
• Should we interfere with military action
in Syria? = moral/political
Criterion: You need to be able to set-up your
study as required by methodological rules
10
Cyber-crime science
2. Explicit formulation of your research
question (contd.)
Relationships between concepts
1. Is crime decreasing or increasing?
2. Is age related to crime?
3. Are unemployment and Crime linked?
Causality
4. What leads to a decrease in crime?
5. Is unemployment causing crime?
11
Cyber-crime science
3. Units of analysis
1. Individuals: children, adolescents, citizens
2. Pairs of individuals: mother & daughter,
offender & victim
3. Groups: countries, provinces,
organization(s)
4. ‘Social artefacts’= product of behaviour:
Crimes/incidents, alcohol related crashes
5. Combinations.
12
Cyber-crime science
4. Identifying main concepts
Crime: how to define it?
 ‘Acts of force or fraud undertaken in the
pursuit of self-interest' [Got90a].
‘Crimes are defined as acts or missions
forbidden by law that can be punished by
imprisonment and/or fine’ [Wil98]
[Got90a] M. R. Gottfredson and T. Hirschi. A General Theory of Crime. Stanford University Press,
1990. http://www.sup.org/book.cgi?id=2686.
[Wil98] J. Q. Wilson and R. J. Herrnstein. Crime & Human Nature: The Definitive Study of the Causes
of Crime. Free Press, Jan 1998.
13
Cyber-crime science
5. Determining how to measure:
‘operationalisation’
How are you going to measure crime?
Where, or from whom can you collect
information?
Any suggestions?
Remember basic lesson of statistics ‘garbage
in = garbage out’
14
Cyber-crime science
How to measure phishing?
1) Behavioral data: respondent
1. Accepted a ‘friend’ request from an unknown person
2. Provided the requested information
2) Self-report: the respondent’s likelihood of
1. Accepting the friend request (mean=3.10, s.d. = 1.36)
2. Accepting the information request (mean=2.79, s.d. = 0.97)
•
1-5 response scale, ranged from ‘Not at all likely’ to ‘Very
likely’
Question: which is to be preferred?
15
[Vis15] A. Vishwanath. Habitual Facebook use and its impact on getting deceived on social
media. J. of Computer-Mediated Communicatio, page to appear, 2015.
http://dx.doi.org/10.1111/jcc4.12100.
Cyber-crime science
How to measure phishing?
- 33 subjects accepted the level 1 request (total N=150)=22%
- 19 responded to the level 2 request = 13%
- the behavioural data correlated with the respective self-report measures
 During exit interviews
• Some subjects who accepted the friend-request indicated they had subsequently
unfriended the phisher
• Some others mentioned they were planning to do the same
• Few others were about to accept the request but were waiting for their friend(s) to first
accept the request
• Some subjects indicated that they were about to accept the second request but had not
gotten down to it
• A few subjects responded to the request but asked for clarification
 ‘Thus, the behavioral measures appeared to have limited validity and did not accurately
reflect the variance in individuals’ likelihood of responding to the phishing requests.’
16
Cyber-crime science
5. Determining how to measure: examples
Self-reports on offending

Truthful, do people remember?
Self-reports on victimization

Truthful, do people remember?
Police records

Information complete, how often are offenders
caught?
What else?
17
Cyber-crime science
Levels of measurement
Nominal: attributes of people/incidents

Sex, city, ethnic group, country of origin…
Ordinal: logical rank order

Levels of education, crime seriousness
Interval: distance can be determined exactly

IQ
Ratio: true zero point

Value of property loss, in €, blood alcohol, length
of incarceration
18
Cyber-crime science
Criteria – questions: go to clinic
 Reliability

Consistency or stability of your measures

Do people give same answers each time?

Even DNA measures are not always reliable [Rot10]
 Internal validity

Do they measure what they ought to?

Self report measure of crime, based on 1 question?
 External validity

19
Is experiment representative?
[Rot10] A. Roth. Database-driven investigations. Criminology & Public Policy, 9(2):421-428,
May 2010.
Cyber-crime
sciencehttp://dx.doi.org/10.1111/j.1745-9133.2010.00638.x.
Validity & Reliability
Which concepts are you interested in?
Your measures need to be

Valid: internal and external validity

Reliable
Units of analysis: on what population or
sample will you collect information
20
Cyber-crime science
6. Choosing a research design
Where all the building blocks/issues come
together
How can you study the relationship
between
21

Crime & unemployment

Laptop theft and building design
Cyber-crime science
Designs – questions: go to clinic
Cross-sectional
Longitudinal

Approximation of longitudinal design

Does longitudinal design solve causality
problem?
Experiments. Advantages
22

Causation

Policy evaluation, (medical) intervention
Cyber-crime science
Randomized Controlled Trials =
the GOLD standard
Pp Pp Pp Pp Pp Pp Pp Pp Pp
Experimental
group
23
Cyber-crime science
Control
group
Why is randomization necessary?
Self-selection:

Persons sort themselves out to situations
Creaming

In organizations: less problematic offenders get
lighter sanctions & get alternative sanctions
Answer = randomization process
24
21/07/2015
Cyber-crime science
Replication
Easy way to think about quality of a study is
Replication
25
Cyber-crime science
Conclusions
(Almost) no study is perfect
Therefore research is about reading & doing
the next study better/smarter than the
previous one
26
Cyber-crime science
That’s it
See you next time
###
27
Cyber-crime science