Chapter 9 Measurement & Data Collection in Research

Transcript Chapter 9 Measurement & Data Collection in Research

Chapter 10 Clarifying Measurement and Data Collection in Quantitative Research

Concepts of Measurement Theory

Directness of Measurement

 Directness of measurement  Direct measures (concrete things such as oxygen saturation, temperature, weight)  Indirect measures (abstract concepts such as pain, depression, coping, self-care, and self esteem) Copyright © 2011 by Saunders, an imprint of Elsevier Inc.

Measurement Error

 Difference between the true measure and what is actually measured   Systematic error: the variation in measurement is in the same direction Random error: the difference is without pattern Copyright © 2011 by Saunders, an imprint of Elsevier Inc.

Levels of Measurement

Nominal-Scale Measurement

    Lowest of the four levels of measurement Categories that are not more or less, but are different from one another in some way Mutually exclusive and exhaustive categories Named categories Copyright © 2011 by Saunders, an imprint of Elsevier Inc.

Example of Nominal Data

Ordinal-Scale Measurement

Interval-Scale Measurement

   Numerical distances between intervals Absence of a zero point Likert scale scores    1 = Strongly disagree 2 = Disagree 3 = Neutral   4 = Agree 5 = Strongly agree Copyright © 2011 by Saunders, an imprint of Elsevier Inc.

Ratio-Scale Measurement

Example of Ratio Data

Reference of Measurement

  Norm-referenced testing Tests performance standards that have been carefully developed over years with large, representative samples using a standardized test with extensive reliability and validity Copyright © 2011 by Saunders, an imprint of Elsevier Inc.

Reference of Measurement (cont’d)  Criterion-referenced testing  Comparison of a subject’s score with a criterion of achievement that includes the definition of target behaviors • When behaviors are mastered, the subject is considered proficient in the behaviors.

Reliability Testing

    Concerned with how consistently the measurement technique measures the concept of interest Needs dependability, consistency, accuracy and comparability If expressed as a correlation coefficient, 1.00 is perfect reliability, whereas 0.00 is no reliability.

The lowest acceptable coefficient for a well developed measurement tool is 0.80.

Types of Reliability

  

Stability:

Concerned with the consistency of repeated measures or test-retest reliability

Equivalence:

Focused on comparing two versions of the same instrument (alternate forms reliability) or two observers (interrater reliability) measuring the same event

Homogeneity:

Addresses the correlation of various items within the instrument or internal consistency; determined by split-half reliability or Cronbach’s alpha coefficient Copyright © 2011 by Saunders, an imprint of Elsevier Inc.

Interrater Reliability

Critiquing for Reliability

What reliability information is provided?

Does the author include reports of the validity of the instrument from previous studies?

Did the author perform pilot studies to examine the validity of the instrument?

Did the researcher report use of data from the present study to examine instrument validity in the discussion section of the report?

What Is Validity?

 It is the extent to which an instrument reflects the concept being examined.

Physiological Measures

Critiquing Methods of Measurement      Were the measurement strategies clearly identified and described?

What level of measurement was achieved by each instrument?

Was the reliability of each measurement method adequate?

Was the validity of each measurement method adequate?

If physiological instruments were used, were they accurate and precise?

Examples of Physiological Measures   Physical measurement methods  ECG, BP, I&O  SVO 2 , pulse oximetry Microbiological    Smears Cultures, number of CFU Sensitivities Copyright © 2011 by Saunders, an imprint of Elsevier Inc.

Observational Measurements

Critiquing Observational Measurement 1.

Is the object of observation clearly identified and defined?

Is interrater reliability described?

Are the techniques for recording observations described?

Types of Interviews: Unstructured

Examples of Unstructured or Open Ended Interview Questions    Tell me about...

What has been your experience with...

What was it like to hear you have cancer?

Types of Interviews: Structured

 Structured interviews   Describing interview questions Pretesting the interview protocol     Training interviewers Preparing for an interview Probing Recording interview data Copyright © 2011 by Saunders, an imprint of Elsevier Inc.

Closed-Ended Interview Questions    Structured Response alternatives fixed Which would you rather do,

Critiquing Interview Methods

Do the interview questions address concerns expressed in the research problem?

Are the interview questions relevant for the research purpose and objectives, questions, or hypotheses?

Does the design of the questions tend to bias subjects’ responses?

Does the sequence of questions tend to bias subjects’ responses?

Focus Groups

     Used to study qualitative issues Obtain participants’ perceptions of narrow subject in a group interview session Give group a feeling of “safety in numbers” Nonverbal approaches are included.

Discussion helps to provide depth of data.

Focus Group Considerations

    May sort participants into smaller groups with common characteristics: segmentation Need to select an effective moderator to keep discussion on track The setting should be relaxed and comfortable.

High-quality tape recordings should be made.

Critiquing the Use of Focus Groups 1.

What was the aim of the focus group?

Was the group size appropriate for the focus group method?

Was group sufficiently homogeneous to speak candidly?

Was moderator successful in keeping discussion focused?

Critiquing the Use of Focus Groups (cont’d) 5.

Was the aim of the focus group achieved?

Did conclusions appear to be representative?

Were minority positions identified and explored?

Measurement Strategies

Questionnaires

Scales

Example of Items in a Likert Scale

Example of Visual Analog Scale

Critiquing a Scale

Is the instrument clearly described?

Are the techniques that were used to administer and score the scale provided?

Is information about validity and reliability of the scale described from previous studies?

Is information about validity and reliability of the scale described for the present sample?

If the scale was developed for the study, was the instrument development process described?

Critiquing the Data Collection Process     Was data collection process clearly described?

Was data collection conducted in a consistent way?

Were research controls maintained?

If data collectors were used, were they adequately trained?

Serendipity

  The accidental discovery of something useful or valuable.