Chapter 4 – Issues in Single Subject Research

Ps534

Dr. Ken Reeve Caldwell College Post-Bac Program in ABA 1

Review…

 RULES TO FOLLOW AS AN APPLIED BEHAVIOR ANALYST – 1. Do no harm!

– 2. Do some good!

– 3. Identify WHAT you did that did some good! (Focus of this course!) 2

Review…

 TARGET BEHAVIOR = DEPENDENT VARIABLE  METHODS TO CHANGE BEHAVIOR (INTERVENTION) = INDEPENDENT VARIABLE  FUNCTIONAL RELATIONSHIP = degree to which the independent variable affects the dependent variable (and can you isolate this effect!!) – Main focus in research is to determine functional relationship between intervention X and learning outcome Y 3

Features in a Functional Relationship

 When a FUNCTIONAL RELATIONSHIP does really exits between IV and DV, then certain other characteristics will be present in the research findings. These include:  Prediction, replication, verification, reliability, and validity (we’ll describe these and other related terms…) 4

PREDICTION

 When an intervention really DOES have an effect on behavior, then there is an expected outcome (behavior goes up/down, occurs faster/slower, topography changes, etc.)  When an intervention really has NO effect on behavior, then there is an expected “NO CHANGE” as the behavior outcome 5

VERIFICATION

 This refers to CONFIRMING that behavior really changed as expected (behavior goes up/down, occurs faster/slower, topography changes, etc.)  VERIFICATION requires that we have a good operational definition of the behavior in question, a good measurement system, a good way to graphically depict the data, and accurate observers 6

REPLICATION

 This refers to REPEATING what you did to see if your prediction about the functional relationship in question holds true more than just once  Why? Sometimes we can get amazing results JUST BY CHANCE and we want to make sure that the results weren’t a fluke 7

REPLICATION: S-S vs. GROUP Research

 In group studies, researchers usually replicate the observed functional relationship with NEW participants  In single-subject research, researchers usually replicate the functional relationship within the SAME participant AND then also replicate it with new individuals 8

REPLICATION: DIRECT vs. SYSTEMATIC

 DIRECT REPLICATION: means to do the study exactly the same way as before.

– Important but boring. – Increases INTERNAL validity but not EXTERNAL validity (we’ll define these in a minute).

 SYSTEMATIC REPLICATION: means to do the study with some variation as compared to before (change population, setting, stimuli, behavior studied, etc.). – Important and more exciting. – Increases INTERNAL AND EXTERNAL validity.

9

RELIABILITY

 Refers to how dependable our observations of the functional relationship are  It asks: Can we put faith in what we are seeing happen before our eyes?

 IF there is a real functional relationship AND we are using operational definitions, good measurement systems, IOA, THEN we should see the same effect repeating to the same degree  Also means that nothing BREAKS DOWN in the research process (think of a “reliable” car!)  REPLICATION also shows us RELIABILITY of an effect 10

What hurts RELIABILITY?

 POOR IOA – observers need to be well trained!

 REACTIVITY of participant – when a person changes behavior because he or she finds out an observation is occurring  OBSERVER DRIFT – over time, the observers change their operational definition of the DV 11

VALIDITY

 Recall, yet again, that our goal is to identify a functional relationship between IV and DV  IF we can say that the IV or intervention was the ONLY thing that changed the DV, THEN we have high INTERNAL VALIDITY – To be able to do this, we need to “rule out extraneous variables” by not letting them change in any way while we are manipulating the IV (called “controlling for potential confounds”) 12

INTERNAL VALIDITY

  Here’s an example you might find in the “results” section of a research journal: “As can be seen in Figure 1, when treatment was introduced, there was a 40% increase in rate of responding as compared to baseline levels. When treatment was removed, rate of responding dropped back down to baseline measures. When treatment was re-introduced, rate of responding again increased by 40% as compared to baseline levels. Because no other variables were free to change during the study, it can be inferred that the behavior change was a direct result of the introduction and removal of the treatment procedure.” 13

DREADED THREATS TO INTERNAL VALIDITY

 We have a “threat to internal validity” when other extraneous variables ARE free to change during the study  If a threat is confirmed (or we have good reason to think one happened), then we have CONFOUNDED the results – As a result, it CANNOT be inferred that the behavior change was a direct result of the introduction and removal of the treatment procedure.

14

GOOD EXAMPLE OF

THREAT TO INTERNAL VALIDITY

 “When facilitated communication is used, participants are able to answer questions that they never could before.”  Sounds like a good functional relationship, right? –

“If no FC, then no responding from participant; If YES FC, then YES responding.”

 But…there is a BIG threat to internal validity that, in fact, has been confirmed as a CONFOUND in pro-FC studies 15

TYPES OF

INTERNAL VALIDITYTHREATS

 HISTORY – when some event or action by another OUTSIDE OF THE “LAB” affects the DV of the participant  If researcher is made aware of these, then we don’t have good internal validity!

– EXAMPLES INCLUDE: giving a new diet, adding another therapy, doing additional training, bad week at home, etc.

16

TYPES OF

INTERNAL VALIDITYTHREATS

 MATURATION – when the DV of the participant changes as a result of growth and development  S-S research controls for this nicely because we observe DV of individual multiple times to make sure maturation is NOT the cause of any changes observed!

17

TYPES OF

INTERNAL VALIDITYTHREATS

 Can this change in DV be due to MATURATION?

18

TYPES OF

INTERNAL VALIDITYTHREATS

 ATTRITION – refers to outcome data that is skewed due to certain participants dropping out of a study – Ex. If only the children with the LEAST severe autism remain in a study, then it is tough to determine if treatment was really good 19

TYPES OF

INTERNAL VALIDITYTHREATS

  MULTIPLE TREATMENT INTERFERENCE – refers to problem of selecting out what part(s) of a treatment “package” were responsible for outcome data  This is a HUGE problem in treatment of autism but it is also very pervasive in psychological therapies in general Problem is that many claims are made about WHAT PART is the cause of the effectiveness – Ex. “Floortime,” “Son-Rise,” and TEEACH use many ABA techniques. So does “Rational Emotive Behavior Therapy” 20

TYPES OF

INTERNAL VALIDITYTHREATS

 TREATMENT DRIFT - refers to problem of therapists changing the intervention over time –

Ex.) Therapists may inadvertently increase amount of reinforcement, may make instructions more salient than before, may prompt more effectively

–

If this happens, again we have (take your pick!):

• • • •

A problem inferring WHAT caused the change in DV A problem identifying a functional relationship A threat to internal validity A confound in our results

21

TYPES OF

INTERNAL VALIDITYTHREATS

 EXPERIMENTER BIAS - refers to MANY possible problems of researcher affecting the results (inadvertently, we hope) –

Ex.) failing to acknowledge confounds, incomplete data collection, selecting participants most likely to do well, etc.

–

If this happens, again we have (take your pick!):

• •

A problem inferring WHAT caused the change in DV A problem identifying a functional relationship

• •

A threat to internal validity A confound in our results

22

EXTERNAL VALIDITY

 Will the functional relationship identified hold true with different participants, settings, materials, variations in instruction, etc. – If so, then we have HIGH external validity.

 Do the findings only work under very narrow circumstances? – If so, then we have LOW external validity  BIG PROBLEM: people often confuse poor external validity with poor internal validity in S-S research. How?

– “We shouldn’t use that intervention because it has only been shown to work with one person! We need more studies with groups of people!” (THIS IS POOR LOGIC!) 23

EXTERNAL VALIDITY’S RELATION TO INTERNAL VALIDITY

 Notice that we need to do SYSTEMATIC REPLICATIONS to increase external validity  But, if each systematic replication continues to show the functional relationship between IV and DV, then we are ALSO increasing INTERNAL validity  Successful replications show us that the results are likely not due to extraneous variables but are caused by the IV manipulated by the researcher!

 This relationship is ONE-SIDED, though: increasing internal validity does NOT increase external validity!

24

ETHICS

  What are they?

A set of standards for behavior that pertains to “doing what is right”    Behavior analysts must abide by a very stringent code of ethics since the behavior change techniques used in ABA are extremely powerful See the BACB’s code for conduct (NOTE: You should download these from the website to study. There will be some general questions about the BACB code on our class exam!) 25

 End of chapter 4 26