Transcript Slide 1
I want to test a wound treatment or educational program but I have no funding or resources, How do I do it? Implementing & evaluating wound research conducted using pre-experimental designs part II AAWC WocSpec Research Committee, Barbara M. Bates-Jensen, PhD, RN ABSTRACT PURPOSE: To provide information on how to evaluate and improve methodology of wound research conducted using pre-experimental designs. BACKGROUND: Research methodology is concerned with how a study’s design is implemented and how the research is conducted. Methodology determines the quality of the data generated by the study and specifies: when and how often data are collected, construction of data collection measures, who data is collected from, how data is analyzed, and how findings are presented. Study design is a part of research methodology but not the only part. METHODS: Pre-experimental studies include: case study design, static group comparison or cross sectional design, and one group pre-test/post-test design including longitudinal time series and panel designs. These designs are frequently concerned with answering questions using an experimental approach when resources are not available for conducting more rigorous experimental designs. We often use pre-experimental designs for description and thus we are concerned with how to answer questions such as: How many? How much? How efficient? How effective? How adequate? We present information on conducting static group comparisons and one group pre-test/post-test designs for wound research. Advantages and disadvantages and methods for increasing the strength of pre-experimental wound studies are outlined. CONCLUSIONS: Pre-experimental designs follow basic experimental steps but do not include a control group. Usually a single group is studied but no comparison between an equivalent control group is made. The aim of preexperimental design is to evaluate the influence of a variable or to determine the influence of a variable on one group and not on another. Limitations of one group pre-test/post-test designs are that the approach provides a measure of change but cannot provide conclusive results. Weakness in static group comparison designs lies in the failure to examine equivalence of groups before the implementation of the wound treatment or educational program. Methods to improve the quality of wound research using pre-experimental designs include attending to: historical events that may have occurred between the pre-test and post-test, maturation changes in the subjects, difficulties with pencil and paper measures for pre-tests and post-tests, instrument changes/calibration, subject/wound selection biases. BACKGROUND Research methodology determines the quality of the data collected in a study, it concerns how the design is implemented and specifies: • when and how often data are collected • development of data collection measures & instruments • who data is collected from (the sample) • how data are analyzed • how findings are presented. Whether or not study results are really the result of the variable on the group under investigation in the study (internal validity) and not some other variable AND Whether or not the study results can be expected to be the same with other groups of people (external validity) are issues that are addressed by methodology. Research design is an important part of methodology. Study design is a part of research methodology but not the only part. In general, a research design is the plan for the research study. The design guides when and how often to collect data, what data to gather and from whom, and how to analyze the data. A research design refers to the type of study that will be conducted, whether it will be preexperimental, quasi-experimental, or true experimental. PRE-EXPERIMENTAL METHODS Pre-experimental research designs: answer questions using an experimental approach when resources are not available for conducting more rigorous experimental designs are used to describe a variable often answer questions such as: How many? How much? How efficient? How effective? How adequate? follow basic experimental steps but do not include a control group usually involve study of a single group with no comparison between an equivalent control group The aim of pre-experimental design is to evaluate the influence of a variable or to determine the influence of a variable on one group and not on another. There are 3 pre-experimental research designs: 1. case study 2. static group comparison/cross sectional 3. one group pre-test/post-test (longitudinal time series and panel) One group pre-test/post-test designs Treatment Group O1 X O2 X = Wound Treatment or Educational Program O1 = Pre treatment or educational program observation measurement or Pretest O2 = Post observation measurement or Posttest 1 group gets the wound treatment or education. A pretest or pre-observation measure is taken first to establish baseline knowledge or initial wound outcomes, then a posttest or post observation measure is determined to measure the difference between the pretest and posttest. There is no comparison group. What happens in a One group pretest/post-test design: Data is collected on relevant variables before and after the wound treatment or education from subjects. The same data is collected for the pretest and posttest. Data is collected repeatedly over a long period of time. If data are collected on the same variable at regular intervals it is a time series design; reveals changes in the aggregate such as in a population. If data are collected from the same subjects over time it is a panel design; reveals changes at individual level. Findings reveal different patterns or trends. advantages TIME SERIES Data are easy to collect Results are easy to present in graphs Interpretation is relatively easy PANEL Reveal changes at the individual level Establishes time order of variables Can explain how relationships emerge. limitations Takes longer to complete the study because of the longitudinal nature of the design TIME SERIES Data collection method may change over time Difficult to show more than one variable at a time May need qualitative research to explain fluctuations Assumes present trends will continue unchanged. PANEL Difficult to obtain initial sample of subjects Difficult to keep the same subjects over time Repeated measures may influence subjects’behavior. How to make One group pre-test/posttest designs stronger: Treatment Group Comparison Group O1 O2 O3 O1 O2 O3 X O4 O5 O4 O5 X = Wound Treatment or Educational Program O1 O2 O3 = Pre treatment or educational program observation measurements or Pretests O4 O5 O6 = Post observation measurements or Posttests Add a comparison group. The O1 O2 O3 pretests allows determination of any differences between the treatment and comparison group before the wound treatment or educational program is delivered so any difference in the posttests or post observation measurements are more likely to be related to the wound treatment or educational program. The comparison group allows evaluation of the effects of: history (something happening during the data collection time points) maturation (something happening to the subjects during the data collection time points) Some considerations to improve One group pre-test/post-test studies Use the same pretest and posttest measures. Gather the same number of pretest measures as you do posttest measures Conduct reliability testing throughout the duration of the study to control for measurement drift and to be sure the same measurement process is used for all pretests and posttests Be sure any equipment is routinely calibrated during long studies Report drop-out rates and run analyses using an intention to treat approach (include those that drop out)