Chapter 7: Control Techniques in Experimental Research ...
Chapter 7: Control Techniques in Experimental Research • Introduction o Differential influence: when the influence of an extraneous variable is different for the various groups o Method of difference: if groups are equivalent on every variable except for one, then that one variable is the cause of the difference between the groups • Control Techniques Carried Out at the Beginning of the Experiment o Randomization (random assignment): most important and basic of all the control methods; it’s a probabilistic control technique designed to equate experimental groups at the start of an experiment on all extraneous variables, both known and unknown Random assignment of participants to the experimental groups assures that each participant has an equal chance of being assigned to each group Random assignment produces control by virtue of the fact that all variables present in a group of participants will be distributed in approximately the same manner in all groups o Matching: using any of a variety of techniques for equating participants on one or more variables When random assignment is not possible, matching can be an effective technique to equate groups if the researcher has the information required for matching Matching variable: the extraneous variable used in matching • The strength of matching is that it ensures that participants in the different groups are equated on the matching variable(s) • The key weakness of matching is that the groups are equated only on the matching variables Matching by Holding Variables Constant • All participants in each treatment group will have the same degree or type of extraneous variable • Creates a more homogenous participant sample because only participants with a certain amount or type of the extraneous variable are included in the participant pool
• Disadvantages: (1) restricts the size of the participant population and (2) the results of the study can be generalized only to the type of participant who participated in the study Matching by Building the Extraneous Variable into the Research Design • Good technique for achieving control over the matching variable • Isolates the variation caused by the extraneous variable • Takes a factor that can operate as a confounding extraneous variable and makes if focal in the experiment as an IV • Statistical control: control of measured extraneous variables during data analysis
Matching by Yoked Control • Yoked control: controls for the possible influence of participant-controlled events Matching by Equating Participants • Attempt to eliminate the influence of the extraneous variable by creating equivalent groups of participants • Matches participants on the variable to be controlled • The number of participants must be some multiple of the number of levels of the IV • Precision control: requires the investigator to match participants in the various treatment groups on a case-by-case basis for each matching variable o Advantage: the participants in the various groups are equal on at least the matched variables; that is, the groups are equated on the matching variables, which rules out these extraneous variables as rival explanations for the relationship between the IVs and DVs o Disadvantages: (1) it’s difficult to know which matching variables should be used and which of the potential matching variables are most critical, (2) difficulty in finding matched participants increases disproportionately as the number of matching variables increases, (3)
matching limits the generality of the results of the study because matching can result in some rather unique groups of participants and (4) some variables are very difficult to use in matching • Frequency distribution control: a matching technique that matches groups of participants by equating ht overall distribution of the chosen variable o Disadvantage: the combinations of variables might be mismatched in the various groups • Control Techniques Carried Out During the Experiment o You must treat the different groups in the same way during the conduct of the experiment, except for administration of the different levels of the IV o Counterbalancing: a technique used to control for sequencing effects; applies only to repeated measures designs Sequencing effects can occur when participants participate in more than one treatment condition • Order effect: arises from the order in which the treatment conditions are administered to the participants o Changes that occur over time within a repeated measures experiment will result in order effects • Carryover effect: occurs when performance in one treatment condition is partially dependent on the treatment conditions that precede it Randomized counterbalancing: experiment is replicated for each participant using different counterbalancing sequences Intrasubject counterbalancing: used when each participant receives all levels of the IV more than one time; controls for sequencing effects by having each participant take the treatment conditions first in one order and then in the reverse order; takes place within participants • ABBA Group counterbalancing: the counterbalancing sequences are varied across two or more groups of participants (participants within each groups receive the same sequences) • Complete counterbalancing: all possible sequences of the treatment conditions are used in the experiment, and an equal proportion of research
participants are randomly assigned to each sequence; rarely used when the research has 3 or more treatment conditions • Incomplete counterbalancing: all possible sequences of treatment conditions are not enumerated o For the sequences enumerated, each treatment condition must appear an equal number of times in each ordinal position and each treatment condition must precede and be followed by every other condition an equal number of times o Differential carryover effect: a treatment condition affects participants’ performance in a later condition in one way and in another way when followed by a different condition • Control of Participant Effects o Double-blind placebo method: neither the experimenter nor the participant is aware of the treatment condition administered to the participant; the demand characteristics surrounding the administration of the treatment conditions are controlled by the double-blind placebo model o Deception: involves providing all research participants with a hypothesis that is unrelated to or different from the real research hypothesis that is unrelated to or different from the real research hypothesis o Control of Participant Interpretation Retrospective verbal report: an oral report in which the participant retrospectively recalls aspects of the experiment • Postexperimental inquiry: an interview of the participant after the experiment is over Concurrent verbal reports: a participant’s oral report of the experiment, which is obtained as the experiment is being performed • Soloman’s sacrifice groups: each group of participants is “sacrificed” by being stopped at a different point in the experiment and probed regarding the participants’ perceptions of the experiment • Concurrent probing: requires participants to report at the end of each trial the perceptions they have regarding the experiment • Think-aloud technique: requires participants to verbalize any thoughts or perceptions they have regarding the experiment while they are performing the experimental task
• Control of Experimenter Effects o Experimenter effects: unintentional biasing effects that the experimenter can have on the results of the experiment o Control of Recording Errors Errors resulting from the misrecording of data can be minimized if the person recording the data remains aware of the necessity of making careful observations to ensure the accuracy of data transcription Use multiple observers or data recorders Eliminate the human data recorder (if possible) and have responses recorded by some mechanical or electronic device o
Control of Experimenter Attribute Errors Never use one experimenter in one condition and a different experimenter in the other condition; this would make the groups different not just on the IV but also on the experimenter
o
Control of Experimenter Expectancy Error Blind technique: corresponds to the experimenter’s half of the double-blind placebo method; the experimenter knows the hypothesis but is blind as to which treatment condition the research participant is in Partial blind technique: the experimenter is kept ignorant of the condition the research participant is in for a portion of the study Automation: the technique of totally automating the experimental procedures so that no experimenter-participant interaction is required
• Disadvantages: (1) restricts the size of the participant population and (2) the results of the study can be generalized only to the type of participant who participated in the study Matching by Building the Extraneous Variable into the Research Design • Good technique for achieving control over the matching variable • Isolates the variation caused by the extraneous variable • Takes a factor that can operate as a confounding extraneous variable and makes if focal in the experiment as an IV • Statistical control: control of measured extraneous variables during data analysis
Matching by Yoked Control • Yoked control: controls for the possible influence of participant-controlled events Matching by Equating Participants • Attempt to eliminate the influence of the extraneous variable by creating equivalent groups of participants • Matches participants on the variable to be controlled • The number of participants must be some multiple of the number of levels of the IV • Precision control: requires the investigator to match participants in the various treatment groups on a case-by-case basis for each matching variable o Advantage: the participants in the various groups are equal on at least the matched variables; that is, the groups are equated on the matching variables, which rules out these extraneous variables as rival explanations for the relationship between the IVs and DVs o Disadvantages: (1) it’s difficult to know which matching variables should be used and which of the potential matching variables are most critical, (2) difficulty in finding matched participants increases disproportionately as the number of matching variables increases, (3)
matching limits the generality of the results of the study because matching can result in some rather unique groups of participants and (4) some variables are very difficult to use in matching • Frequency distribution control: a matching technique that matches groups of participants by equating ht overall distribution of the chosen variable o Disadvantage: the combinations of variables might be mismatched in the various groups • Control Techniques Carried Out During the Experiment o You must treat the different groups in the same way during the conduct of the experiment, except for administration of the different levels of the IV o Counterbalancing: a technique used to control for sequencing effects; applies only to repeated measures designs Sequencing effects can occur when participants participate in more than one treatment condition • Order effect: arises from the order in which the treatment conditions are administered to the participants o Changes that occur over time within a repeated measures experiment will result in order effects • Carryover effect: occurs when performance in one treatment condition is partially dependent on the treatment conditions that precede it Randomized counterbalancing: experiment is replicated for each participant using different counterbalancing sequences Intrasubject counterbalancing: used when each participant receives all levels of the IV more than one time; controls for sequencing effects by having each participant take the treatment conditions first in one order and then in the reverse order; takes place within participants • ABBA Group counterbalancing: the counterbalancing sequences are varied across two or more groups of participants (participants within each groups receive the same sequences) • Complete counterbalancing: all possible sequences of the treatment conditions are used in the experiment, and an equal proportion of research
participants are randomly assigned to each sequence; rarely used when the research has 3 or more treatment conditions • Incomplete counterbalancing: all possible sequences of treatment conditions are not enumerated o For the sequences enumerated, each treatment condition must appear an equal number of times in each ordinal position and each treatment condition must precede and be followed by every other condition an equal number of times o Differential carryover effect: a treatment condition affects participants’ performance in a later condition in one way and in another way when followed by a different condition • Control of Participant Effects o Double-blind placebo method: neither the experimenter nor the participant is aware of the treatment condition administered to the participant; the demand characteristics surrounding the administration of the treatment conditions are controlled by the double-blind placebo model o Deception: involves providing all research participants with a hypothesis that is unrelated to or different from the real research hypothesis that is unrelated to or different from the real research hypothesis o Control of Participant Interpretation Retrospective verbal report: an oral report in which the participant retrospectively recalls aspects of the experiment • Postexperimental inquiry: an interview of the participant after the experiment is over Concurrent verbal reports: a participant’s oral report of the experiment, which is obtained as the experiment is being performed • Soloman’s sacrifice groups: each group of participants is “sacrificed” by being stopped at a different point in the experiment and probed regarding the participants’ perceptions of the experiment • Concurrent probing: requires participants to report at the end of each trial the perceptions they have regarding the experiment • Think-aloud technique: requires participants to verbalize any thoughts or perceptions they have regarding the experiment while they are performing the experimental task
• Control of Experimenter Effects o Experimenter effects: unintentional biasing effects that the experimenter can have on the results of the experiment o Control of Recording Errors Errors resulting from the misrecording of data can be minimized if the person recording the data remains aware of the necessity of making careful observations to ensure the accuracy of data transcription Use multiple observers or data recorders Eliminate the human data recorder (if possible) and have responses recorded by some mechanical or electronic device o
Control of Experimenter Attribute Errors Never use one experimenter in one condition and a different experimenter in the other condition; this would make the groups different not just on the IV but also on the experimenter
o
Control of Experimenter Expectancy Error Blind technique: corresponds to the experimenter’s half of the double-blind placebo method; the experimenter knows the hypothesis but is blind as to which treatment condition the research participant is in Partial blind technique: the experimenter is kept ignorant of the condition the research participant is in for a portion of the study Automation: the technique of totally automating the experimental procedures so that no experimenter-participant interaction is required
