Chapter XI: Factorial Designs
Chapter XI: Factorial Designs 11.1 •
Introduction To Factorial Designs In most research situations, the goal is to examine the relationship between two variables by isolating those variables within the research study. • The idea is to eliminate or reduce the influence of any outside variables that may disguise or obscure the specific relationship under navigation. • An experimental research typically focuses on the independent variable (which is expected to influence behavior) and one dependent variable (which is a measure of the behavior). • The non-experimental and quasi-experimental designs usually investigate the relationship between one quasi-independent variable and one dependent variable. • Behavior usually is influenced by a variety of different variables acting and interacting simultaneously. • Researchers often design research studies that include more than one independent variable (or quasi-independent variable). • Recall, that in non-experimental and quasi-experimental research. The variable that differentiates the groups of participants or the groups of scores is called the quasiindependent variable. • When two or more independent variables are combined in a single study, the independent variables are commonly called factors. • A research study involving two or more factors is called a factorial design. • A research study with only one independent variable is often called a single-factor design. • Factorial designs use a notation system that identifies both the number of factors and the number of values or levels that exist for each factor. • A 2 × 2 factorial design is considered the simplest factorial design. • A 2 × 2 factorial design represents a two-factor design with two levels of the first factor and two levels of the second, with a total of four treatment conditions. • A 2 × 3 × 2 factorial design would represent a three-factor design with two, three, and two levels of each of the factors, respectively, for a total of 12 conditions. In an experiment, an independent variable is often called a factor, especially in experiments that include two or more independent variables. A research design that includes two or more factors is called a factorial design. • One advantage of a factorial design is that it creates a more “realistic” situation than can be obtained by examining a single factor in isolation. • Behavior is influenced by a variety of factors usually acting together. • It is sensible to examine two or more factors simultaneously in a single study. • Combining two or more factors within one study provides researchers with a unique opportunity to examine how the factors influence behavior and how they influence or interact with each other. • Combing two variables permits researchers with a unique opportunity to examine how the factors influence behavior and how they influence or interact with each other. • The idea that two factors can act together, creating unique conditions that are different from either factor acting alone, underlies the value of a factorial design.
11.2 • •
Main Effects & Interactions The primary advantage of a factorial design is that it allows researchers to examine how unique combinations of factors acting together influence behavior. The structure of a two factor design can be represented by a matrix in which the levels of one factor determine the columns.
• •
The levels of the second factor determine the rows. Each cell in the matrix corresponds to a specific combination of the factors; that is, a separate treatment. • The research study would involve observing and measuring a group of individuals under the conditions described by each cell. • The data from a two factor study provides three separate and distinct sets of information describing how the two factors independently and jointly affect behavior. • The difference among three column means is called the main effect. • For reaction time, smaller numbers indicate faster times. • The differences between the column means define the main effect for one factor, and the differences between the row means define the main effect for the second factor. • A factorial design allows researchers to examine how combinations of factors working together affect behavior. • In some situations, the effects of one factor simply add on to the effects of the other factor. • Although the two factors are being applied simultaneously, the result is the same as if one factor contributed its effect, and then the second factor followed and added its effect. • The two factors are operating independently and neither factor has a direct influence on the other. • One factor will have a direct influence on the effect of a second factor, producing an interaction between factors. • When two factors interact, their combined effect is different from the simple sum of the two effects separately. The main differences among the levels of one factor are called the main effect of that factor. When the research study is represented as a matrix with one factor defining the rows and the second factor defining the columns, then the mean differences among the rows define the main effect for one factor, and the mean differences among the columns define the main effect for the second factor. Note that a two-factor study has two main effects; one for each of the two factors. An interaction between factors (or simply an interaction) occurs whenever one factor modifies the effects of a second factor. If the effects of one factor simply add onto the effects of another factor, then the two factors are independent there is no interaction. Identifying Interactions • To identify an interaction in a factorial study, you must compare the mean differences between cells with the mean differences predicted from the main affects. • If there is no interaction, the main effects will simply add together and completely explain the mean differences between cells. • An interaction between factors will produce mean differences between cells that cannot be explained by the main effects. • Results from a two-factor design reveal how each factor independently affects behavior (the main effects) and how the two factors operating together (the interaction) can affect behavior. • When data from a two-factor study are organized in a matrix, the mean differences between the columns describe the main effect for one factor and the mean differences between rows describe the main effect for the second factor. • The main effects reflect the results that would be obtained if each factor were examined in its own separate experiment. • The extra mean differences that exist between cells in the matrix (differences that are not explained by the overall main effects) describe the interaction and represent the unique information that is obtained by combining the two factors in a single study. 11.3
More About Interactions
•
Interaction focuses on the notion of interdependency between the factors, as opposed to independence. • More specifically, if the two factors are independent so that the effect of one is not influenced by the other, then there is no interaction. • If the two factors are interdependent so that one factor does influence the effects of the other, then there is an interaction. • The notion of interdependence is consistent with our earlier discussion of interactions; if one factors does influence the effects of the other, then unique combinations of the factors produce unique effects. When the effects of one factor depend on the different levels of a second factor, then there is an interaction between the factors. • When the effects of a factor vary depending on the levels of another factor, the two factors are combing to produce unique effects. • When the results of a two-factor study are presented in a graph, the concept of interaction can be defined in terms of the pattern displayed in the graph. When the results of a two-factor study are graphed, the existence of nonparallel lines (lines that cross or converge) is an indication of an interaction between the two factors. Note that a statistical test is needed to determine whether or not the interaction is significant. Interpreting Main Effects & Interactions • The mean differences between columns and between rows describe the main effects in a two-factor study, and the extra mean differences between cells describe the interaction. • Mean differences are simply descriptive and must be evaluated by a statistical hypothesis test before they can be considered significant. • Obtained mean differences may not represent a real treatment effect but rather simply be due to chance or error. • Until the data are evaluated by a hypothesis test, be cautious about interpreting any results from a two-factor study. • When a statistical analysis does indicate significant effects, you must still be careful about interpreting the outcome. • The main effect for one factor is obtained by averaging all the different levels of the second factor. • Each main effect is an average; it may not accurately represent any of the individual effects that were used to compute the average. • The presence of an interaction can obscure or distort the main effects of either factor. • Whenever a statistical analysis produces a significant interaction, you should take a close look at the data before giving any credibility to the main effects. Independence of Main Effects & Interactions • The two factor study allows researchers to evaluate three separate sets of mean differences: (1) the mean differences from the main effect of factor A, (2) the mean differences from the main effect of factor B, and (3) the mean differences from the interaction between factors. • The three sets of mean differences are separate and completely independent. • It is possible for the results from a two-factor study to show any possible combination of main effects and interaction. • Extra mean differences within rows and columns cannot be explained by overall main effects and, therefore, indicate an interaction. 11.4 •
Types of Factorial Designs It is possible to have a separate group for each of the individual cells (a between subjects design), it is also possible to have the same group of individuals participate in all of the different cells (a within-subjects design).
• • • • • • •
It is possible to construct a factorial design where the factors are not manipulated but rather are quasi-independent variables. A factorial design can use any combination of factors. A factorial study can combine elements of experimental and non-experimental research strategies, and it can combine elements of between-subjects and within-subjects designs within a single research study. A two-factor design may include one between-subjects factor (with a separate group for each level of the factor) and one within-subjects factor (with each group measured in several different treatment conditions). The same study could also include one experimental factor (with a manipulated independent variable) and one non-experimental factor (with a pre-existing, nonmanipulated variable). The ability to mix designs within a single research study provides researchers with the potential to blend several different research strategies within one study. This potential allows researchers to develop studies that address scientific questions that could not be answered by any single strategy.
Between-Subjects & Within-Subjects Designs • It is possible to construct a factorial study that is purely a between-subjects design; that is, in which there is a separate group of participants for each of the treatment conditions. • A particular disadvantage for a factorial study is that a between-subjects design can require a large number of participants. • Another disadvantage of between-subjects designs is that individual differences (characteristics that differ from one participant to another) can become confounding variables and increase the variance of the scores. • A between-subjects design completely avoids any problem from order effects because each score is completely independent of every other score. • Between-subjects designs are best suited to situations where a lot of participants are available, where individual differences are relatively small, and where order effects are likely. • It is possible to construct a factorial study that is purely a within-subjects design. • In this case, a single group of individuals participates in all of the separate treatment conditions. • A particular disadvantage for a factorial study is the number of different treatment conditions that each participant must undergo. • The large number of different treatments can be very time consuming, which increase the chances that participants will quit and walk away before the study is ended (attrition) • Having each participant undergo a long series of treatment conditions can increase the potential for testing effects (such as fatigue or practice effects) and make it more difficult to counterbalance the design to control for order effects. • Within subjects design requires only one group of participants. • Within subjects design eliminates or greatly reduces the problems associated with individual differences. • Within-subjects designs are best suited for individual differences are relatively large, and where there is little reason to expect order effects to be large and disruptive. Mixed-Subjects: Within-and-Between Subjects Designs • A mixed design – one between-subjects factor and one within-subjects factor. • If the design is pictured as a matrix with one factor defining the rows and the second factor defining the columns, then the mixed design has a separate group for each row with each group participating in all the different columns.
A factorial study that combines two different research designs is called a mixed design. A common example of a mixed design is a factorial study with one between-subjects factor and one within-subjects factor. • A mixed design – one between-subjects factor and one within-subjects factor. Experimental & Non-Experimental or Quasi-Experimental Research Strategies • To construct a factorial study that is purely an experimental research design, both factors are to be considered true independent variables that are manipulated by the researcher. • It is also possible to construct a factorial study for which all the factors are nonmanipulated, quasi-independent variables. Combined Strategies: Experimental & Quasi-Experimental or Non-Experimental • In the behavioral sciences, it is common for a factorial design to mix an experimental strategy for one factor and a quasi-experimental or non-experimental strategy for another factor. • This type of study is an example of a combined strategy. • Typically, this kind of study involves one factor that is a true independent variable consisting of a set of manipulated treatment conditions, and a second factor that is a quasi-independent variable. • Quasi-Independent variable can fall into one of the following categories: (I) The second factor is a pre-existing participant characteristic such as age or gender. (II) The second factor is time. A combined strategy study uses two different research strategies in the same factorial design. One factor is a true independent variable (experimental strategy) and one factor is a quasi-independent variable (non-experimental or quasi-experimental strategy). Pretest-Posttest Control Group Designs • Pretest-Posttest non-equivalent group design: This design involves two separate groups of participants. • One group – the treatment group- is measured before and after receiving a treatment. • A second group – the control group – also is measured twice (pretest and posttest) but does not receive any treatment between the two measurements. Higher-Order Factorial Designs • The basic concepts of a two-factor research design can be extended to more complex designs involving three or more factors; such designs are referred to as higher-order factorial designs. • A three-factor evaluates main effects for each of three factors, as well as a set of two-way interactions. • The logic for defining and interpreting higher-order interactions follows the pattern set by two-way interactions. • A two-way interaction such as A × B indicates that the effect of factor A depends on the levels of factor B. • Extending this definition, a three way interaction such as A × B × C indicates that the twoway interaction between A and B depends on the levels of factor C. 11.5 • • •
Applications of Factorial Designs Factorial designs provide researchers with a tremendous degree of flexibility and freedom for constructing research studies. The primary advantage of factorial studies is that they allow researchers to observe the influence of two (or more) variables acting and interacting simultaneously. Factorial designs have an almost unlimited range of potential applications.
Adding A Second Factor To A Previous Study • Factorial designs are developed when researchers plan studies that are intended to build on previous research results. • Current research tends to build on past research; factorial designs are fairly common and very useful. • In a single study, a researcher can replicate and expand previous research. • The replication involves repeating the previous study by using the same factor or independent variable exactly as it was used in the earlier study. • The expansion involves adding a second factor in the form of new conditions or new participant characteristics to determine whether or not the previously reported effects can be generalized to new situations or new populations. • The effectiveness of a study method depends on the type of test. • Creating a new research study by adding a second factor to an existing study – The new two-factor study replicates the original study, then extends the study by introducing a set of conditions. The question is then whether or not the original treatments will have the same effects under the new conditions. Reducing Variance In Between-Subjects Designs • Differences between participants can result in large variance for the scores within a treatment condition. • Large variance can make it difficult to establish any significant differences between treatment conditions. • Often a researcher has reason to suspect that a specific participant characteristic such as age is major factor contributing to the variance of the scores. • By reducing variance it may also produce homogenous groups with less variance, but it will also limit the researcher’s ability to generalize the results. • Limiting generalization reduces the external validity of the study. • There is a relatively simple solution to this dilemma that allows the researcher to reduce variance within groups without sacrificing external validity. • The solution involves using the specific variable as a second factor, thereby creating a two-factor study. Evaluating Order Effects In Within-Subjects Designs • Order effects can be a serious problem for within-subjects research studies. • Specifically, in a within-subjects design, each participant goes through a series of treatment conditions in a particular order. • It is possible that treatments that occur early in the order may influence a participant’s scores for treatments that occur later in the order. • Order effects can alter and distort the true effects of a treatment condition, they are generally considered a confounding variable that should be eliminated from the study. • It is possible to create a research design that actually measures the order effects and separates them from the rest of the data. Using Order of Treatments as a Second Factor • To measure and evaluate order effects, it is necessary to use counterbalancing. • Counterbalancing requires separate groups of participants with each group going through the set of treatments in a different order. • By using the order of treatments as a second factor, it is possible to evaluate any order effects that exist in the data. • There are three possible outcomes that can occur, and each produces its own pattern of results. (I) No order effects. When there are no order effects, it does not matter if a treatment is presented first or second. Similarly, the order of presentation has no
• • •
effect on the mean for treatment B. Thus, the treatment effect does not depend on the order of treatments. You should recognize this pattern as an example of data with no interaction. Where there are no order effects, the data show a pattern with no interaction. It makes no difference whether a treatment is presented first or second; the mean is the same in either case. (II) Symmetrical order effects. When order effects exist, the scores in the second treatment are influenced by participation in the first treatment. The effect of one factor depends on the other factor. You should recognize this as an example of interaction. When order effects exist, they show up in the two-factor analysis as an interaction between treatments and the order of treatments. (III) Asymmetrical order effects. The existence of an asymmetrical interaction is an indication that order effects exist. In general, asymmetrical order effects produce a lopsided or asymmetrical interaction between treatments and orders. In an actual experiment, a researcher cannot see order effects. Using order of treatments as a second factor makes it possible to examine any order effects that exist in a set of data; their magnitude and nature are revealed in the interaction. Researchers can observe the order effects in their data and separate them from the effects of the different treatments.
Glossary Chapter 11 Combined strategy
A factorial study that combines two different research strategies such as experimental and non-experimental or quasi-experimental in the same factorial design.
Factor
A variable that differentiates a set of groups or conditions being compared in a research study. In an experimental design, a factor is an independent variable.
Factorial design
A research design that includes two or more factors.
Higherorder factorial design
A factorial research design with more than two factors.
Interaction
See interaction between factors.
Interaction between factors
In a factorial design, the mean differences between individual treatment conditions or cells that are not consistent with the main effects. Also, when the effects of one factor depend on the different levels of a second factor. Indicated by the existence of nonparallel (converging or crossing) lines in a graph showing the means for a two-factor design. Also known as interaction.
Main effect In a factorial study, the mean differences among the levels of one factor. Mixed design
A factorial study that combines two different research designs such as between-subjects and within-subjects in the same factorial design.
Introduction To Factorial Designs In most research situations, the goal is to examine the relationship between two variables by isolating those variables within the research study. • The idea is to eliminate or reduce the influence of any outside variables that may disguise or obscure the specific relationship under navigation. • An experimental research typically focuses on the independent variable (which is expected to influence behavior) and one dependent variable (which is a measure of the behavior). • The non-experimental and quasi-experimental designs usually investigate the relationship between one quasi-independent variable and one dependent variable. • Behavior usually is influenced by a variety of different variables acting and interacting simultaneously. • Researchers often design research studies that include more than one independent variable (or quasi-independent variable). • Recall, that in non-experimental and quasi-experimental research. The variable that differentiates the groups of participants or the groups of scores is called the quasiindependent variable. • When two or more independent variables are combined in a single study, the independent variables are commonly called factors. • A research study involving two or more factors is called a factorial design. • A research study with only one independent variable is often called a single-factor design. • Factorial designs use a notation system that identifies both the number of factors and the number of values or levels that exist for each factor. • A 2 × 2 factorial design is considered the simplest factorial design. • A 2 × 2 factorial design represents a two-factor design with two levels of the first factor and two levels of the second, with a total of four treatment conditions. • A 2 × 3 × 2 factorial design would represent a three-factor design with two, three, and two levels of each of the factors, respectively, for a total of 12 conditions. In an experiment, an independent variable is often called a factor, especially in experiments that include two or more independent variables. A research design that includes two or more factors is called a factorial design. • One advantage of a factorial design is that it creates a more “realistic” situation than can be obtained by examining a single factor in isolation. • Behavior is influenced by a variety of factors usually acting together. • It is sensible to examine two or more factors simultaneously in a single study. • Combining two or more factors within one study provides researchers with a unique opportunity to examine how the factors influence behavior and how they influence or interact with each other. • Combing two variables permits researchers with a unique opportunity to examine how the factors influence behavior and how they influence or interact with each other. • The idea that two factors can act together, creating unique conditions that are different from either factor acting alone, underlies the value of a factorial design.
11.2 • •
Main Effects & Interactions The primary advantage of a factorial design is that it allows researchers to examine how unique combinations of factors acting together influence behavior. The structure of a two factor design can be represented by a matrix in which the levels of one factor determine the columns.
• •
The levels of the second factor determine the rows. Each cell in the matrix corresponds to a specific combination of the factors; that is, a separate treatment. • The research study would involve observing and measuring a group of individuals under the conditions described by each cell. • The data from a two factor study provides three separate and distinct sets of information describing how the two factors independently and jointly affect behavior. • The difference among three column means is called the main effect. • For reaction time, smaller numbers indicate faster times. • The differences between the column means define the main effect for one factor, and the differences between the row means define the main effect for the second factor. • A factorial design allows researchers to examine how combinations of factors working together affect behavior. • In some situations, the effects of one factor simply add on to the effects of the other factor. • Although the two factors are being applied simultaneously, the result is the same as if one factor contributed its effect, and then the second factor followed and added its effect. • The two factors are operating independently and neither factor has a direct influence on the other. • One factor will have a direct influence on the effect of a second factor, producing an interaction between factors. • When two factors interact, their combined effect is different from the simple sum of the two effects separately. The main differences among the levels of one factor are called the main effect of that factor. When the research study is represented as a matrix with one factor defining the rows and the second factor defining the columns, then the mean differences among the rows define the main effect for one factor, and the mean differences among the columns define the main effect for the second factor. Note that a two-factor study has two main effects; one for each of the two factors. An interaction between factors (or simply an interaction) occurs whenever one factor modifies the effects of a second factor. If the effects of one factor simply add onto the effects of another factor, then the two factors are independent there is no interaction. Identifying Interactions • To identify an interaction in a factorial study, you must compare the mean differences between cells with the mean differences predicted from the main affects. • If there is no interaction, the main effects will simply add together and completely explain the mean differences between cells. • An interaction between factors will produce mean differences between cells that cannot be explained by the main effects. • Results from a two-factor design reveal how each factor independently affects behavior (the main effects) and how the two factors operating together (the interaction) can affect behavior. • When data from a two-factor study are organized in a matrix, the mean differences between the columns describe the main effect for one factor and the mean differences between rows describe the main effect for the second factor. • The main effects reflect the results that would be obtained if each factor were examined in its own separate experiment. • The extra mean differences that exist between cells in the matrix (differences that are not explained by the overall main effects) describe the interaction and represent the unique information that is obtained by combining the two factors in a single study. 11.3
More About Interactions
•
Interaction focuses on the notion of interdependency between the factors, as opposed to independence. • More specifically, if the two factors are independent so that the effect of one is not influenced by the other, then there is no interaction. • If the two factors are interdependent so that one factor does influence the effects of the other, then there is an interaction. • The notion of interdependence is consistent with our earlier discussion of interactions; if one factors does influence the effects of the other, then unique combinations of the factors produce unique effects. When the effects of one factor depend on the different levels of a second factor, then there is an interaction between the factors. • When the effects of a factor vary depending on the levels of another factor, the two factors are combing to produce unique effects. • When the results of a two-factor study are presented in a graph, the concept of interaction can be defined in terms of the pattern displayed in the graph. When the results of a two-factor study are graphed, the existence of nonparallel lines (lines that cross or converge) is an indication of an interaction between the two factors. Note that a statistical test is needed to determine whether or not the interaction is significant. Interpreting Main Effects & Interactions • The mean differences between columns and between rows describe the main effects in a two-factor study, and the extra mean differences between cells describe the interaction. • Mean differences are simply descriptive and must be evaluated by a statistical hypothesis test before they can be considered significant. • Obtained mean differences may not represent a real treatment effect but rather simply be due to chance or error. • Until the data are evaluated by a hypothesis test, be cautious about interpreting any results from a two-factor study. • When a statistical analysis does indicate significant effects, you must still be careful about interpreting the outcome. • The main effect for one factor is obtained by averaging all the different levels of the second factor. • Each main effect is an average; it may not accurately represent any of the individual effects that were used to compute the average. • The presence of an interaction can obscure or distort the main effects of either factor. • Whenever a statistical analysis produces a significant interaction, you should take a close look at the data before giving any credibility to the main effects. Independence of Main Effects & Interactions • The two factor study allows researchers to evaluate three separate sets of mean differences: (1) the mean differences from the main effect of factor A, (2) the mean differences from the main effect of factor B, and (3) the mean differences from the interaction between factors. • The three sets of mean differences are separate and completely independent. • It is possible for the results from a two-factor study to show any possible combination of main effects and interaction. • Extra mean differences within rows and columns cannot be explained by overall main effects and, therefore, indicate an interaction. 11.4 •
Types of Factorial Designs It is possible to have a separate group for each of the individual cells (a between subjects design), it is also possible to have the same group of individuals participate in all of the different cells (a within-subjects design).
• • • • • • •
It is possible to construct a factorial design where the factors are not manipulated but rather are quasi-independent variables. A factorial design can use any combination of factors. A factorial study can combine elements of experimental and non-experimental research strategies, and it can combine elements of between-subjects and within-subjects designs within a single research study. A two-factor design may include one between-subjects factor (with a separate group for each level of the factor) and one within-subjects factor (with each group measured in several different treatment conditions). The same study could also include one experimental factor (with a manipulated independent variable) and one non-experimental factor (with a pre-existing, nonmanipulated variable). The ability to mix designs within a single research study provides researchers with the potential to blend several different research strategies within one study. This potential allows researchers to develop studies that address scientific questions that could not be answered by any single strategy.
Between-Subjects & Within-Subjects Designs • It is possible to construct a factorial study that is purely a between-subjects design; that is, in which there is a separate group of participants for each of the treatment conditions. • A particular disadvantage for a factorial study is that a between-subjects design can require a large number of participants. • Another disadvantage of between-subjects designs is that individual differences (characteristics that differ from one participant to another) can become confounding variables and increase the variance of the scores. • A between-subjects design completely avoids any problem from order effects because each score is completely independent of every other score. • Between-subjects designs are best suited to situations where a lot of participants are available, where individual differences are relatively small, and where order effects are likely. • It is possible to construct a factorial study that is purely a within-subjects design. • In this case, a single group of individuals participates in all of the separate treatment conditions. • A particular disadvantage for a factorial study is the number of different treatment conditions that each participant must undergo. • The large number of different treatments can be very time consuming, which increase the chances that participants will quit and walk away before the study is ended (attrition) • Having each participant undergo a long series of treatment conditions can increase the potential for testing effects (such as fatigue or practice effects) and make it more difficult to counterbalance the design to control for order effects. • Within subjects design requires only one group of participants. • Within subjects design eliminates or greatly reduces the problems associated with individual differences. • Within-subjects designs are best suited for individual differences are relatively large, and where there is little reason to expect order effects to be large and disruptive. Mixed-Subjects: Within-and-Between Subjects Designs • A mixed design – one between-subjects factor and one within-subjects factor. • If the design is pictured as a matrix with one factor defining the rows and the second factor defining the columns, then the mixed design has a separate group for each row with each group participating in all the different columns.
A factorial study that combines two different research designs is called a mixed design. A common example of a mixed design is a factorial study with one between-subjects factor and one within-subjects factor. • A mixed design – one between-subjects factor and one within-subjects factor. Experimental & Non-Experimental or Quasi-Experimental Research Strategies • To construct a factorial study that is purely an experimental research design, both factors are to be considered true independent variables that are manipulated by the researcher. • It is also possible to construct a factorial study for which all the factors are nonmanipulated, quasi-independent variables. Combined Strategies: Experimental & Quasi-Experimental or Non-Experimental • In the behavioral sciences, it is common for a factorial design to mix an experimental strategy for one factor and a quasi-experimental or non-experimental strategy for another factor. • This type of study is an example of a combined strategy. • Typically, this kind of study involves one factor that is a true independent variable consisting of a set of manipulated treatment conditions, and a second factor that is a quasi-independent variable. • Quasi-Independent variable can fall into one of the following categories: (I) The second factor is a pre-existing participant characteristic such as age or gender. (II) The second factor is time. A combined strategy study uses two different research strategies in the same factorial design. One factor is a true independent variable (experimental strategy) and one factor is a quasi-independent variable (non-experimental or quasi-experimental strategy). Pretest-Posttest Control Group Designs • Pretest-Posttest non-equivalent group design: This design involves two separate groups of participants. • One group – the treatment group- is measured before and after receiving a treatment. • A second group – the control group – also is measured twice (pretest and posttest) but does not receive any treatment between the two measurements. Higher-Order Factorial Designs • The basic concepts of a two-factor research design can be extended to more complex designs involving three or more factors; such designs are referred to as higher-order factorial designs. • A three-factor evaluates main effects for each of three factors, as well as a set of two-way interactions. • The logic for defining and interpreting higher-order interactions follows the pattern set by two-way interactions. • A two-way interaction such as A × B indicates that the effect of factor A depends on the levels of factor B. • Extending this definition, a three way interaction such as A × B × C indicates that the twoway interaction between A and B depends on the levels of factor C. 11.5 • • •
Applications of Factorial Designs Factorial designs provide researchers with a tremendous degree of flexibility and freedom for constructing research studies. The primary advantage of factorial studies is that they allow researchers to observe the influence of two (or more) variables acting and interacting simultaneously. Factorial designs have an almost unlimited range of potential applications.
Adding A Second Factor To A Previous Study • Factorial designs are developed when researchers plan studies that are intended to build on previous research results. • Current research tends to build on past research; factorial designs are fairly common and very useful. • In a single study, a researcher can replicate and expand previous research. • The replication involves repeating the previous study by using the same factor or independent variable exactly as it was used in the earlier study. • The expansion involves adding a second factor in the form of new conditions or new participant characteristics to determine whether or not the previously reported effects can be generalized to new situations or new populations. • The effectiveness of a study method depends on the type of test. • Creating a new research study by adding a second factor to an existing study – The new two-factor study replicates the original study, then extends the study by introducing a set of conditions. The question is then whether or not the original treatments will have the same effects under the new conditions. Reducing Variance In Between-Subjects Designs • Differences between participants can result in large variance for the scores within a treatment condition. • Large variance can make it difficult to establish any significant differences between treatment conditions. • Often a researcher has reason to suspect that a specific participant characteristic such as age is major factor contributing to the variance of the scores. • By reducing variance it may also produce homogenous groups with less variance, but it will also limit the researcher’s ability to generalize the results. • Limiting generalization reduces the external validity of the study. • There is a relatively simple solution to this dilemma that allows the researcher to reduce variance within groups without sacrificing external validity. • The solution involves using the specific variable as a second factor, thereby creating a two-factor study. Evaluating Order Effects In Within-Subjects Designs • Order effects can be a serious problem for within-subjects research studies. • Specifically, in a within-subjects design, each participant goes through a series of treatment conditions in a particular order. • It is possible that treatments that occur early in the order may influence a participant’s scores for treatments that occur later in the order. • Order effects can alter and distort the true effects of a treatment condition, they are generally considered a confounding variable that should be eliminated from the study. • It is possible to create a research design that actually measures the order effects and separates them from the rest of the data. Using Order of Treatments as a Second Factor • To measure and evaluate order effects, it is necessary to use counterbalancing. • Counterbalancing requires separate groups of participants with each group going through the set of treatments in a different order. • By using the order of treatments as a second factor, it is possible to evaluate any order effects that exist in the data. • There are three possible outcomes that can occur, and each produces its own pattern of results. (I) No order effects. When there are no order effects, it does not matter if a treatment is presented first or second. Similarly, the order of presentation has no
• • •
effect on the mean for treatment B. Thus, the treatment effect does not depend on the order of treatments. You should recognize this pattern as an example of data with no interaction. Where there are no order effects, the data show a pattern with no interaction. It makes no difference whether a treatment is presented first or second; the mean is the same in either case. (II) Symmetrical order effects. When order effects exist, the scores in the second treatment are influenced by participation in the first treatment. The effect of one factor depends on the other factor. You should recognize this as an example of interaction. When order effects exist, they show up in the two-factor analysis as an interaction between treatments and the order of treatments. (III) Asymmetrical order effects. The existence of an asymmetrical interaction is an indication that order effects exist. In general, asymmetrical order effects produce a lopsided or asymmetrical interaction between treatments and orders. In an actual experiment, a researcher cannot see order effects. Using order of treatments as a second factor makes it possible to examine any order effects that exist in a set of data; their magnitude and nature are revealed in the interaction. Researchers can observe the order effects in their data and separate them from the effects of the different treatments.
Glossary Chapter 11 Combined strategy
A factorial study that combines two different research strategies such as experimental and non-experimental or quasi-experimental in the same factorial design.
Factor
A variable that differentiates a set of groups or conditions being compared in a research study. In an experimental design, a factor is an independent variable.
Factorial design
A research design that includes two or more factors.
Higherorder factorial design
A factorial research design with more than two factors.
Interaction
See interaction between factors.
Interaction between factors
In a factorial design, the mean differences between individual treatment conditions or cells that are not consistent with the main effects. Also, when the effects of one factor depend on the different levels of a second factor. Indicated by the existence of nonparallel (converging or crossing) lines in a graph showing the means for a two-factor design. Also known as interaction.
Main effect In a factorial study, the mean differences among the levels of one factor. Mixed design
A factorial study that combines two different research designs such as between-subjects and within-subjects in the same factorial design.
