Chapter 11: Factorial Experimental Designs One Two- way Design ...
Chapter 11: Factorial Experimental Designs One Two- way Design (2x2) - 4 conditions - Two Independent Variables- Drug A or Drug B (two levels in each variable) - DV- health outcome - Drug A works regardless of Drug B, and Drug B works regardless of Drug A - Have 2 main effects plus interactions Main effect- when one variable has an effect regardless of other one - No interactions b/c effects are same Interaction - Drug a doesn’t help if don’t get drug b, but if do get drug b then drug a has effect Crossover Interaction- different/ reversed simple effects - Signifies very strong test - Drugs work together well/ taking nothing works but taking only Drug A or Drug B has neg effects Factorial Experimental Designs - More than one IV (Factor) and Factors normally crossed - More efficient and informative - Schematic Diagram >, < expected means of IV on DV are greater or less then in each condition Use different words for variables and levels - Physical appearance- attractive, unattractive - Effects of IV variable w/in one level of noise in comparison to other IV variable with other level Interaction- when moderator variable is working - In graph if lines aren’t parallel to each other then there is interaction b/c is a difference Chapter 12 Correlation between IV and DV while controlling for the other IV variables Label with operationalization rather than conceptual variable (instead of Description of Librarian: Consistent or Inconsistent) use (Traits: Librarian or Boozer) b- probability of making Type II error Factorial Experimental Designs Describe as factorial design, use more than one IV (call factors) Moderator variable changes relationship *Replication variable goes on left and moderator goes on top in schematic diagram Statistical analysis - Main effects- one for each factor, marginal means - Interaction o Line chart, as long as lines aren’t parallel then there is an interaction o Main effect, difference between the variables (compare middle values)
-
Simple effects( ^ or v in diagram, compares conditions within one level of moderator variable) ANOVA summary table 11/14/12
Explaining Schematic Diagram: expect to find that replication variable has effect in regular condition, but adding moderator variable makes effect change Start w/ explaining control condition (in absence of frustration the violent films created more aggression) but (in the presence of frustration, those who saw the nonviolent film showed more agg) 2x3 Two independent variables, six conditions Main effects compare marginal means Simple effects compare two different conditions (V or ^ in diagram) Pairwise Comparisons- any possible comparisons between 2 conditions Planned comparisons- make comparisons planned to make ahead of time, only compare planned ones Post- hoc Comparisons- make all comparisons want to make but adjust type 1 error so each test has more stringent significance test Complex Comparisons- if a lot of different comparisons, then use when have three conditions and compare all three at once CHAPTER 12 Invalidity - Construct invalidity - Statistical conclusion invalidity o Type 1 error- falsely rejected null o Type 2 error- fail to reject null when should’ve - Internal invalidity- External invalidity Experimental Example: Experimental manipulation Conceptual IV Conceptual DV Measured DV Manipulation Check - Operational DV- how long takes to pick up papers - Experimental manipulation causes IV o Comedy or Tragedy Film (manipulation) changes mood - Film Mood (mood measure) Helping pick up papers - Manipulation check always conducted after experiment (very short, usually one question) Recruit students who want to get into law school and give them what they think is copy of exam
Reduce Type 2 Errors - Increase the signal o Impact (experimental realism) o Strong manipulations o Manipulation checks - Reduce random error o Increase reliability o Standardize conditions o Limited- participant designs o Matched participant design Pepsi labeled 2 glasses M or Q, M was Pepsi and Q was coke - Experimental condition (one measure created not measured and is manipulated) o IV- type of drink, 2 levels o Within participant b/c tried both - Participants preferred glass M over Q - Can conclude that Pepsi was preferred to coke? Two variables are confounded that leaves alternative explanation open (not only are drinks different, but letters are so some people might like letter M more than Q) o Run again and switch letters, or not use letters 11/19/12 More internal validity in experiments - Does IV really cause DV - Reduce systematic error - Confounding: variables are mixed up, produces alternative explanations Alcohol study: those who drank alcohol found opposite more attractive can’t be determined because told what condition being in (placebo effect) - Alcohol and expectancy of alcohol are the variables - Dv- rated attractiveness - Research hypothesis: main effect of alcohol - Constructive replication- replicate existing finding but add new conditions to original procedure Experimenter bias - Naïve experimenters - Blind experimenters - Expectations (smart vs dumb rat) created difference in results (smart rats found to finish maze faster) Participants given series of resumes so might guess hypothesis (demand characteristics?) - Cover story- judge aesthetics, typeface, paper of resume, say that judgment of paper influenced by quality of applicant Demand Characteristics- when participant guesses research hypothesis
-
Cover story- don’t tell people what point of study is (tell them something different, Milgram study told them studying learning not obedience) Unrelated- experiments technique- primed w/ neg or pos emotions or adding filler quesitons, reduces demand Between subjects design
Chapter 13 External Validity- being able to generalize the results - Generalization across people and operational definitions Participant Selection - Limited- reduces w/in group variability - Broad- increases generalization - Using only college students reduces random error, but don’t learn about other age groups Conceptual replication- new IV and/or DVs Constructive Replication- ruling out alternative explanations, limited conditions Participant replication- cross cultural participation Chapter 14- Quasi Experiments Ex. Study was operationalized by # of papers picked up, Program Evaluation research Correlational- don’t randomly assign (ex. ask did you go on trip or not) - Lower internal validity - Sometimes is the only way Single Group Design- just look at people in program - Comparison group: Students Group 1 (participate) Group 2 (do not participate) - IV: participate in exchange program - DV- measure attitudes - Between subjects design Simple Designs - Single group design o Descriptive only o No comparisons possible - Comparison group design o Selection threat (no random assignment, assume students in program were different to begin with and program didn’t change attitude) COMPLEX Designs Single Group Before- After Design - Measure attitude before going on exchange program and then measure after - Within subjects design
-
Longitudinal (vs cross sectional) Attrition threats- not everyone stays in (only completes first variable, ex track people in therapy program but some drop out) Maturation threat- growing older causes change not the variable (something different in students at end of program but their age might cause difference not program) History- maybe other things happened that can’t control (war, president change)
Comparison Group Before- After Design - Ex. Measure attitudes before and after, one group does/ doesn’t participate in program - Between subjects design, DV: + attitude - Two variables: Participation (did or didn’t) and Time of measurement (before or after) - Hyp- people in exchange program will become more tolerant o Interaction Before
After
Participation Not participation Regression to the Mean- those who participate in the exchange program are + to begin w/ so if so positive to begin w/ then expectations not met and come back with less + attitude N=1 Designs (follow one participant) - Single Participant design Example Questions: - Hippocampus lesions: participant status is the comparison group (not experiment b/c not assigned whether to have a lesion, is correlational) o IV- lesion/ healthy, time (when measured) o DV- cortisol levels o Analyzed using ANOVA - Flight instructor- regression to the mean Main effect- effect of one IV on DV, controlling for other IV. male confederate cause more distance moved, Female pedestrians move more than males [each IV has a main effect] - Main effects means are outside cells Simple effect- difference between conditions - Effects within cells on chart Crossover interaction- one simple effect is in opposite direction of other simple effect - Cross over is when they aren’t parallel and actually cross - In chart is when arrows are in opposite directions Factors- manipulated independent variables Levels- the subsets of independent variable
Conditions- all levels combined 6 levels in 2 x 4 design, add to get levels and multiply to get conditions which is 8 External validity- extent to which findings are generalizable and applicable to population and can increase by increasing sample size, random sampling Hand Classroom setting Nature setting Right 2 2 Avg 2 Left 5 7 Avg 4 Since 2< 4, using left hand more creative than right, relationship increased as people outside will be more creative than those inside Participant Variable- splitting them into groups on something inherent to participants Comparison group- anytime comparing groups in quasi experiment Beta- probability of making Type II error Chi Square use when both variables are nominal (contingency table) 12. Alternative explanations- confounding variable (about M or Q not hypothesis coke or pepsi)
-
Simple effects( ^ or v in diagram, compares conditions within one level of moderator variable) ANOVA summary table 11/14/12
Explaining Schematic Diagram: expect to find that replication variable has effect in regular condition, but adding moderator variable makes effect change Start w/ explaining control condition (in absence of frustration the violent films created more aggression) but (in the presence of frustration, those who saw the nonviolent film showed more agg) 2x3 Two independent variables, six conditions Main effects compare marginal means Simple effects compare two different conditions (V or ^ in diagram) Pairwise Comparisons- any possible comparisons between 2 conditions Planned comparisons- make comparisons planned to make ahead of time, only compare planned ones Post- hoc Comparisons- make all comparisons want to make but adjust type 1 error so each test has more stringent significance test Complex Comparisons- if a lot of different comparisons, then use when have three conditions and compare all three at once CHAPTER 12 Invalidity - Construct invalidity - Statistical conclusion invalidity o Type 1 error- falsely rejected null o Type 2 error- fail to reject null when should’ve - Internal invalidity- External invalidity Experimental Example: Experimental manipulation Conceptual IV Conceptual DV Measured DV Manipulation Check - Operational DV- how long takes to pick up papers - Experimental manipulation causes IV o Comedy or Tragedy Film (manipulation) changes mood - Film Mood (mood measure) Helping pick up papers - Manipulation check always conducted after experiment (very short, usually one question) Recruit students who want to get into law school and give them what they think is copy of exam
Reduce Type 2 Errors - Increase the signal o Impact (experimental realism) o Strong manipulations o Manipulation checks - Reduce random error o Increase reliability o Standardize conditions o Limited- participant designs o Matched participant design Pepsi labeled 2 glasses M or Q, M was Pepsi and Q was coke - Experimental condition (one measure created not measured and is manipulated) o IV- type of drink, 2 levels o Within participant b/c tried both - Participants preferred glass M over Q - Can conclude that Pepsi was preferred to coke? Two variables are confounded that leaves alternative explanation open (not only are drinks different, but letters are so some people might like letter M more than Q) o Run again and switch letters, or not use letters 11/19/12 More internal validity in experiments - Does IV really cause DV - Reduce systematic error - Confounding: variables are mixed up, produces alternative explanations Alcohol study: those who drank alcohol found opposite more attractive can’t be determined because told what condition being in (placebo effect) - Alcohol and expectancy of alcohol are the variables - Dv- rated attractiveness - Research hypothesis: main effect of alcohol - Constructive replication- replicate existing finding but add new conditions to original procedure Experimenter bias - Naïve experimenters - Blind experimenters - Expectations (smart vs dumb rat) created difference in results (smart rats found to finish maze faster) Participants given series of resumes so might guess hypothesis (demand characteristics?) - Cover story- judge aesthetics, typeface, paper of resume, say that judgment of paper influenced by quality of applicant Demand Characteristics- when participant guesses research hypothesis
-
Cover story- don’t tell people what point of study is (tell them something different, Milgram study told them studying learning not obedience) Unrelated- experiments technique- primed w/ neg or pos emotions or adding filler quesitons, reduces demand Between subjects design
Chapter 13 External Validity- being able to generalize the results - Generalization across people and operational definitions Participant Selection - Limited- reduces w/in group variability - Broad- increases generalization - Using only college students reduces random error, but don’t learn about other age groups Conceptual replication- new IV and/or DVs Constructive Replication- ruling out alternative explanations, limited conditions Participant replication- cross cultural participation Chapter 14- Quasi Experiments Ex. Study was operationalized by # of papers picked up, Program Evaluation research Correlational- don’t randomly assign (ex. ask did you go on trip or not) - Lower internal validity - Sometimes is the only way Single Group Design- just look at people in program - Comparison group: Students Group 1 (participate) Group 2 (do not participate) - IV: participate in exchange program - DV- measure attitudes - Between subjects design Simple Designs - Single group design o Descriptive only o No comparisons possible - Comparison group design o Selection threat (no random assignment, assume students in program were different to begin with and program didn’t change attitude) COMPLEX Designs Single Group Before- After Design - Measure attitude before going on exchange program and then measure after - Within subjects design
-
Longitudinal (vs cross sectional) Attrition threats- not everyone stays in (only completes first variable, ex track people in therapy program but some drop out) Maturation threat- growing older causes change not the variable (something different in students at end of program but their age might cause difference not program) History- maybe other things happened that can’t control (war, president change)
Comparison Group Before- After Design - Ex. Measure attitudes before and after, one group does/ doesn’t participate in program - Between subjects design, DV: + attitude - Two variables: Participation (did or didn’t) and Time of measurement (before or after) - Hyp- people in exchange program will become more tolerant o Interaction Before
After
Participation Not participation Regression to the Mean- those who participate in the exchange program are + to begin w/ so if so positive to begin w/ then expectations not met and come back with less + attitude N=1 Designs (follow one participant) - Single Participant design Example Questions: - Hippocampus lesions: participant status is the comparison group (not experiment b/c not assigned whether to have a lesion, is correlational) o IV- lesion/ healthy, time (when measured) o DV- cortisol levels o Analyzed using ANOVA - Flight instructor- regression to the mean Main effect- effect of one IV on DV, controlling for other IV. male confederate cause more distance moved, Female pedestrians move more than males [each IV has a main effect] - Main effects means are outside cells Simple effect- difference between conditions - Effects within cells on chart Crossover interaction- one simple effect is in opposite direction of other simple effect - Cross over is when they aren’t parallel and actually cross - In chart is when arrows are in opposite directions Factors- manipulated independent variables Levels- the subsets of independent variable
Conditions- all levels combined 6 levels in 2 x 4 design, add to get levels and multiply to get conditions which is 8 External validity- extent to which findings are generalizable and applicable to population and can increase by increasing sample size, random sampling Hand Classroom setting Nature setting Right 2 2 Avg 2 Left 5 7 Avg 4 Since 2< 4, using left hand more creative than right, relationship increased as people outside will be more creative than those inside Participant Variable- splitting them into groups on something inherent to participants Comparison group- anytime comparing groups in quasi experiment Beta- probability of making Type II error Chi Square use when both variables are nominal (contingency table) 12. Alternative explanations- confounding variable (about M or Q not hypothesis coke or pepsi)
