PSYC202 Chapter 13: Introduction to Analysis of Variance 13.1 ...
PSYC202 Chapter 13: Introduction to Analysis of Variance 13.1 Introduction Hypothesis testing procedure to evaluate mean differences between 2 or more treatments (or populations) Similar to t tests, but allows you to work with 2 or more treatments Can be used with independent or repeated measures design Can also be used to evaluate more than one factor Factor: the variable (independent or quasiindependent) that designates the groups being compared Levels: the individual conditions or values that make up a factor are called levels of the factor (ex. three different temperature conditions = 3 levels of temperature) Twofactor or Factorial Design: a study that combines two factors (ex. 2 treatment and time) Null hypothesis states that there is no treatment effect, alternative hypothesis states that “at least one population mean is different from another” For ANOVA, the t statistic is called an F Ratio: F = Variance between sample means Variance expected with no treatment effect Calculate F for all sets of sample means. Compare sample mean variance T statistic measured how much difference reasonable to expect between sample means ANOVA measures how big differences should be if there is no treatment effect Testwise alpha level individual alpha levels selected for each hypothesis test Experimentwise alpha level total probability of a Type I error accumulated from all the separate tests in the experiment 13.2 The Logic of Analysis of Variance BetweenTreatments Variance: calculate variance for all treatment conditions (sample means) WithinTreatment Variance: a measure of variability within each treatment condition ANOVA interprets the differences between sample means as either naturally occurring and accidental (sampling error) or significant treatment effect Within treatment variance provides a measure of how much difference is reasonable to expect from random and unsystematic factors (naturally occurring differences within each group) F = Variance between treatments = MSbetween Variance within treatments MSwithin F Ratio near 1 differences are random and unsystematic. Not significant. F Ration larger than 1 (larger numerator) significant effect Denominator of F Ration is called the ‘error term’ 13.3 ANOVA Notation and Formulas k = number of treatment conditions n = number of scores in each treatment condition N = total number of scores in the entire study
T = sum of scores for each treatment condition (∑X) G = sum of all the scores in the study S2= SS df Variance (MS) Between Treatments: SS between Df between Variance (MS) Within Treatments: SS within Df within SStotal = ∑X2 G 2 N SSwithin treatments = all SS values for each treatment condition added together SSbetween treatments = SStotal SSwithin OR SSbetween = n(SSmeans) SS = ∑X2 – (∑X) 2 n SSmeans=∑ T 2 – G 2 n N SStotal must always = SSwithin + SSbetween Total degrees of freedom: dftotal = N1 Withintreatments degrees of freedom: dfwithin = ∑(n1) OR dfwithin =Nk Betweentreatments df: dfbetween = k1 dftotal must always = dfwithin + dfbetween Variance also called mean square (MS) Use ANOVA summary table to organize information/calculation data
13.4 The Distribution of FRatios F values/variance is always positive If null true, F = zero, and numerator & denominator are the same Shape of F distribution depends on df. Large df = clustered near 1.00. Smaller df = distribution more spread out (more significant) Use dfbetween & dfwithin with F Distribution table to find critical boundaries 13.5 Examples of Hypothesis Testing and Effect Size with ANOVA F tells us that a difference between means either exists or doesn’t. Does not tell us which treatments specifically differs Common to also calculate effect size using r2 = SSbetween r2 written as eta2 SStotal Tells us percentage from the treatments Standard deviations from SS of each treatment: s= √SS/(n1) ANOVA Pooled Variance MS within = SSwithin = SS1 + SS2 +SS3 … dfwithin df1 + df2 +df3 ANOVA more accurate when all treatments have equal sample sizes 13.6 Post Hoc Tests
Post Hoc Test: additional hypothesis tests that are done after an ANOVA to determine exactly which mean differences are significant and which are not Post hoc tests conducted when: You reject H0 and There are 3 or more treatments Experimentwise Alpha Level: the overall probability of a Type I error that accumulates over a series of separate hypothesis tests. Typically, the experimentwise alpha level is substantially greater than the value of alpha used for any one of the individual tests More tests = greater chance of experimentwise alpha level/type one error The Dunn Test: when comparing planned comparisons, protect against inflated experimentwise alpha level by dividing alpha equally among the comparisons. Ex. if using alpha level 0.05, then divide in half and use alpha 0.025 for each comparison. Since you’re comparing 2 tests at a time, don’t have to use ANOVA, can just use t test If comparisons unplanned (don’t predict a significant difference before test), use Tukey’s HSD test or Scheffe Test to limit the risk of a Type I Error Tukey’s Honestly Significant Diffference (HSD) Test: Allows you to calculate minimum value necessary for significant difference. If mean exceeds HSD, conclude there is a significant difference HSD = q √ MSwithin √ n n for each treatment must be constant q can be found in q table (p.734) Scheffe Test: Safest post hoc test smallest risk of Type I error. Does this by requiring higher sample mean to be significant. Uses FRatio to evaluate significance of the difference between any 2 treatment conditions Use F Ratio but numerator is only with the 2 treatments you want to compare. Denominator is the same MSwithin that was used for the overall ANOVA 13.7 The Relationship Between ANOVA and t Tests ANOVA Assumptions: Observations within each sample independent Populations from which samples selected must be normal Populations must have equal variances
T = sum of scores for each treatment condition (∑X) G = sum of all the scores in the study S2= SS df Variance (MS) Between Treatments: SS between Df between Variance (MS) Within Treatments: SS within Df within SStotal = ∑X2 G 2 N SSwithin treatments = all SS values for each treatment condition added together SSbetween treatments = SStotal SSwithin OR SSbetween = n(SSmeans) SS = ∑X2 – (∑X) 2 n SSmeans=∑ T 2 – G 2 n N SStotal must always = SSwithin + SSbetween Total degrees of freedom: dftotal = N1 Withintreatments degrees of freedom: dfwithin = ∑(n1) OR dfwithin =Nk Betweentreatments df: dfbetween = k1 dftotal must always = dfwithin + dfbetween Variance also called mean square (MS) Use ANOVA summary table to organize information/calculation data
13.4 The Distribution of FRatios F values/variance is always positive If null true, F = zero, and numerator & denominator are the same Shape of F distribution depends on df. Large df = clustered near 1.00. Smaller df = distribution more spread out (more significant) Use dfbetween & dfwithin with F Distribution table to find critical boundaries 13.5 Examples of Hypothesis Testing and Effect Size with ANOVA F tells us that a difference between means either exists or doesn’t. Does not tell us which treatments specifically differs Common to also calculate effect size using r2 = SSbetween r2 written as eta2 SStotal Tells us percentage from the treatments Standard deviations from SS of each treatment: s= √SS/(n1) ANOVA Pooled Variance MS within = SSwithin = SS1 + SS2 +SS3 … dfwithin df1 + df2 +df3 ANOVA more accurate when all treatments have equal sample sizes 13.6 Post Hoc Tests
Post Hoc Test: additional hypothesis tests that are done after an ANOVA to determine exactly which mean differences are significant and which are not Post hoc tests conducted when: You reject H0 and There are 3 or more treatments Experimentwise Alpha Level: the overall probability of a Type I error that accumulates over a series of separate hypothesis tests. Typically, the experimentwise alpha level is substantially greater than the value of alpha used for any one of the individual tests More tests = greater chance of experimentwise alpha level/type one error The Dunn Test: when comparing planned comparisons, protect against inflated experimentwise alpha level by dividing alpha equally among the comparisons. Ex. if using alpha level 0.05, then divide in half and use alpha 0.025 for each comparison. Since you’re comparing 2 tests at a time, don’t have to use ANOVA, can just use t test If comparisons unplanned (don’t predict a significant difference before test), use Tukey’s HSD test or Scheffe Test to limit the risk of a Type I Error Tukey’s Honestly Significant Diffference (HSD) Test: Allows you to calculate minimum value necessary for significant difference. If mean exceeds HSD, conclude there is a significant difference HSD = q √ MSwithin √ n n for each treatment must be constant q can be found in q table (p.734) Scheffe Test: Safest post hoc test smallest risk of Type I error. Does this by requiring higher sample mean to be significant. Uses FRatio to evaluate significance of the difference between any 2 treatment conditions Use F Ratio but numerator is only with the 2 treatments you want to compare. Denominator is the same MSwithin that was used for the overall ANOVA 13.7 The Relationship Between ANOVA and t Tests ANOVA Assumptions: Observations within each sample independent Populations from which samples selected must be normal Populations must have equal variances
