Within Subjects ANOVAs
ANOVA stands for Analysis of Variance. It is used to determine whether there is a significant difference between the means of three or more groups. It is an omnibus test, which means that it cannot tell you where the significant difference lies, such as whether group A is significantly different from group B or group C. You determine which groups were significantly different from each other by carrying out post-hoc tests, which will also be covered here.
There are three types:
A between-subjects/independent samples ANOVA
A within-subjects/repeated measures ANOVA
A mixed ANOVA
We will be looking at the second type.
The dependent variable (DV) should be scale data (either interval or ratio)
The independent variable (IV) should consist of two categorical groups
There should be no obvious outliers
The data should be roughly normally distributed for each group of the independent variable
Within-Subjects One-Way ANOVA(example)
In the Repeated Measures Define Factor(s) dialogue box, type the name of the IV into the Within-Subject Factor Name box and enter the number of levels into the Number of Levels box. Then click Add, followed by Define
Move each of the levels of the IV on the left hand side of the new dialogue box called Repeated Measures into the Within-Subjects Variable (Interval) section on the right using the arrow. This will replace the _?_(1), _?_(2), and _?_(3).
Click on Options... and move the IV (Interval) into the Display Means for: box. Below this box, select Compare main effects, and in the scroll menu below, select Bonferroni. In the bottom section of the dialogue box, click Descriptive statistics, Estimates of effect size, and Observed power, (and Parameter estimates if you want).
Click "Continue" and then "OK" to get the output.
Step 4: Look at the Output
The means and the upper and lower bounds in the "Estimates" table can be used in the write-up of the descriptive statistics and the Confidence Intervals. The "Descriptives Statistics" table can be used to help you calculate the effecti sizes.
For the inferential statistics, the main tables you need to look at are the "Mauchly's Test of Sphericity" table, "Test of Within-Subjects Effects" table, and "Pairwise Comparisons" table.
Mauchly's test of sphericity tells you whether the variances of the differences between the conditions are equal. The "Mauchly's Test of Sphericity" table displays this. If Mauchly's test statistic is significant, it means that that there are significant differences, and that the assumption of sphericity has not been met. In this case, you read from the Greenhouse-Geisser row in the "Tests of Within-Subjects Effects" table when reporting the results of the ANOVA. If Machly's test statistical is not significant, you read from the first row of the "Tests of Within-Subjects Effects" table. In this example, Mauchly's test statistic is significant (< 0.05), meaning that we need to read from the Greenhouse-Geisser row when reporting the outcome of the ANOVA.
Finally, the "Pairwise Comparisons" table displays the results of the post-hoc tests, telling us which groups differed from each other. In this case, there are significant differences between all the interval pairs.
Step 5: Calculate the Effect Sizes
The effect size of the ANOVA will be calulcated for you (Partial Eta Squared), but the effect sizes for the differences revealed by the post-hoc tests will need to be calculated by hand.
To calculate the effect size (Cohen's d), you need the means (x) and standard deviations (SD).
d = (x1 - x2) ÷ ((SD1 + SD2) ÷ 2)
Very Small < 0.3
Small-Medium > 0.3; < 0.5
Medium-Large > 0.5; < 0.8
Large > 0.8
Step 6: Write up
IV = Test Intervals (Levels: Time 0, excerise programme not yet begun; Time 1, at completion of 12-week programme; Time 2, follow-uo at 6 months after completing the programme)
DV = Anxiety score
Analyze > General Linear Model > Repeated Measures...
Figure 2. Mean Anxiety Score and Confidence Intervals before, immmediately after, and 6 months after the exercised programme
Students had lower anxiety scores after completion of the exercise programme. The mean anxiety score before the exercise programme was 66.25 compared to 37.1 immediately after completion of the exercise programme and 38.75 six months after that. The confidence intervals show that the means are reasonably close to the population mean. *Report upper and lower bounds of the confidence intervals*
A One-Way Within-Subjects ANOVA, correcting for a violation of sphericity by using the Greenhouse-Geisser value, revealed a significant difference between the means [f(1.033, 19.624) = 121.384, p < 0.001]. The Global Effect Size using Partial Eta Squared was 0.865 which is large, and the Observed Power was 1 which is very strong. A Bonferroni post hoc test revealed a significant difference in between anxietey scores before participants started the exercise programme and immediately after completing it (p < 0.001) with a very large effect size (d = 2.78), between anxietey scores before participants started the exercise programme and 6 months after completing it (p < 0.001) with a very large effect size (d = 2.57), and between anxietey scores immediately after participants completed the exercise programme and 6 months after completing it (p < 0.001) with a very small effect size (d = 0.13).
Note on Confidence Intervals
When 95% confidence intervals for the means of two independent populations don’t overlap, you can confidentally claim that there is a statistically significant difference between the means (at the 0.05 level of significance).
However, the opposite is not always true. Confidence intervals can overlap even when there is a statistically significant difference between the means.
In a between-subjects design, the confidence intervals can overlap by 25% and still be statistically significant. In a within-subjects design, confidence intervals can overlap quite substantially and still reveal a significant difference (as with the example above). They can be too big, because they don't incorporate the benefit of comparing people to themselves, and instead include between-subjects variance. Some argue that it is best to used the standard error of the difference instead, as this is less misleading. However, as an undergrad psychologist, I was always told to report the confidence intervals for ANOVA.