Analysis of Variance

Analysis of Variance, better known as ANOVA for short, allows the researcher to compare the differences among many sample groups. Where t is for "two", the F ratio, which is the calculated value obtained from ANOVA, can handle any number of group comparisons. T testing allows us to only have two groups (experimental and control). With ANOVA, we can set up a number of experimental groups to compare with the control group. The F ratio receives its name from its founder Ronald Fisher.

For example, consider the cholesterol example we had for homework. In the homework we had two groups (Those exposed to exercise and those who were not), now we can also add in a group who is taking an experimental cholesterol-lowering medication.

The question might arise "why not just do T tests". After all, we could conduct successive t tests. If we have 3 groups, then we could test group 1 versus group 2, group 2 versus group 3, and group 1 versus group 3. Why would this not be advisable?

Because if we set alpha level to .05, we are saying that we are willing to accept a mistake 5 times out of 100, if we perform multiple t tests then aren’t we stating that we accept greater alpha levels. Of course, if we perform multiple tests there are multiple opportunities to commit an error.

ANOVA is limited in that, unlike the t test, we cannot conduct directional tests. Therefore, while the test is limited, the writing of the null hypothesis is easier to write. For example, suppose we are interested in seeing if there is a statistically significant difference between the mean shooting scores between three different shooting techniques taught to correctional officers during their academy. The null hypothesis would be written as follows:

Since we are comparing the groups all at one time, then if we set the alpha level at .05, then the alpha level stays at .05; regardless of how many groups are included in the study.

HOW ANOVA WORKS

Variance, if we think back, is a measure of the amount that all scores in a distribution vary around the mean of the distribution. A large variance value indicates a large amount of difference from the mean and a small variance value indicates very little difference from the mean of the distribution.

The question that inevitably arises when several groups of scores are being compared, is which mean a score should be compared to. After all, with t tests the answer was simple – there were two means and they were compared to each other. With ANOVA there are three or more means. So which mean do we compare with? This is where Ronald Fisher came in—he argued that the solution was to examine them all. To do this he developed a formula that would sort out the variance between the different means and then compare these variance components (hence, the name Analysis of Variance); this procedure is known as partitioning the variance.

Variance is separated into the variance between and the variance within. (Illustrate this concept on the board with an example). Between variance is the variance we want to look for because this the variance that we can attribute to our treatment effect. Within variance is variance that we cannot explain because it happens within the group (this variance is referred to as error).

POST HOC IN ANOVA

Post hoc testing allows us to go back and examine the differences between the groups in an attempt to discover where the differences lie. After all, finding a statistically significant difference in ANOVA only tells us that there is a difference we still have to figure out where the differences lie. The main thing to remember about post hoc testing is that it is only required if we find that there is a statistically significant difference. If there is not, then there is no need to continue examining the post hoc. Once we have information relating to where the differences lie, we can then make clear-cut statements about which group performed the best in the study. We will walk through a couple of these.

ASSUMPTIONS OF ANOVA