The General Linear Model (GLM)

[1] What's the General Linear Model (GLM)?

In GLM, there is a continuous response variable, and one or more categorical/continuous explanatory variables. The response Y is linear in the beta's. In SAS, we use proc glm or proc mixed statement. In R we use lm().  For more information, please refer to the reference below.

Reference: http://support.minitab.com/en-us/minitab/17/topic-library/modeling-statistics/anova/basics/what-is-a-general-linear-model/ 

One way ANOVA is a special case of a general linear model. Of course ANOVA, ANCOVA, linear regression, mixed model also are the GLM. In GLM, we can predict beta by using the least squared or the best linear unbiased prediction. Don't confuse the difference between General Linear Model and Generalized Linear Model later. Let's find out what the hypothesis and assumptions of GLM and what the one-way ANOVA is.

[2] GLM: Hypothesis and Assumptions

The null hypothesis is $\beta_{1}=\beta_{2}=...=\beta_{p}=0$

The GLM assumptions should be the errors are normally distributed with mean 0 and a constant variance, and they are uncorrelated.

 

We can predict the beta with the least squares estimates.

$\widehat{\beta}=(X^TX)^{-1}X^TY$

- The columns of X should be linearly independent, and $X^TX$ should be invertible.  

[3] One Way ANOVA : Assumptions, Table

In the One way ANOVA, the reponse variable is continuous, there is one categorical factor with at least 2 levels. The group variances are equal or at least fairly similar, and the errors are normality distributed. We are not assuming no outliers, also balanced groups are not required. The one way ANOVA table is following.

The P-value is come from high observed F statistics, and this statistics is come from the ratio of variances of two groups. Therefore, the main idea is if the between groups SS is larger than within the groups SS, there is evidence that means are different.