Generalized Linear Models

Overview

A generalized linear model (GLM) extends ordinary linear regression so that the outcome can be continuous, binary, count, or another exponential-family outcome.

Core idea

A GLM models the expected outcome through a link function applied to a linear predictor:

g (E [Y_{i}]) = η_{i} = α + β_{1} X_{i 1} + β_{2} X_{i 2} + \dots

where $g (\cdot)$ is the link function and $η_{i}$ is the linear predictor.

Three Components

Random component: the outcome distribution, e.g. Normal, Bernoulli, Poisson.
Systematic component: the linear predictor $η = X β$ .
Link function: maps the expected outcome to the linear predictor scale.

In other words, a GLM combines:

a linear predictor: $X β$
a link function / nonlinearity: $g (μ)$
a noise model / outcome distribution: $Y \sim$ chosen family

Common GLM Examples

Model	Outcome type	Distribution	Link function	Typical question
Linear regression (Regression#Linear Regression)	Continuous	Normal	Identity	How does $Y$ change with $X$ ?
Logistic regression (Regression#Logistic Regression)	Binary	Bernoulli	Logit	How does $X$ affect the probability of an event?
Poisson regression	Count	Poisson	Log	How does $X$ affect an event rate or count?

Linear Regression as a GLM

Y_{i} \sim Normal (μ_{i}, σ)

μ_{i} = α + β_{X} X_{i} + β_{Z} Z_{i}

Link: identity
Interpretation: coefficients are additive changes in the expected value of $Y$ .

Logistic Regression as a GLM

Y_{i} \sim Bernoulli (p_{i})

logit (p_{i}) = \log (\frac{p_{i}}{1 - p_{i}}) = α + β_{X} X_{i} + β_{Z} Z_{i}

Link: logit
Interpretation: coefficients are changes in log-odds; exponentiated coefficients are odds ratios.

Poisson Regression as a GLM

Y_{i} \sim Poisson (λ_{i})

\log (λ_{i}) = α + β_{X} X_{i} + β_{Z} Z_{i}

Equivalently:

λ_{i} = \exp (α + β_{X} X_{i} + β_{Z} Z_{i})

Link: log
Interpretation: coefficients are changes in the log expected count; exponentiated coefficients are rate ratios.

Relationship to Other Models

Relationship to Mixed Models

GLMs assume that observations are independent unless the model is extended. This is where Linear Mixed Models and generalized mixed models become important.

GLM, LMM, and GLMM

GLM = generalized outcome distribution + link function, but usually independent observations.
LMM = linear/Normal outcome model + random effects for clustered or repeated data.
GLMM = GLM + random effects; useful for binary/count repeated-measures or clustered outcomes.

Example:

Binary outcome, independent patients → logistic regression GLM.
Continuous repeated measurements → LMM.
Binary repeated measurements from the same patients → logistic GLMM.

Relationship to GEE

Generalized estimating equations (GEE) extend the GLM idea to correlated observations, such as repeated measurements from the same patient in Longitudinal Data Analysis.

Feature	GLM	GEE
Data	Independent observations	Correlated or repeated observations
Target	Covariate–outcome association	Population-average association
Outcomes	Continuous, binary, count, etc.	Same outcome families via link functions
Correlation	Usually assumes independence	Specifies a working within-subject correlation
Interpretation	Model-specific mean association	Marginal / population-average association

GEE is closely related to GLMs, but it is not simply another GLM distribution. It is an estimation framework for correlated data.

Regression — linear and logistic regression foundations
Linear Mixed Models — mixed-effects models for continuous repeated or clustered outcomes
Longitudinal Data Analysis — choosing among LMM, GEE, and other longitudinal methods
ANOVA & Post-hoc Tests — classical linear-model comparisons
Random Variables & Probability distributions — distributions used as GLM outcome families