In calculus, we study the convergence properties of series. Similarly, in probability theory, we are interested in the convergence of series of random variables. Consider the series:
$$ \sum_{n=1}^\infty \frac{Z_n}{n} $$
where each $Z_n$ is an independent and identically distributed (i.i.d.) random variable taking values in $\{ -1, 1 \}$ with equal probability (i.e., $\mathbb P(Z_n=1)=\mathbb P(Z_n=−1)=\frac 12$).
Question: How can we determine the convergence properties of this series? Specifically, does the series converge almost surely, in mean square, or in distribution?
To answer the question, we prove the following theorem:
Theorem 1(Three-series theorem). Let $X_1, X_2, . . .$ be independent. Let $A > 0$ and $Y_n = X_n\mathbb 1_{\{|X_n|≤A\}}$. In order for $\sum_n X_n$ to converge a.s., it is necessary and sufficient that:
$$ \sum_{n=1}^\infty\mathbb{P}[|X_n|>A]<+\infty,\tag{1} $$
$$ \sum_{n=1}^\infty\mathbb{E}[Y_n]\textit{ converges},\tag{2} $$
and
$$ \sum_{n=1}^\infty\mathrm{Var}[Y_n]<+\infty.\tag{3} $$
Remark. This theorem controls the convergence of $X_n$ via a real number $A$ and another series $Y_n$, we could simply view (1) as the condition to control the terms in $X_n$ which is greater than $A$.
Sufficiency
Lemma 2(First Borel-Cantelli lemma (BC1)). Let $(A_n)_n$ be a sequence of events. If
$$ \sum_n\mathbb{P}[A_n]<+\infty, $$
then
$$ \mathbb{P}[A_n\text{ i.}o.]=0. $$
Proof. We claim it suffices to prove that
$$ \sum_n(Y_n-\mathbb{E}[Y_n])\text{ converges.} $$