Intuitive explanation of double expectation
This has been bugging me for some time. The famous result in probability is like
$E[Y] = E[E[Y|X]]$
Can someone write an intuitive explanation of the above?
$\endgroup$ 51 Answer
$\begingroup$The object $\mathbb{E} \left[Y \, \middle| \, X \right]$ is actually a random variable. There are very good attempts to explain how it works, e.g. this answer, but the idea is to make $Y$ "coarser" in the sense that it only varies over certain subsets of the underlying set $\Omega$, or, more specifically, over a coarser $\sigma$-algebra (this picture from Wikipedia is what I'm getting at. Note how the conditional expectations of $X$ only vary over certain subsets of the original range).
Now, the random variable $\mathbb{E} \left[Y \, \middle| \, X \right]$ should be consistent with the original random variable $Y$. This is in the sense that if $\mathbb{E} \left[Y \, \middle| \, X \right]$ remains constant in $U \subset \Omega$, it's value should be the "average" value of $Y$ in this set (see, again the picture). Now taking the expectation, i.e. integrating over $\Omega$, the average value of the random variable over the sets $U$ is what matters. Since these are the same for $\mathbb{E} \left[Y \, \middle| \, X \right]$ and $Y$, we have your identity $\mathbb{E}[\mathbb{E} \left[Y \, \middle| \, X \right]] = \mathbb{E}[Y].$
This is all very informal, but the rigorous explanation is a lot easier to find than answer like this. I hope this helps.
$\endgroup$ 2
