Friday, October 22, 2010

Kolmogorrov's 0-1 Law

We learned the Kolmogorrov's 0-1 Law today in class.

And I had this (very very informal) intuition:

In the 0-1 Law we talk about the tail events, i.e., the events belonging to the tail sigma algebra. The tail sigma algebra is defined as

F_tail = \cap_{n >= 1}  \sigma(X_n, X_n+1, X_n+2, ... )

That is, it consists of events that are independent of finitely many initial X_n's. 

Obviously, F_tail \subset F := \sigma(X_1, X_2, ....), the "full" sigma algebra. 

In a way, to construct the F_tail, we 'drop' the initial X_n's one by one. For each X_n dropped, the sigma algebra becomes smaller but is still a sigma algebra. Note that this sigma algebra encodes lesser bits of information.

Thus, 'in the end', when you have 'dropped all the X's', you have a sigma algebra but you do not have any information bits left - and that sigma algebra is effectively the trivial sigma algebra. Thus any event in it must have probability 0 or 1!

Of course, this is very very vague and there is a LOT of handwaving in the arguments - but I thought the intuition was cool.

As a corollary, it would also be interesting to consider the effective information content of a probability measure on a sigma algebra. That is, in the tail sigma algebra above, the effective information content is 1 bit. 

No comments: