Tuesday, October 23, 2012

Information and noisy channel

This gonna be a nerdy and cynical post. Close the window if you do not like that.

In information theory, there is a channel called Z channel. A channel sends information around and Z channel is not a perfect channel in that it makes mistakes. Suppose it sends codes made of 0 and 1's. When it sends 0, it is perfect, always sending a 0. However, when it sends 1's, it makes mistakes. Sometimes, it sends 0 instead.
http://en.wikipedia.org/wiki/Z-channel_%28information_theory%29

I was often curious how can such a channel exist. Today I think I saw something similar. The media!

Some media (since I am not worried about naming things I despise, I will name NYT), when reporting on some country acts like a Z channel. When there are bad things, it does a perfect job. When it comes to good things, it somehow report bad things.

How much information can we get via such a channel. That is measured by channel capacity C. Unfortunately, the channel capacity assumes the optimal input distribution (that is a particular distribution on 0 and 1's for Z channel, and a distribution on good things and bad things for our NYT).  The optimal distribution for a Z channel, (after some tedious algebra) must favor the 0 (bad things). That is if the real distribution favor 1 the good things, then the information we are able to get from that channel is way below channel capacity. Well, we sort of hope that good things happen more often, right? Too bad, that means, that NYT is pretty useless.

A big problem is that we often do not know the distribution of good things and bad things--that is part of the reason why we read NYT (if we ever bother to), right? And we often do not know how screwed up it is when it comes to good things. So the best we can do is to form a lower bound on the percentage of good things happening to that country. Other than that, no information (unless some genius statistician found a way, but I think it is more likely we can prove there is NO way). So why bother to read NYT?

No comments:

Post a Comment