Information and noisy channel

This gonna be a nerdy and cynical post. Close the window if you do not like that.

In information theory, there is a channel called Z channel. A channel sends information around and Z channel is not a perfect channel in that it makes mistakes. Suppose it sends codes made of 0 and 1's. When it sends 0, it is perfect, always sending a 0. However, when it sends 1's, it makes mistakes. Sometimes, it sends 0 instead.
http://en.wikipedia.org/wiki/Z-channel_%28information_theory%29

I was often curious how can such a channel exist. Today I think I saw something similar. The media!

Some media (since I am not worried about naming things I despise, I will name NYT), when reporting on some country acts like a Z channel. When there are bad things, it does a perfect job. When it comes to good things, it somehow report bad things.

How much information can we get via such a channel. That is measured by channel capacity C. Unfortunately, the channel capacity assumes the optimal input distribution (that is a particular distribution on 0 and 1's for Z channel, and a distribution on good things and bad things for our NYT).  The optimal distribution for a Z channel, (after some tedious algebra) must favor the 0 (bad things). That is if the real distribution favor 1 the good things, then the information we are able to get from that channel is way below channel capacity. Well, we sort of hope that good things happen more often, right? Too bad, that means, that NYT is pretty useless.

A big problem is that we often do not know the distribution of good things and bad things--that is part of the reason why we read NYT (if we ever bother to), right? And we often do not know how screwed up it is when it comes to good things. So the best we can do is to form a lower bound on the percentage of good things happening to that country. Other than that, no information (unless some genius statistician found a way, but I think it is more likely we can prove there is NO way). So why bother to read NYT?

Soul, style and character

You are supposed to do whatever you want, to think about new ideas, to express your views freely, to do things in the way that you choose and on top you will be rewarded nicely. These privileges should not be taken for granted. We are extremely lucky -- we owe something in return.

--Ariel Rubinstein

A digression is in order:


I will do what I think it is right. If I believe that something is non-sense, I refuse to work in the non-sensical manner. Too many times, I see professional researchers doing research that is totally relevant (i.e. nobody cares!) just because that is the mainstream in the literature. There are great people, who rebel against that.  It is very tough, as evident by Ariel Rubinstein's experience. When Rubinstein wrote a paper criticizing hyperbolic discounting, which was becoming increasingly popular at the time. The editor wrote:

"There are certainly many smart things in the piece but... this
seems like a critique of the current approach which is right in many ways, but critiques ...
of existing research are best fit to more specialized outlets".

While I am agnostic about the merit of hyperbolic discounting, I nevertheless found the editor's comment "curious" and Rubinstein's style refreshing (hint: understatement). What bothers me is that in some disciplines, people seem to write papers for the sake of it (I am reluctant to call that research, a brain teaser will be more fitting). They produce things no one will care about or have any use, other than being published on a journal. I always like to envision researchers as those who contribute to society by making available ideas that will be useful. Whether it is new economic insights that lead to better policy, or data compression algorithms that make youtubes possible, or better statistical methods that come to aid in daily life, they make the world better. I have no problem with it if doing such arcane "research" is just personal interest, but I highly doubt it. Furthermore, it is curious why the government should provide grants for such personal entertainment. The greatest philosophy I learned last year is that always ask "Who cares?". If no one cares, then you probably should not waste your time.

The problem is not restrained in academia. We see non-sense on our daily life. If you want a good sample, watch the presidential campaign. I really admire it when some great professional takes a stance against non-sense. In the world of political correctness dominating over truth, the battle could be tough. If you are not convinced, just google Larry Summers' encounter with Cornel West and the drama he ran into by his unpopular remarks of woman.

Now back to topic: I admire people with a soul, style and character, and I enjoy their books, because they will project their style to their books.  Let me end with some recommendation of books with souls, sytle, and character ( I do not necessarily agree with everything they say, but I love that they take stance against what they think is nonsense):
A Course in Game Theory by Ariel Rubinstein. I enjoyed his comments on the fuss over "he/she". I also tremendously enjoyed his skepticism of game theory itself.
Information Theory, Inference, and Learning Algorithms, by David MacKay. I enjoyed his promotion of Bayesian methods and they are persuasive. I also enjoyed his putting his book on his website. I enjoyed his dedication to peace.

Weighing the Odds: A Course in Probability and Statistics by David Williams. He made it very clear that hypothesis testing is not his favorite way to kill the time. And he really has a style.

The Intelligence Paradox: Why the Intelligent Choice Isn't Always the Smart One. by Kanazawa. The author made a really good point against claims along the line that since all humans are equally worthy, different gender and ethnic groups must be equally intelligent.

Translational musings 1

雁群休息时，必有一雁放哨，叫 雁奴，轮流担任，稍有异动，鸣叫示众。一人猎雁数次，皆无功而返。后想一法，径直奔雁群，待雁奴鸣叫后隐藏。群雁逃逸 数次发现无异情，迁怒于雁奴，后竟群而啄之。雁奴悲叹无语，待猎人再近，隐而不发，群雁死伤过半。世道亦如此，忠诚的被误解，被误解的不坚持。

There is one kind of geese that always travel in packs. Whenever they take a break, there will be a "guard goose" on surveillance--it will alert all the others when something is suspicious. One hunter failed hunt down a single geese after several trails. One day, he came up with an idea. He rushed to the geese, and after the "guard goose" sent the alarm, he hid himself. When the geese was alerted, they found no imminent danger. The hunter repeated this several times and the geese got angry at the "guard goose" and attacked it. The "guard goose" was sad but could not explain. When the hunter comes again, it kept silent--not surprisingly, the hunter got lots of geese. It is sad that the loyal get distrusted and the distrusted do not persist.

Neither shame nor pride

It is curious when I observe that for some people, they take pride in mathematicalization of their work while the others do exactly the opposite.

For the former, they would like to judge the work by how advanced mathematically it is. For them, mathematics, is good, people deal with hard stuff. Physics, equally good, but perhaps not as good. Bio and Chem, nah, the maths is too simple--it is for people who cannot handle maths.

For the latter, they hate to see maths. It is nerdy in a bad way. Not everything can be quantified. What is the point of being consumed in such abstract and mind-boggling things. why over-complicate things?

Personally, I feel maths is a tool for our use. We should be neither proud nor shamed of the tool we use. What matters is what we make out of it. If it is crap that came out, does it matter that we use the best best and most sophisticated tool? Actually if we only made crap with such good tools, it is even more shameful--what does this say about the person who used the tool?

There is much lament about getting mathematical for the sake of it in more applied sciences. In statistics, there are people who rejects Mathematical Statistics as Theoretical Statistics, as some of the results are just applying maths in a statistical setting without having any meaning, or serves as theory for something useful. Similar in economics, many exotic theories have been developed using super fancy maths, and I cannot see them more than a mathematician's game.

For fun, sure. I don't care. It is your choice. But to say it is a contribution. I dissent.

Personally, we need to do the "science" right. We must start with a "right" (important) question and a good attack on it. We do not start with some fancy tool and ask, hmm, where can I use this damn expensive tool that it took me some much time to learn?

A related issue is consent and dissent. Some like to believe that to dissent is to be original and to conform is lame. They like to categorize people who conform with their government as brain-washed, and people who dissent from their government as heroes. I could not find that to be more ridiculous.

When A or not A (A=mathematical, consent,...) becomes an objective, the original objective  (truth!!!!) gets blurred or even lost. That is sad sad sad.

As for A, neither shame nor pride should it carry. After all, it is only "truth" that can carry pride!

Nah, it is not dramatic enough

Years ago, I read a sarcastic essay. The specifics I remember not, but I do remember the first line about: When a boy hangs out with a girl on the street, and the girl intends to Jaywalk, the boy can either stops her, or go along and pulls her back when a car rushes by. It is much more likely that the latter choice will impress and move the girl.

It sounds silly, but I cannot deny there is an element of truth. When we do things, it does not matter how worthy they are, it matters (for publicity) how dramatic it is.

For the public, it is much better to rant against, ridicule and rebel against a government than work in the system and improve from within. Why? Rebellion is much more dramatic.

A state man is seldom acknowledged for keeping his country stable and thriving; we would rather give credit to someone who overthrow his own government. As for what happens afterwards, like is there a good government that takes over or is there just anarchy, we could not care less. (Personally, I abhor at the bloodiness of the French Revolution, rather than admire the courage of the people).

A movie about someone who does good job in good times, working relentlessly to prevent things getting out of control could not be exciting. We want to see things our of control and someone comes and bring it back to order.

Such worship of heroism will be counter-productive if it divert energy away from non-dramatic but helpful contributions (come on, working from within and reform gradually takes up patience and skills). What is more worrisome is it might probably distort incentives. Would someone or some country be tempted to create trouble so they could play a more dramatic role.

Drama, I love not.

China, growth star or growth disaster? (1)

When it comes to China, you can hardly find a neutral stance, especially concerning is economic development. While some see a Chinese model and predict that China will overtake US as the largest economy, others constantly predicts a collapse. To the discredit of the latter group, the crisis never materialized. While I can hardly find any common ground with the reasoning of the latter group, I do reach a similar conclusion--China is unlikely to be a growth star in the future.
 1. Economic inclusiveness or extractiveness?
I will discuss many of the problems with Chinese economic development, and some of the problems have largely been overlooked, but I wish to start the analysis with a talk given by James Robinson. James Robinson defined four concepts--economic inclusive policy, economic extractive policy, political inclusive institutions, and political extractive institutions. The claim is only inclusive-inclusive economies will sustain growth. In the talk he ascribed China as politically extractive and economically inclusive. Comparing to the grand predictions given to USSR, he point out that China might well be another disappointment in linear projection. While I certainly felt uncomfortable with the sarcasm he ridiculed such projections, the impotence of linear extrapolation is a real one. However, I fear economic extractiveness rather than political extractiveness is putting China's future in danger.

China's economic system is NOT inclusive. The SOE is the epitome of this extractiveness. The state grants many SOE's monopoly power--eg. in the area of petroleum, telecommunication and banking. With monopoly power in such key area, it is little wonder that those SOE's make gigantic profits--in other words, those SOE extract money from the citizens. I would not object to this system as strongly if either 1)the profits are turned to the state as a substitute of taxation; or 2) the employment within SOE is inclusive. Unfortunately, neither is not true. I will not go into details of the former since it has been extensively critiqued by astute observers. The second problem is more server. Some anecdotes might start this. Some of less-able cousins got jobs in banks. I was not surprised because I have enough aunts and uncles working in banks. I will not disclose the specific numbers, but they can take home quite big bonuses and salary immediately after they "launched" their career. On the other hand, I have some friends who are already working were not fortunate enough to secure that quick money. The difference is my friends graduated from the top high school and colleges in China, while some of my cousins graduated from "never-heard-of-colleges". The static welfare ill-distribution is unattractive, but it is not important. The real problem with stuffing those key areas (like banking and petroleum) with dumb people. Will it be any surprise if China's banking industry collapsed and undergo massive bankruptcy when they have to face foreign competition? No! Quite the contrary, I will be surprised if any bank survive. It is easy to lose competitiveness but it is very very hard, if possible, to regain them. The maintenance of SOE privileges serve as a way for economic and political elites to enrich their impotent kids  (of course if their kids were potent, they often choose to adopt more extractive method--an easy example is the son of the current prime minister of China). There has been attempts to curtail those extractiveness by introducing competition, but perhaps too little, too late.

There are many other forms of economic extractiveness in China--financial repression is one of them. But those are more discussed than the employment issue of SOE. Hence I will not repeat the wisdom of those acute observers.

Next time, I will continue to discuss the problem of SOE, though in a very different (almost opposite) flavor.

Revelation of blogging

Blogging is a tiring activity. Each time I write on a topic, it often takes me about seven hours. It is forming an opinion that sucks all the time away--the actual writing (as I do not polish them) once I formed an opinion, is quick and brisk. I always need to read extensively on the topic, trying to get a more comprehensive understanding of all relevant details so as to come to a judgement. Most often, under scrutiny, most issues defy an easy judgement, at least not one that is simple to state. Consequently, I blog quite seldom.

However, I form opinions all the time about things I hear and see. Most likely, I fail to do the kind of research that I do for my blogs.  It is often that when writing a blog that I changed my opinions, sometimes even reversed them. Then how do I form an opinion in daily life? I guess prejudice, bias, and sloppy thinking all creep in. This problem of sloppy thinking, I fear, haunts most people. Probably, we should be a little bit more cautious when voicing our "opinions", at least I do not wish to be embarrassed by them later in my life.