The background is that I went to the Boston RMV to apply for a learner's permit. I had to go through the driver's manual. Unfortunately, there is some statistical non-sense in the manual that annoys me greatly.
In order to convince drivers to wear safetybelt all the time, they say "Most motor vehicle crashes happen within 25 miles of home. In fact eight out of every ten accidents occur when drivers are going 40 mph or less". The problem of this sentence is that it is irrelevant and misleading. This is what is called a prosecutor's fallacy (http://en.wikipedia.org/wiki/Prosecutor's_fallacy). The idea is the the probability of event A given B could be very different from the probability of event B given A. Whether we want to wear a seatbelt is dependent on the probability of having an accident conditional on we are only driving within 25 miles of home. If that is too low to justify the cost of putting on a seatbelt, a rational person might decide not to. However, we are not interested in the probability of being within 25 miles of home given an accident happen. Reductio ad absurdum, Imagine a world where everyone drives within 25 miles of home. Even if the accident rate is negligible (less than the accident rate of air plane), we can still claim that most (actually in this case, all) motor vehicle crashes happen within 25 miles of home.
I completely understand that the intention of RMV is good--to convince people to wear safety belt. But good intention does not justify ignorance/stupidity or outright misleading. (I myself have lots against the compulsory seat belt wearing thing, as I think it makes the road less safe, but that is a topic for another day)
Prosecutor's fallacy is rampant, to my dismay. Let me digress and point out how prevalent it is. The Phi Beta Kappa society, which claims itself to be an elite society, points out that "Since inception, 17 U.S. Presidents, 38 U.S. Supreme Court Justices, and 136 Nobel Laureates have been inducted members." seems likes a high percentage for a lay person? Not really? I have an even better number. All of US presidents, US Supreme Court Justics, and Nobel Prize winners are human beings!!! So the right thing to consider is what is the probability of doing great things given a PBK membership? Probably pathetically low. My fellow PBKs, I do not think you have any reason to register any pride with this society.
Back to RMV. When I entered RMV at 11:51. I got a slip saying that the estimated waiting time is 14 min. I thought to myself, great!!! Then I was called when it was past 13:00. I doubt if takes super advanced statistics to get a slightly more accurate estimates.
Thursday, August 29, 2013
Friday, May 24, 2013
Why I hate dogs
I hate dogs.
I like to bike and many times when I bike pass some house, there will be a dog barking fiercely at me, sometimes even chase me, as if I am about to invade their house.
From the intelligence of a human being, I am so unlikely to be an invader, by just biking past a house. But for the intelligence of a dog, what do you expect? Perhaps, I should not hate them, but I am annoyed.
Some politicians like to call this country a threat and that country a threat when those countries are peacefully minding their own business, just like me. Perhaps, even more ridiculous, a dog won't even bark at an ant walking past its house.
paranoid, or?
I like to bike and many times when I bike pass some house, there will be a dog barking fiercely at me, sometimes even chase me, as if I am about to invade their house.
From the intelligence of a human being, I am so unlikely to be an invader, by just biking past a house. But for the intelligence of a dog, what do you expect? Perhaps, I should not hate them, but I am annoyed.
Some politicians like to call this country a threat and that country a threat when those countries are peacefully minding their own business, just like me. Perhaps, even more ridiculous, a dog won't even bark at an ant walking past its house.
paranoid, or?
Monday, May 13, 2013
No pressure, my friend
I like to be pressure-free. Neither do I like to pressure myself, nor do I like to get pressure from others.
Friends. I have always believed in the Chinese saying that the friendship between noble people should be as bland as water 君子之交淡如水。It means that when we make friends, we should not expect anything from friends, and we should not make friends feel pressured to do anything because of us. I am always hesitant to ask favors from friends, especially close friends---being close friends, I fear they will tend to feel the pressure to agree, and that makes me feel upset. I really hope one day people around me, will always tell me honestly, whenever they feel inconvenient to do something I asked for.
People now like to show "support" for friends. For me, I think it is quite unnecessary. We are friends because we share similar interests, but we do not have identical interests. When I am doing something I am interested in but you are not, why should you feel obliged to show support? Friendship, I feel, should be based on common interests and goals, not on reciprocity. One of my friends told me after my thesis defense that when he arrived, he knew he would not understand anything, and said those things are about friendship. I replied that is exactly the reason why I was reluctant to tell him the time and place of my defense. I feel guilty for creating pressure on you, my friend. I showed up to your history thesis defense not because you are my friend, but because I am genuinely intrigued by your paper.
Friendship should be simple. No burden. Simple joy.
Friends. I have always believed in the Chinese saying that the friendship between noble people should be as bland as water 君子之交淡如水。It means that when we make friends, we should not expect anything from friends, and we should not make friends feel pressured to do anything because of us. I am always hesitant to ask favors from friends, especially close friends---being close friends, I fear they will tend to feel the pressure to agree, and that makes me feel upset. I really hope one day people around me, will always tell me honestly, whenever they feel inconvenient to do something I asked for.
People now like to show "support" for friends. For me, I think it is quite unnecessary. We are friends because we share similar interests, but we do not have identical interests. When I am doing something I am interested in but you are not, why should you feel obliged to show support? Friendship, I feel, should be based on common interests and goals, not on reciprocity. One of my friends told me after my thesis defense that when he arrived, he knew he would not understand anything, and said those things are about friendship. I replied that is exactly the reason why I was reluctant to tell him the time and place of my defense. I feel guilty for creating pressure on you, my friend. I showed up to your history thesis defense not because you are my friend, but because I am genuinely intrigued by your paper.
Friendship should be simple. No burden. Simple joy.
Friday, May 3, 2013
Why I gave up martial art
I used to be a student of martial art, a fervent one. During my worst days, that was my only companion--it was only during those hours when I practice martial art that I could focus and not distracted by painful thoughts. I seriously thought I would take it as a life-long pursuit.
But I gave it up in college. Williams is not to be blamed. It is true we do not have such a professional martial art club, but I gave it up more because I wanted to.
My first encounter with Martial Art is Tai Chi, an ancient form of Chinese martial art. The first day we did not learn forms, but about the philosophies. I remember very clearly what the master said: " The purpose of martial art is not violence, but is to stop violence. We learnt that our biggest enemy is never others people, but ourselves, our inner urge toward violence and brutality, and to master martial art, we need to master the art to curb such aggression. （克己） This is not an idealistic romantic tenet of some ancient Chinese philosophers, but a very wise idea---you can be very strong, as strong as you can, but if you cannot curb your urge to violence and aggression, soon it will backfire when you upset enough people and/or strong enough people. We will turn old and weak, and none of us can stay on the top, aggression and violence will turn back toward us if we do not curb them in the first place. Respect (礼）and benevolence (任）are always stressed. As these guide the previous principle, which literally means hold back yourself from you should not do. Even in Japan, the tenet of Bushido includes respect and benevolence, but they over-stressed loyalty and honor, and when being loyal to the wrong thing, and taking honor in the wrong thing, things get bad. What happened in WWII is only a sick and disgusting deviation from the true spirit, not the orthodox martial art philosophy. In the movie Yip Man, Yip Man said rightly: " You Japanese invaders are unworthy of martial art".
That is all good and beautiful. Later I realized however, mastering martial art will not help stop violence. In this age, violence never takes the form of bare-fist fight between individuals, but are staged by countries, organizations, and using deadly weapons that easily out-power the most skillful martial artist. In the movie Yip Man, Yip Man felt the same futility when he saw Japanese invaders did horrible things, and killed people but he could do nothing though he is the best martial artist of the time. What is more, I realized I began to develop knee problems from martial art practice. The purpose of martial art is to protect others from violence, and not only is that impossible, but also, instead of protecting ourselves, we are hurting ourselves. That was the last straw.
In this world, when the media continuously distort the truth, we do not even know what is right and what is wrong, then what is the use of power?
But I gave it up in college. Williams is not to be blamed. It is true we do not have such a professional martial art club, but I gave it up more because I wanted to.
My first encounter with Martial Art is Tai Chi, an ancient form of Chinese martial art. The first day we did not learn forms, but about the philosophies. I remember very clearly what the master said: " The purpose of martial art is not violence, but is to stop violence. We learnt that our biggest enemy is never others people, but ourselves, our inner urge toward violence and brutality, and to master martial art, we need to master the art to curb such aggression. （克己） This is not an idealistic romantic tenet of some ancient Chinese philosophers, but a very wise idea---you can be very strong, as strong as you can, but if you cannot curb your urge to violence and aggression, soon it will backfire when you upset enough people and/or strong enough people. We will turn old and weak, and none of us can stay on the top, aggression and violence will turn back toward us if we do not curb them in the first place. Respect (礼）and benevolence (任）are always stressed. As these guide the previous principle, which literally means hold back yourself from you should not do. Even in Japan, the tenet of Bushido includes respect and benevolence, but they over-stressed loyalty and honor, and when being loyal to the wrong thing, and taking honor in the wrong thing, things get bad. What happened in WWII is only a sick and disgusting deviation from the true spirit, not the orthodox martial art philosophy. In the movie Yip Man, Yip Man said rightly: " You Japanese invaders are unworthy of martial art".
That is all good and beautiful. Later I realized however, mastering martial art will not help stop violence. In this age, violence never takes the form of bare-fist fight between individuals, but are staged by countries, organizations, and using deadly weapons that easily out-power the most skillful martial artist. In the movie Yip Man, Yip Man felt the same futility when he saw Japanese invaders did horrible things, and killed people but he could do nothing though he is the best martial artist of the time. What is more, I realized I began to develop knee problems from martial art practice. The purpose of martial art is to protect others from violence, and not only is that impossible, but also, instead of protecting ourselves, we are hurting ourselves. That was the last straw.
In this world, when the media continuously distort the truth, we do not even know what is right and what is wrong, then what is the use of power?
Friday, April 19, 2013
A puzzle and my critique
Foreword: I started the blog with the intention to write only two paragraphs with the title "A puzzle or not". Fortunately for me, and unfortunately for the reluctant readers, many related ideas came to me when I wrote this, and before I know it, this blog contains a hodgepodge of ideas.
In Economics, there are often "puzzles". These puzzles refer to the situations that the observed behavior deviates from the optimal one. For example, the famous equity premium puzzle refers to the fact that "mysteriously" people are under-investing in equity, given the high equity premium. There are numerous such puzzles in game theory, when numerous experiments confirmed that people are not behaving optimally on average.
Are these puzzles?
Many economists say "yes". Their argument is this, while we do not expect complete individual rationality there is no reason to expect people's behavior to systematically deviate from the optimal. In other words, people's deviation from the optimal should cancel each out! Since many earlier famous economists are trained in Physics, they might even draw an analogy with Physics (I am not aware if anyone did indeed draw such an analogy, but I imagine some would), when there is a voltage difference, the electrons will on average predictably flow in one direction, even individual electron might deviate from the predicted flow. More philosophically, local disorder does not preclude global order. This is indeed such a powerful idea, upon which many sciences, physical or social rest upon. I personally am obsessed with this idea, and I think if I can only remember one idea, this will be the one. Nevertheless, this idea has its bounds.
While local disorder does not preclude global order, it does not GUARANTEE global order either. While we do not expect people's behavior to systematically deviate from the optimal, we do not have any reason to expect that the average behavior will converge either! Then what is this average behavior then? Let me first divert you to a story the mighty Richard Feynman told:
Nobody was permitted to see the Emperor of China, and the question was, What is the length
of the Emperor of China’s nose? To find out, you go all over the country asking people what
they think the length of the Emperor of China’s nose is, and you average it. And that would
be very “accurate” because you averaged so many people. But it’s no way to find anything out;
when you have a very wide range of people who contribute without looking carefully at it, you
don’t improve your knowledge of the situation by averaging.
That's the point! When we are confronted with a very complicated economic decision (it does not have to be really complicated, it could be as simple as a game of flipping coins), many of us the laypeople are confused and have no idea what is optimal. When the welfare consequences are not that high, we basically just choose one action that looks right to us, and there is no guarantee what looks right to us will average out to be optimal. Yeah, you are right, we have some sense of what could be optimal and what could not. But even in the story of the nose length, the people have some sense. They know 100 inch is unlikely, neither is 0.005 inch. It will be obvious to people that some options are bad, but that's it.
Why I bother to write this blog? For one thing, I like to test out my ideas on the internet and see if there are loopholes or counter-arguments obvious to other readers. But I actually think this is an important idea (no claim of originality here, as Feynman definitely saw this): semi-optimization (eliminating obviously bad choices) or bounded rationality will not lead us to optimal behavior on average. This calls into question lots of the work we do in Economics. When we run regression to estimate parameters like elasticity, many of the implicit assumption is that the average behavior we observe is optimal. This assumption actually does not need to hold, and when that's the case, the fancy regression some people do might be of the same quality as the estimation of the emperor's nose length.
One challenge facing social scientists is causality. For example, when we try to estimate the marginal value of education, we worry about omitted variable bias. We worry that one's education level might be correlated with the person's innate ability, which might again correlate with his earnings. Thus what some people do is to construct a model where the person choose the level of education optimally based on his innate ability. From that model, they design some statistical inference procedure. If only the average behavior is optimal.
Causality is indeed a huge challenge confronting social scientists. However, there are other challenges. The fact that the objects social scientists study (us!) do not fully optimize is a serious challenge. If we do not optimize at all, it is easy to deal with. If we fully optimize, it is slightly more complicated, but handleable. But semi-optimization is tough to deal with. We need to keep in mind people look to social science to guide their optimization--we might choose whether we want to continue education based on economists' estimates of marginal value of education, and it will be pretty funny to know that the very estimates economists come up with is based on the assumption that people have on average optimized.
What is the right attitude to do science? It is often tempting to make some assumptions so that things are tractable. If those are innocuous assumptions, then go ahead, and I think it is great! However, when those assumptions might invalidate the results we get, I would suggest caution. We do not do science because we want to be able to say something about it, but because we want to say something right! I do not mean, however, that we should stop, we might still benefit from a model with dubious assumption as that model might act as a benchmark. Nevertheless, we need to be aware of the model's limit. To act or implement policy assuming the model is correct is a "pretense of knowledge". Austrian school distrust econometrics, for a reason---so many things could go wrong. Mainstream economists dislike them, and I think part of the reason is that they will not be able to do much with the data under the Austrian critique. While I believe we could still do thoughtful empirical work, I have to admit the market is flooded with mindless regressions with no value what so ever!
In Economics, there are often "puzzles". These puzzles refer to the situations that the observed behavior deviates from the optimal one. For example, the famous equity premium puzzle refers to the fact that "mysteriously" people are under-investing in equity, given the high equity premium. There are numerous such puzzles in game theory, when numerous experiments confirmed that people are not behaving optimally on average.
Are these puzzles?
Many economists say "yes". Their argument is this, while we do not expect complete individual rationality there is no reason to expect people's behavior to systematically deviate from the optimal. In other words, people's deviation from the optimal should cancel each out! Since many earlier famous economists are trained in Physics, they might even draw an analogy with Physics (I am not aware if anyone did indeed draw such an analogy, but I imagine some would), when there is a voltage difference, the electrons will on average predictably flow in one direction, even individual electron might deviate from the predicted flow. More philosophically, local disorder does not preclude global order. This is indeed such a powerful idea, upon which many sciences, physical or social rest upon. I personally am obsessed with this idea, and I think if I can only remember one idea, this will be the one. Nevertheless, this idea has its bounds.
While local disorder does not preclude global order, it does not GUARANTEE global order either. While we do not expect people's behavior to systematically deviate from the optimal, we do not have any reason to expect that the average behavior will converge either! Then what is this average behavior then? Let me first divert you to a story the mighty Richard Feynman told:
Nobody was permitted to see the Emperor of China, and the question was, What is the length
of the Emperor of China’s nose? To find out, you go all over the country asking people what
they think the length of the Emperor of China’s nose is, and you average it. And that would
be very “accurate” because you averaged so many people. But it’s no way to find anything out;
when you have a very wide range of people who contribute without looking carefully at it, you
don’t improve your knowledge of the situation by averaging.
That's the point! When we are confronted with a very complicated economic decision (it does not have to be really complicated, it could be as simple as a game of flipping coins), many of us the laypeople are confused and have no idea what is optimal. When the welfare consequences are not that high, we basically just choose one action that looks right to us, and there is no guarantee what looks right to us will average out to be optimal. Yeah, you are right, we have some sense of what could be optimal and what could not. But even in the story of the nose length, the people have some sense. They know 100 inch is unlikely, neither is 0.005 inch. It will be obvious to people that some options are bad, but that's it.
Why I bother to write this blog? For one thing, I like to test out my ideas on the internet and see if there are loopholes or counter-arguments obvious to other readers. But I actually think this is an important idea (no claim of originality here, as Feynman definitely saw this): semi-optimization (eliminating obviously bad choices) or bounded rationality will not lead us to optimal behavior on average. This calls into question lots of the work we do in Economics. When we run regression to estimate parameters like elasticity, many of the implicit assumption is that the average behavior we observe is optimal. This assumption actually does not need to hold, and when that's the case, the fancy regression some people do might be of the same quality as the estimation of the emperor's nose length.
One challenge facing social scientists is causality. For example, when we try to estimate the marginal value of education, we worry about omitted variable bias. We worry that one's education level might be correlated with the person's innate ability, which might again correlate with his earnings. Thus what some people do is to construct a model where the person choose the level of education optimally based on his innate ability. From that model, they design some statistical inference procedure. If only the average behavior is optimal.
Causality is indeed a huge challenge confronting social scientists. However, there are other challenges. The fact that the objects social scientists study (us!) do not fully optimize is a serious challenge. If we do not optimize at all, it is easy to deal with. If we fully optimize, it is slightly more complicated, but handleable. But semi-optimization is tough to deal with. We need to keep in mind people look to social science to guide their optimization--we might choose whether we want to continue education based on economists' estimates of marginal value of education, and it will be pretty funny to know that the very estimates economists come up with is based on the assumption that people have on average optimized.
What is the right attitude to do science? It is often tempting to make some assumptions so that things are tractable. If those are innocuous assumptions, then go ahead, and I think it is great! However, when those assumptions might invalidate the results we get, I would suggest caution. We do not do science because we want to be able to say something about it, but because we want to say something right! I do not mean, however, that we should stop, we might still benefit from a model with dubious assumption as that model might act as a benchmark. Nevertheless, we need to be aware of the model's limit. To act or implement policy assuming the model is correct is a "pretense of knowledge". Austrian school distrust econometrics, for a reason---so many things could go wrong. Mainstream economists dislike them, and I think part of the reason is that they will not be able to do much with the data under the Austrian critique. While I believe we could still do thoughtful empirical work, I have to admit the market is flooded with mindless regressions with no value what so ever!
Friday, April 12, 2013
Decision
First let me quote from The Art of Choosing,
It was tough. Actually no. It was tormenting. Lots of people made it very hard, but I do not mean it in a bad way---I loved those people and really wish I could hang out with them in the next five years. I care about the feelings of lots of people--scoff at as you may, but that is part of my upbringing.
The above quote from The Art of Choosing serves to illustrate the very different mode of thinking that those brought up in the Western world is unfamiliar with. I am not trying to explain my decisions, I just wish to say because of my mindset, it is just tougher.
Chuan, Nick and Jenny, I really wish to join you all for the greater Williams community in Bay area, I am sorry I cannot. Jessy, Kelly and all the other friends I have or I have made in Stanford, how I wish to be able to hang out with you during the next five years. I am really sorry. Fate is relentless---I won't get it all. I hope it will somehow bring us back together sometime in the future.
I wish to say no more about the decision. I wish to explain it no more. However I made the decision is no longer important. The important thing is that I will live with it for the rest of my life. I will not regret. I will do my best.
I am really exhausted. I long for a cup of nicely marked caffe machiatto.
the more choice you have, the greater the number of appealing options, no matter how discriminating your tastes...So you'll have to make some sacrifices, and each of these carries a psychological cost. Your enjoyment of the chosen will be diminished by your regret over what you had to give up. In fact, the sum total of the regret over all the "lost" options may end up being greater than your joy over your chosen options, leaving you less satisfied than you would have been if you had had less choice to begin with.I read the book when I felt I could not deal with all the tough choices anymore, I chose to study the very process of choosing. As it turns out, theory enlightens but does not solve everything. In the end, I had to face the tough problem head-on.
It was tough. Actually no. It was tormenting. Lots of people made it very hard, but I do not mean it in a bad way---I loved those people and really wish I could hang out with them in the next five years. I care about the feelings of lots of people--scoff at as you may, but that is part of my upbringing.
In the words o Harry Triandis, they are "primarily motivated by the norms of, and duties imposed by, those collectives" and "are willing to give priority to the goals of these collectives over their personal goals," emphasizing above all else "their connectedness to members of these collectives"...it's believed that individuals can be happy only when the needs of the group as a whole are met.
The above quote from The Art of Choosing serves to illustrate the very different mode of thinking that those brought up in the Western world is unfamiliar with. I am not trying to explain my decisions, I just wish to say because of my mindset, it is just tougher.
Chuan, Nick and Jenny, I really wish to join you all for the greater Williams community in Bay area, I am sorry I cannot. Jessy, Kelly and all the other friends I have or I have made in Stanford, how I wish to be able to hang out with you during the next five years. I am really sorry. Fate is relentless---I won't get it all. I hope it will somehow bring us back together sometime in the future.
I wish to say no more about the decision. I wish to explain it no more. However I made the decision is no longer important. The important thing is that I will live with it for the rest of my life. I will not regret. I will do my best.
I am really exhausted. I long for a cup of nicely marked caffe machiatto.
Monday, April 1, 2013
Let the data speak!
I am against the trend that we let data, and only data to speak for themselves. But I do have some thoughts on what they should speak. Namely, questions!
Asking the right question is an important, important skill. What is a good question? It is relevant, and we have a good attack! In statistical science, the second point means we have the right data! If we only have like 10 data points and you wish to estimate the parameters for a super-complicated models involving 10 parameters, I wish you good luck.
When I took Econometrics, I was cynical. I am unhappy with linear models. As far as I see, the relation we try to estimate could be quite non-linear. However, there is a point, that I just begin to see. For a long time, (still in many areas today), data are scarce. It will be unrealistic to do a highly structural estimation, and it makes perfect sense for us to restrict us to linear models. At least, we recover correlations. The caveat many miss is that, once the model is ill-specified, the coefficients we get might not be interpretable, or requires more thinking, and quantifiers.
If I indeed get lots of observations for each possible vector of independent variable, I could impose no structure I could simply estimate the mean of y_i for each possible vector of independent variable, and here we go. I am not saying that's the right way to do it, but it will be ok.
A related talk I heard last semester is on spatial statistics. It talks about estimating earthquake frequency. The talk discusses how fine the grid should be. The solution is let the data decide! If you happen to have lots of observations, you can have finer grids...duh.
My view is that there is no physical probability! The world is deterministic. (For example, the coin flip is not random! At Persi can toss it however he wants) There is only belief. Lots of things we observe in life are just very complicated, and obtaining enough data to recover the deterministic relation becomes a bad bad question (because it is impossible). What is our job? We try our best to collect data, and based on the available data, we try to come up with the right question! The questions involves the model with the parameters to be estimated. The complexity of the model should match the data. What we could not be incorporated, we take them as "noise". It is our discretion to decide what is noise (the log term, or quadratic term, or that variable). This is where reasoning comes in. From research in the corresponding fields, we get a sense of what might be important and what might be negligible, and we make our assumptions accordingly. Data and assumptions are complementary not substitutable!
We are in the era of big data. There is no deny. But instead of being lazy and striving to make as few assumptions as we could, the availability of new data grants us the opportunity to ask questions that could not be asked before! Ask yourself not, what I can leave the data to do, ask yourself, what can I do with the data!
Asking the right question is an important, important skill. What is a good question? It is relevant, and we have a good attack! In statistical science, the second point means we have the right data! If we only have like 10 data points and you wish to estimate the parameters for a super-complicated models involving 10 parameters, I wish you good luck.
When I took Econometrics, I was cynical. I am unhappy with linear models. As far as I see, the relation we try to estimate could be quite non-linear. However, there is a point, that I just begin to see. For a long time, (still in many areas today), data are scarce. It will be unrealistic to do a highly structural estimation, and it makes perfect sense for us to restrict us to linear models. At least, we recover correlations. The caveat many miss is that, once the model is ill-specified, the coefficients we get might not be interpretable, or requires more thinking, and quantifiers.
If I indeed get lots of observations for each possible vector of independent variable, I could impose no structure I could simply estimate the mean of y_i for each possible vector of independent variable, and here we go. I am not saying that's the right way to do it, but it will be ok.
A related talk I heard last semester is on spatial statistics. It talks about estimating earthquake frequency. The talk discusses how fine the grid should be. The solution is let the data decide! If you happen to have lots of observations, you can have finer grids...duh.
My view is that there is no physical probability! The world is deterministic. (For example, the coin flip is not random! At Persi can toss it however he wants) There is only belief. Lots of things we observe in life are just very complicated, and obtaining enough data to recover the deterministic relation becomes a bad bad question (because it is impossible). What is our job? We try our best to collect data, and based on the available data, we try to come up with the right question! The questions involves the model with the parameters to be estimated. The complexity of the model should match the data. What we could not be incorporated, we take them as "noise". It is our discretion to decide what is noise (the log term, or quadratic term, or that variable). This is where reasoning comes in. From research in the corresponding fields, we get a sense of what might be important and what might be negligible, and we make our assumptions accordingly. Data and assumptions are complementary not substitutable!
We are in the era of big data. There is no deny. But instead of being lazy and striving to make as few assumptions as we could, the availability of new data grants us the opportunity to ask questions that could not be asked before! Ask yourself not, what I can leave the data to do, ask yourself, what can I do with the data!
Subscribe to:
Posts (Atom)