I have been listening to Harvard Open Course on Justice. It has been an awesome experience, though from time to time I get deeply troubled by some of the responses students give. What troubles me most is in the class where Rawlsian idea of justice was debated.
One guy named Mike argued that the society should be based on meritocracy. To illustrate his point, he gave an example of college admission, where he argued forcefully that he worked hard, and put in effort, so he deserve to be admitted based on his merit alone.
To begin with, I must confess I embrace meritocracy, a topic for another day, but I am uncomfortable with the fact that he is so confident that his performance and his admission is due solely to his own merit (you need to watch the video or at least hear the conversation to see that conviction), and oblivious to the fact that he is fortunate to have many who helped him during the process.
His intrinsic assumption is that his excellence today, which secured him a place in Harvard, is all his doing, his merit and his effort. Let us put aside whether merit or talent is arbitrary in themselves, it is a mistake to think our performance or excellence is a deterministic function of our decisions, talents or other merits. Whenever I reflect upon my past, I am often amazed at how fragile this path has been, and how much uncertainty it involves---I was extremely lucky to meet numerous people, who were so kind to give me a helping hand when I need them, provide guidance when I am lost, and share their wisdom in discussions. They never had to. My teachers in high school and professors in college never had to discuss ideas with me to foster my interest and kindle my curiosity, they never had to spend time writing an enthusiastic recommendation letter which were essential for my admission to Williams, and then to Harvard and Stanford. Without their enthusiasm, things would be very different.
I doubt if anyone has a different story. To say the very least, everyone in Harvard got in with at least a strong recommendation letter. Your recommender does not have to write such a good one. It takes time. Even if you are extremely talented, he still does not have to. It is a favor done for you. Don't take it for granted.
I think when people are given some favor, even for an arbitrary reason, they get used to it, and think somehow they deserve it. It is because of this, we feel entitled to everything we earn. But we don't. We owe a lot to those who helped us. We are lucky to get those help.
This blog is about uncertainty, which shapes our world more profoundly than we typically recognize. It is via recognition of its role, can we understand our proper role, can we shred undue arrogance, can we fully appreciate our fortune that makes us who we are today.
My hope is from now on, I can write a series of blogs on several issues and dedicate them to the people who mostly influenced my thinking, and inspired my interest. Ten years later, I can look back at those blogs, I will not lose perspective of who have helped me along the way, will not falsely think that I deserve everything, and will be reminded of the responsibility I have because I owe so many people.
Saturday, November 23, 2013
Friday, November 22, 2013
A Letter from UC Berkeley Professor
In response to some strike activity at US Berkeley, a Math Professor sent the following letter to his students. It is a powerful letter, and it is worth reading through. I feel his message is so important, that I am obliged to summarize his point for those who do not have time to read.
He offers to cover the sections for his students since two teaching assistant will be on strike, and he refuses to cancel his class. He then moves on to give his rationals----The world we live in is complicated, and we need to solve lots of difficult and complicated questions now and in the future. Students in UC Berkeley are immensely talented, and they should have the responsibility to tackle those questions in the future. Because of that, their education is extremely important for the future task:
Dear All,
As some of you may have heard, there is some strike activity taking place on campus tomorrow.
I want to let you know that I will not be striking, which means that I will be, so-to-speak, crossing a picket line. Moreover, I know that two of your GSIs have decided to strike, but because I happen to be free in the afternoon when they teach, and because I enjoy teaching smaller classes from time to time and I haven’t had a chance to in a while, I’ll be covering those sections. If you were planning to see me at office hours tomorrow afternoon, then feel free to come to one of the sections I’ll be covering. I will be in Stephens 230c from 2:10 to 4pm, Cory Hall 285 from 4:10pm to 5pm, and Evans Hall 6 from 5:10pm-6pm.
The reason for me taking this decision is extremely simple: We have 7 class days left until the end of the course. Despite the fact that we've made good time and are likely to finish the syllabus with a few lectures in hand for review, class hours are valuable and your education is too important to just cancel a class if we don’t have to. Whatever the alleged injustices are that are being protested about tomorrow, it is clear that you are not responsible for those things, whatever they are, and I do not think you should be denied an education because of someone else’s fight that you are not responsible for. I say this with no disrespect whatsoever to the two GSIs who have decided to strike. Societies where people stand up for what they believe in are generally better than societies where people do not, sometimes dramatically so. Further, I cannot discount the possibility that I may be in the wrong on this and they may be right. I have certainly been on the wrong side of political judgements before and I’m sure I will be again. However from a practical point of view I’ve made my decision and you should all turn up to class and discussion tomorrow as normal.
Beyond practical matters, I think it’s also worth reflecting a little on the broader relationship between politics and your education, and I think I have some important things to share on this topic that may be helpful to you.
I do this with some trepidation. Normally I try to avoid talking about politics with my students and also my professional colleagues because people have a wide variety of views, sometimes held with great conviction and feeling. If I was to get into a political disagreement with one of you or one of my colleagues, it might get in the way of or distract us from the central mission we have of working together to give you a great education.
However sometimes political events reach into our lives without our invitation or control, and we have no choice but to engage with each other about politics. Many times in history it has done so with far more violence and disruption than a strike, and it is wise to be psychologically prepared for this fact.
If I’ve learned one thing about politics since I was your age, it is this: Politics, like most things in life worth thinking about, including mathematics, is very big, very complicated, and very interconnected. I’ve lived and worked in four countries on four continents, all with societies set up differently both politically and socially. I’ve discovered that there is no unique or obviously best way of setting up society. For every decision and judgement you reach, there are people who benefit and people who lose out. It’s the same with the way I teach my classes. I know that for every decision I make about how to teach you there are some of you who benefit and there are others who would do better if I did things differently. There is no way of getting around that. Every judgement you make in life is a question of balancing different interests and ideals. Reasonable good people can disagree on political questions like whether to strike or not, and they can disagree about far more contentious topics also.
All this may sound like speaking in platitudes. However it is a point worth making to all of you because you are so young. One of the nice things about being young is that your thinking can be very clear and your mind not so cluttered up with memories and experiences. This clarity can give you a lot of conviction, but it can also lead you astray because you might not yet appreciate just how complicated the world is. As you get older you tend to accumulate life experiences to learn from, and this is the source of wisdom, but the trouble is that the lessons we glean from life do not all point in the same direction. Sometimes it is hard to tease the correct learning from the experiences life throws at us.
So what are we to do with the fact that when we are young we lack a lot of the perspective we need to make definitive judgements about what is right, but that as we get older our judgements tend to be informed by our experiences, and these experiences guide us in contradictory ways, both between different people and within the same person?
I don’t know.
However one thing I do know is that you are not going to be able to avoid making these kinds of judgements, just as I cannot avoid making a judgment about whether to strike or not. Like it or not, I have to make a political choice, and I have to talk to you about it. For me, the choice not to strike is quite easy, but for you the kinds of judgements and choices you are going to face in your lives are going to be far from easy; they are going to be of a complexity and importance that will rival that faced by any previous generation. To an extent that you may not yet appreciate, the world is changing incredibly quickly. In just a decade, since I was your age, the internet and telecommunications has truly transformed the way we live, not just in rich countries but around the world. When I was an undergraduate, if I wanted to check my email I went to a little room in the basement to use a computer, and if I wanted to learn something I went to a library. The kinds of breakthroughs we are seeing in biotechnology remind me of the way people were talking about electricity in 1900. Of course I don’t know - nobody knows - but my guess is that biotechnology in the 21st century could be similarly transformative to the way the full power of electricity only hit prime-time in the 20th century. The recent controversy about the NSA has shown that the role of information technology on society can be, or at least might become, double edged. There is climate change, another controversial and difficult topic, the exact impact of which we do not yet know. These are just a few of the challenges we can see, and we should remember that history has a habit of throwing curve balls at each generation that nobody saw coming. And among all this tumult, our search for common human peace and happiness on some level becomes more difficult, though no less important. A previous generation dodged the bullet of nuclear armageddon when things looked bleak, but for your generation the bullets are coming thicker and faster than ever before. The potential all of you in your generation are going to have for both good and harm is tremendous.
I suspect many of you have heard sentiments along these lines before. However I also suspect that many of you will think something in response along the lines of `I know all that, but these things are for someone else to figure out, not me.’
That is a mistake.
One of the things you can lose track of when you attend a top tier university like Berkeley is just how exceptional and amazing you really are. I’m blown away every time I talk to you. The way you ask penetrating questions, the way you improved so much between midterm 1 and 2, the way you challenge me to be a better teacher, it just knocks my socks off. You really are amazing. I’ve taught students all over the world, and I’ve never seen a group of students so talented. I’m not just talking about some of you. I’m talking about all of you. It’s a privilege to be your professor. Sadly, however, I know many of you don’t feel that way. The difficulty you all face is that as you look around at all your fellow students, it’s easy to have your eye drawn by people doing better than you. Or rather, I should say people who look like they’re doing better than you. In reality the true extent of how much people are learning can be difficult to measure. Sometimes failures and adversity are better preparations for long term success than effortless progress.
Why am I telling you all this?
I’m telling you this because you all need to know that there is not some great pool of amazing people in some other place who are going to shape the way our species navigates the coming decades. The simple fact is that, like it or not, technology is going to change the way we live in the future, and you’re going to have to solve some very hard problems, as well as figure out how best to use new technology for good, while at the same time facing human dangers that have haunted humanity throughout history.
Part of the work of your generation is going to be technological, using scientific ideas to serve the interests of society, and part of the work is going to be fundamentally human, tied inexorably with qualities of the human condition - human emotion - that dominate the whole of history. These things are not separate, but are inexorably linked, and you are in a better place to understand that connection than me.
I can’t tell you what your particular role should be in the new realities of the 21st century. It’s up to you to decide if you want to make the focus of your life technological, focused on new innovations to drive society forward, or essentially human, focused on the age-old struggles of trying to get along, work together, and find happiness, or some combination of the two.
However I can tell you this:
Whatever you decide to do with your life, it’s going to be really, really complicated.
Science and technology is complicated. History and politics is complicated. People are complicated. Figuring out how to be happy, and do simple things like take care of our kids and maintain friendships and relationships, is complicated.
In order for you to navigate the increasing complexity of the 21st century you need a world-class education, and thankfully you have an opportunity to get one. I don’t just mean the education you get in class, but I mean the education you get in everything you do, every book you read, every conversation you have, every thought you think.
You need to optimize your life for learning.
You need to live and breath your education.
You need to be *obsessed* with your education.
Do not fall into the trap of thinking that because you are surrounded by so many dazzlingly smart fellow students that means you’re no good. Nothing could be further from the truth.
And do not fall into the trap of thinking that you focusing on your education is a selfish thing. It’s not a selfish thing. It’s the most noble thing you could do.
Society is investing in you so that you can help solve the many challenges we are going to face in the coming decades, from profound technological challenges to helping people with the age old search for human happiness and meaning.
That is why I am not canceling class tomorrow. Your education is really really important, not just to you, but in a far broader and wider reaching way than I think any of you have yet to fully appreciate.
He offers to cover the sections for his students since two teaching assistant will be on strike, and he refuses to cancel his class. He then moves on to give his rationals----The world we live in is complicated, and we need to solve lots of difficult and complicated questions now and in the future. Students in UC Berkeley are immensely talented, and they should have the responsibility to tackle those questions in the future. Because of that, their education is extremely important for the future task:
And do not fall into the trap of thinking that you focusing on your education is a selfish thing. It’s not a selfish thing. It’s the most noble thing you could do.Learn well, it is not just a luxury, it is a responsibility.
Society is investing in you so that you can help solve the many challenges we are going to face in the coming decades, from profound technological challenges to helping people with the age old search for human happiness and meaning.
Dear All,
As some of you may have heard, there is some strike activity taking place on campus tomorrow.
I want to let you know that I will not be striking, which means that I will be, so-to-speak, crossing a picket line. Moreover, I know that two of your GSIs have decided to strike, but because I happen to be free in the afternoon when they teach, and because I enjoy teaching smaller classes from time to time and I haven’t had a chance to in a while, I’ll be covering those sections. If you were planning to see me at office hours tomorrow afternoon, then feel free to come to one of the sections I’ll be covering. I will be in Stephens 230c from 2:10 to 4pm, Cory Hall 285 from 4:10pm to 5pm, and Evans Hall 6 from 5:10pm-6pm.
The reason for me taking this decision is extremely simple: We have 7 class days left until the end of the course. Despite the fact that we've made good time and are likely to finish the syllabus with a few lectures in hand for review, class hours are valuable and your education is too important to just cancel a class if we don’t have to. Whatever the alleged injustices are that are being protested about tomorrow, it is clear that you are not responsible for those things, whatever they are, and I do not think you should be denied an education because of someone else’s fight that you are not responsible for. I say this with no disrespect whatsoever to the two GSIs who have decided to strike. Societies where people stand up for what they believe in are generally better than societies where people do not, sometimes dramatically so. Further, I cannot discount the possibility that I may be in the wrong on this and they may be right. I have certainly been on the wrong side of political judgements before and I’m sure I will be again. However from a practical point of view I’ve made my decision and you should all turn up to class and discussion tomorrow as normal.
Beyond practical matters, I think it’s also worth reflecting a little on the broader relationship between politics and your education, and I think I have some important things to share on this topic that may be helpful to you.
I do this with some trepidation. Normally I try to avoid talking about politics with my students and also my professional colleagues because people have a wide variety of views, sometimes held with great conviction and feeling. If I was to get into a political disagreement with one of you or one of my colleagues, it might get in the way of or distract us from the central mission we have of working together to give you a great education.
However sometimes political events reach into our lives without our invitation or control, and we have no choice but to engage with each other about politics. Many times in history it has done so with far more violence and disruption than a strike, and it is wise to be psychologically prepared for this fact.
If I’ve learned one thing about politics since I was your age, it is this: Politics, like most things in life worth thinking about, including mathematics, is very big, very complicated, and very interconnected. I’ve lived and worked in four countries on four continents, all with societies set up differently both politically and socially. I’ve discovered that there is no unique or obviously best way of setting up society. For every decision and judgement you reach, there are people who benefit and people who lose out. It’s the same with the way I teach my classes. I know that for every decision I make about how to teach you there are some of you who benefit and there are others who would do better if I did things differently. There is no way of getting around that. Every judgement you make in life is a question of balancing different interests and ideals. Reasonable good people can disagree on political questions like whether to strike or not, and they can disagree about far more contentious topics also.
All this may sound like speaking in platitudes. However it is a point worth making to all of you because you are so young. One of the nice things about being young is that your thinking can be very clear and your mind not so cluttered up with memories and experiences. This clarity can give you a lot of conviction, but it can also lead you astray because you might not yet appreciate just how complicated the world is. As you get older you tend to accumulate life experiences to learn from, and this is the source of wisdom, but the trouble is that the lessons we glean from life do not all point in the same direction. Sometimes it is hard to tease the correct learning from the experiences life throws at us.
So what are we to do with the fact that when we are young we lack a lot of the perspective we need to make definitive judgements about what is right, but that as we get older our judgements tend to be informed by our experiences, and these experiences guide us in contradictory ways, both between different people and within the same person?
I don’t know.
However one thing I do know is that you are not going to be able to avoid making these kinds of judgements, just as I cannot avoid making a judgment about whether to strike or not. Like it or not, I have to make a political choice, and I have to talk to you about it. For me, the choice not to strike is quite easy, but for you the kinds of judgements and choices you are going to face in your lives are going to be far from easy; they are going to be of a complexity and importance that will rival that faced by any previous generation. To an extent that you may not yet appreciate, the world is changing incredibly quickly. In just a decade, since I was your age, the internet and telecommunications has truly transformed the way we live, not just in rich countries but around the world. When I was an undergraduate, if I wanted to check my email I went to a little room in the basement to use a computer, and if I wanted to learn something I went to a library. The kinds of breakthroughs we are seeing in biotechnology remind me of the way people were talking about electricity in 1900. Of course I don’t know - nobody knows - but my guess is that biotechnology in the 21st century could be similarly transformative to the way the full power of electricity only hit prime-time in the 20th century. The recent controversy about the NSA has shown that the role of information technology on society can be, or at least might become, double edged. There is climate change, another controversial and difficult topic, the exact impact of which we do not yet know. These are just a few of the challenges we can see, and we should remember that history has a habit of throwing curve balls at each generation that nobody saw coming. And among all this tumult, our search for common human peace and happiness on some level becomes more difficult, though no less important. A previous generation dodged the bullet of nuclear armageddon when things looked bleak, but for your generation the bullets are coming thicker and faster than ever before. The potential all of you in your generation are going to have for both good and harm is tremendous.
I suspect many of you have heard sentiments along these lines before. However I also suspect that many of you will think something in response along the lines of `I know all that, but these things are for someone else to figure out, not me.’
That is a mistake.
One of the things you can lose track of when you attend a top tier university like Berkeley is just how exceptional and amazing you really are. I’m blown away every time I talk to you. The way you ask penetrating questions, the way you improved so much between midterm 1 and 2, the way you challenge me to be a better teacher, it just knocks my socks off. You really are amazing. I’ve taught students all over the world, and I’ve never seen a group of students so talented. I’m not just talking about some of you. I’m talking about all of you. It’s a privilege to be your professor. Sadly, however, I know many of you don’t feel that way. The difficulty you all face is that as you look around at all your fellow students, it’s easy to have your eye drawn by people doing better than you. Or rather, I should say people who look like they’re doing better than you. In reality the true extent of how much people are learning can be difficult to measure. Sometimes failures and adversity are better preparations for long term success than effortless progress.
Why am I telling you all this?
I’m telling you this because you all need to know that there is not some great pool of amazing people in some other place who are going to shape the way our species navigates the coming decades. The simple fact is that, like it or not, technology is going to change the way we live in the future, and you’re going to have to solve some very hard problems, as well as figure out how best to use new technology for good, while at the same time facing human dangers that have haunted humanity throughout history.
Part of the work of your generation is going to be technological, using scientific ideas to serve the interests of society, and part of the work is going to be fundamentally human, tied inexorably with qualities of the human condition - human emotion - that dominate the whole of history. These things are not separate, but are inexorably linked, and you are in a better place to understand that connection than me.
I can’t tell you what your particular role should be in the new realities of the 21st century. It’s up to you to decide if you want to make the focus of your life technological, focused on new innovations to drive society forward, or essentially human, focused on the age-old struggles of trying to get along, work together, and find happiness, or some combination of the two.
However I can tell you this:
Whatever you decide to do with your life, it’s going to be really, really complicated.
Science and technology is complicated. History and politics is complicated. People are complicated. Figuring out how to be happy, and do simple things like take care of our kids and maintain friendships and relationships, is complicated.
In order for you to navigate the increasing complexity of the 21st century you need a world-class education, and thankfully you have an opportunity to get one. I don’t just mean the education you get in class, but I mean the education you get in everything you do, every book you read, every conversation you have, every thought you think.
You need to optimize your life for learning.
You need to live and breath your education.
You need to be *obsessed* with your education.
Do not fall into the trap of thinking that because you are surrounded by so many dazzlingly smart fellow students that means you’re no good. Nothing could be further from the truth.
And do not fall into the trap of thinking that you focusing on your education is a selfish thing. It’s not a selfish thing. It’s the most noble thing you could do.
Society is investing in you so that you can help solve the many challenges we are going to face in the coming decades, from profound technological challenges to helping people with the age old search for human happiness and meaning.
That is why I am not canceling class tomorrow. Your education is really really important, not just to you, but in a far broader and wider reaching way than I think any of you have yet to fully appreciate.
Friday, November 15, 2013
Admission Bias?
Update: I realized there is a coding error in my previous version, and I have corrected it. The result now is not as dramatic, but I think it is still striking. I mistakenly plotted the conditional (on wealth) probability of admission, when we are interested in what is the implied percentage of student population based on wealth. I apologize for my mistake.
Let $Y_i$ be the relevant performance of an individual for admission process. We have
\[
Y_i=\beta_x X_i+\beta_z Z_i+e_i
\]
where $X_i$ is talent, $Z_i$ is work ethics, and $e_i$ is pure luck.
Now we consider the family wealth of that student $T_i$:
\[
T_i=\alpha_x X'_i+\epsilon'_i
\]
where $X'_i$ is the talent of the parents, and $\epsilon$ is all the other factors. We would expect there to be a inter-generational correlation of talents. We could decompose $X_i$ into the inherited part ($E(X_i|X'_i)$) and the innovation part $\xi'_i$. For simplicity, however, we will decompose in the following way:
\[
X_i=E(X_i|T_i)+\xi_i=\delta T_i+\xi_i
\]
This means that talent is not independent of family wealth simply because
1) talent has positive inter-generational correlation;
2) higher talent means higher wealth level in expectation.
Now we can write the performance in the following way
\[
Y_i=\beta T_i+e'_i
\]
Note that $\beta=\beta_x\cdot \delta$, and $e'_i=\beta_z Z_i+e_i+\xi_i$.
Technical fuss: I know wealth does not come from Normal distribution, but consider a function acting on wealth to make the output normally distributed. For example, log-normal distribution is considered to be a good approximation for the distribution of wealth, so instead of saying $T_i$ is wealth, we could say $T_i$ models the log of wealth, which then becomes normally distributed. If you work through the argument, you will find none of these matters.
So the rest is a Bayesian exercise. Conditional on observing getting admitted to Harvard $Y_i>cutoff$, what is the conditional distribution of $T_i$? I calculated the cutoff and did the corresponding calculation, (no worry, I will show you the code and how I calculate the cutoff in the end for transparency). Here is what I find.
The conditional distribution is highly skewed to the right---that is conditional on admission through this fair process, we would expect most of the students come from rich families. Here is the distribution:
The above plot is the percentage of students admitted as a function of income. On the y-axis, it is the percentage of admitted student with respect to total college student population---to find the population percentage among admits, you need to multiply the number by 1000. Note that unconditional $T_i$ comes from $N(0,1)$. Notice the lack of admission in the lower income group until 1 standard deviation below mean.
Another graph. What is the probability of admission in each income group?
As you can see, as the family income goes up, the chance of getting admitted is monotonically increasing. According to our model, this correlation is not causal (family wealth does not cause better performance), and is a result of the centrality of talent.
Ok, let me give you a number in the end. What will the predicted percentage of students coming from the top 4.5% of income distribution? 49.4238%
I am of the minority view that in empirical work, we need to think harder about the structure of the problem. As those tiny details matter for policy. We need to know the industry we are talking about, understand its structure, and use data to give as precise a picture of the industry. We could gain a lot from reduced-form estimation, but in some applications, without forcing us to be clear the data generating process and the structure, and link them with data, the policy implication could be very limited.
So for the first equation
\[
Y_i=\beta_x X_i+\beta_z Z_i+e_i
\],
I let $\beta$'s to be 1 and all variables come from iid $N(0,4)$. That is talent accounts only 1/3 of the performance. (I am using variance 4 to reduce fractions later in calculations)
then for family wealth,
\[
T_i=\alpha_x X'_i+\alpha_\epsilon \epsilon'_i
\]
I let $\alpha$'s to be $\frac{1}{\sqrt{2}}$, and both random variables come from $N(0,4)$. That is talent accounts for only half of the variation in wealth.
Finally, for inherited talent, I assume the correlation is only 1/2.
\[
X'_i=\frac{1}{\sqrt{2}} X_i+\frac{1}{\sqrt{2}}\xi_i
\]
The weird scaling is to make sure that the variance of talent is steady over generation.
If you work through the algebra, you will find the following regression function:
\[
X_i=\frac{1}{2} T_i+\frac{\sqrt{3}}{2}\epsilon'_i
\]
substituting in, we have
\[
Y_i=\frac{1}{2}T_i+\frac{\sqrt{3}}{2}\epsilon'_i+Z_i+e_i
\]
We can pack the last three terms as one error with variance 11, and $\frac{1}{2}T_i$ as scaled wealth $\sim N(0,1)$. In essence we have
\[
Y_i=\tilde{T}_i+\sqrt{11}\tilde{\epsilon}_i
\]
where both random variables are iid standard normal.
What will the predicted percentage of students coming from the top 4.5% of income distribution be? 22.56%.
Feel free to try different numbers.
So to make this transparent, I will show you what I did.
How to calculate cutoff? The number of students enrolled in Harvard is 6700, which means 6700/4 students each year. In US about 25 m students enroll in colleges each year. Of course there are other great universities, like Princeton, Williams (yes!), Yale, Stanford. So let me be generous, and say Harvard get the top 0.1%. So with this we can find the 99.9 quantile in terms of Y, which is about 6.18. The rest is coding it up (I used Mathematica):
In[49]:= cutoff = Quantile[NormalDistribution[0, 2], 1 - 10^(-3)];
In[50]:= N[cutoff]
Out[50]= 6.18046
In[51]:= F[x_] :=
NIntegrate[
PDF[NormalDistribution[0, 1], x] PDF[NormalDistribution[0, Sqrt[3]],
y - x], {y, cutoff, +\[Infinity]}]
In[52]:= Plot[F[x], {x, 0, 5}]
In[43]:= wealth = Quantile[NormalDistribution[], 0.955]
Out[43]= 1.6954
In[54]:= ratio = NIntegrate[F[x], {x, wealth, +\[Infinity]}]/0.001
Motivation:
A friend of mine emailed me a note she wrote on Harvard's admission status. The main finding is that the admitted students tend to come disproportionately from wealthy families, suggesting that, despite the bind-need policy, there is still a selection bias towards the rich. For example, the author found:While I do agree certain selection criterion like so called leadership, and versatility are unfair to the poor, I think we need to dig deeper into the selection process and data to get a better sense. This phenomonon, could be the result of fair admission process with no bias towards the poor. What do I mean by this?About 150 students come from family with less than \$20,000 income, that is about 2.2\% in Harvard’s 6,700 undergraduate population. In the whole country, there are about 20% of families with household income below \$20,000 (Census, 2012)..... 40\% of Harvard graduates presumably come from families with annual income higher than [200,000]. But in the US, only 4.5\% families make more than \$200,000. 40\% of Harvard’s students come from the top 4.5\% of America’s family income spectrum.
Theoretical Framework:
Let me illustrate with a model.Let $Y_i$ be the relevant performance of an individual for admission process. We have
\[
Y_i=\beta_x X_i+\beta_z Z_i+e_i
\]
where $X_i$ is talent, $Z_i$ is work ethics, and $e_i$ is pure luck.
Now we consider the family wealth of that student $T_i$:
\[
T_i=\alpha_x X'_i+\epsilon'_i
\]
where $X'_i$ is the talent of the parents, and $\epsilon$ is all the other factors. We would expect there to be a inter-generational correlation of talents. We could decompose $X_i$ into the inherited part ($E(X_i|X'_i)$) and the innovation part $\xi'_i$. For simplicity, however, we will decompose in the following way:
\[
X_i=E(X_i|T_i)+\xi_i=\delta T_i+\xi_i
\]
This means that talent is not independent of family wealth simply because
1) talent has positive inter-generational correlation;
2) higher talent means higher wealth level in expectation.
Now we can write the performance in the following way
\[
Y_i=\beta T_i+e'_i
\]
Note that $\beta=\beta_x\cdot \delta$, and $e'_i=\beta_z Z_i+e_i+\xi_i$.
A calibration and result
Let us do some calibration. We will scale $e'_i$ so that $\beta=1$. We let $e'_i\sim N(0,3)$ and $T_i\sim N(0,1)$.Technical fuss: I know wealth does not come from Normal distribution, but consider a function acting on wealth to make the output normally distributed. For example, log-normal distribution is considered to be a good approximation for the distribution of wealth, so instead of saying $T_i$ is wealth, we could say $T_i$ models the log of wealth, which then becomes normally distributed. If you work through the argument, you will find none of these matters.
So the rest is a Bayesian exercise. Conditional on observing getting admitted to Harvard $Y_i>cutoff$, what is the conditional distribution of $T_i$? I calculated the cutoff and did the corresponding calculation, (no worry, I will show you the code and how I calculate the cutoff in the end for transparency). Here is what I find.
The conditional distribution is highly skewed to the right---that is conditional on admission through this fair process, we would expect most of the students come from rich families. Here is the distribution:
The above plot is the percentage of students admitted as a function of income. On the y-axis, it is the percentage of admitted student with respect to total college student population---to find the population percentage among admits, you need to multiply the number by 1000. Note that unconditional $T_i$ comes from $N(0,1)$. Notice the lack of admission in the lower income group until 1 standard deviation below mean.
Another graph. What is the probability of admission in each income group?
As you can see, as the family income goes up, the chance of getting admitted is monotonically increasing. According to our model, this correlation is not causal (family wealth does not cause better performance), and is a result of the centrality of talent.
Ok, let me give you a number in the end. What will the predicted percentage of students coming from the top 4.5% of income distribution? 49.4238%
Discussion
What is the point? The point is not that the education system is fair. The point is we need to be more careful in our diagnosis. Let me give some far-fetched general comments, not validated by the work I did in this blog. The problem might very well lie in the inequality in elementary education and secondary education, and if that is the case, targeting higher education admission process is unlikely to be effective. Efforts like affirmative action aiming to change the result, without addressing the root of the problem, is not only doomed to fail but also likely to create new problems.I am of the minority view that in empirical work, we need to think harder about the structure of the problem. As those tiny details matter for policy. We need to know the industry we are talking about, understand its structure, and use data to give as precise a picture of the industry. We could gain a lot from reduced-form estimation, but in some applications, without forcing us to be clear the data generating process and the structure, and link them with data, the policy implication could be very limited.
Robustness Check and Technical addendum:
How I choose the parameters for the random variables? Well, I did some exercise finding a reasonable range for the parameters. As it turned out, they tell similar stories. so I choose the parameters that is simple for expository purposes. I hate blackbox, and I will disclose the workings, which lead to a different set of parameters (slightly more complicated), and show you that the result is basically the same.So for the first equation
\[
Y_i=\beta_x X_i+\beta_z Z_i+e_i
\],
I let $\beta$'s to be 1 and all variables come from iid $N(0,4)$. That is talent accounts only 1/3 of the performance. (I am using variance 4 to reduce fractions later in calculations)
then for family wealth,
\[
T_i=\alpha_x X'_i+\alpha_\epsilon \epsilon'_i
\]
I let $\alpha$'s to be $\frac{1}{\sqrt{2}}$, and both random variables come from $N(0,4)$. That is talent accounts for only half of the variation in wealth.
Finally, for inherited talent, I assume the correlation is only 1/2.
\[
X'_i=\frac{1}{\sqrt{2}} X_i+\frac{1}{\sqrt{2}}\xi_i
\]
The weird scaling is to make sure that the variance of talent is steady over generation.
If you work through the algebra, you will find the following regression function:
\[
X_i=\frac{1}{2} T_i+\frac{\sqrt{3}}{2}\epsilon'_i
\]
substituting in, we have
\[
Y_i=\frac{1}{2}T_i+\frac{\sqrt{3}}{2}\epsilon'_i+Z_i+e_i
\]
We can pack the last three terms as one error with variance 11, and $\frac{1}{2}T_i$ as scaled wealth $\sim N(0,1)$. In essence we have
\[
Y_i=\tilde{T}_i+\sqrt{11}\tilde{\epsilon}_i
\]
where both random variables are iid standard normal.
What will the predicted percentage of students coming from the top 4.5% of income distribution be? 22.56%.
Feel free to try different numbers.
Appendix:
So to make this transparent, I will show you what I did.
How to calculate cutoff? The number of students enrolled in Harvard is 6700, which means 6700/4 students each year. In US about 25 m students enroll in colleges each year. Of course there are other great universities, like Princeton, Williams (yes!), Yale, Stanford. So let me be generous, and say Harvard get the top 0.1%. So with this we can find the 99.9 quantile in terms of Y, which is about 6.18. The rest is coding it up (I used Mathematica):
In[49]:= cutoff = Quantile[NormalDistribution[0, 2], 1 - 10^(-3)];
In[50]:= N[cutoff]
Out[50]= 6.18046
In[51]:= F[x_] :=
NIntegrate[
PDF[NormalDistribution[0, 1], x] PDF[NormalDistribution[0, Sqrt[3]],
y - x], {y, cutoff, +\[Infinity]}]
In[52]:= Plot[F[x], {x, 0, 5}]
In[43]:= wealth = Quantile[NormalDistribution[], 0.955]
Out[43]= 1.6954
In[54]:= ratio = NIntegrate[F[x], {x, wealth, +\[Infinity]}]/0.001
Sunday, November 3, 2013
Cleaning the Wound
I had a bike crash yesterday. Something dirty got under my skin. Fearing pain, I only did a superficial wash and applied some antibiotics.
Unfortunately, this morning, I discovered that the site is swollen with pus coming out---a clear sign of ongoing infection. I had to cut open the wound and thoroughly clean it with a scrub. It was quite painful.
I knew I could not just bury those things away. I need to clean it eventually, and it gonna be more painful if I do it later. But I was too much of a wimp to cut it open and clean it yesterday. I am not even sure if I cleaned it thoroughly enough today.
Unfortunately, this morning, I discovered that the site is swollen with pus coming out---a clear sign of ongoing infection. I had to cut open the wound and thoroughly clean it with a scrub. It was quite painful.
I knew I could not just bury those things away. I need to clean it eventually, and it gonna be more painful if I do it later. But I was too much of a wimp to cut it open and clean it yesterday. I am not even sure if I cleaned it thoroughly enough today.
Sunday, October 27, 2013
Research
If I have to pick one single most important thing I have learned in college, it will be an easy choice. It is not an answer to any question. It is a question that I learnt to ask: "Who cares?" or "So what?" I am grateful for being instilled that no-nonsense ideal.
This has since defined my attitudes toward research. As Ed Leamer nicely put, we need to think if that research has any value beyond "mere mathematical amusement".
This has since defined my attitudes toward research. As Ed Leamer nicely put, we need to think if that research has any value beyond "mere mathematical amusement".
Saturday, October 26, 2013
An epidemic in China
Two pieces of news from Forbes first.
China's Zoomlion Tangles With Feisty Newspaper Over Fraud Claims
Arrested Reporter in Zoomlion Case Admits To Taking Bribes And Inaccurate Reporting
Underlying these two pieces of news is the real big challenge for Chinese Economic development---loose accounting standard and dishonest accounting practices. It is a epidemic haunting China's business community. The tax system is partly to blame together with the loose enforcement, which makes a prisoners' dilemma---no one can afford to be honest.
I happen to know some accountants in China and what they told me is shocking beyond the kind of news you will see. I will quote anonymously what one accountant told me.
"Accounting is a stressful job. I hesitated a lot before taking another accounting job after I retired. It is just you have to make up the books when you do the job, otherwise, no one will hire you. The trouble is the accounts will be kept for ten more years, and I do not want to be haunted by the possibility of getting arrested five years down the road....audition is not the way to go either. You are too naive. Audition is super lucrative, but that does not come from its legal pay. You are certain to find problems in the books you audit, and you need to look the other way. That is where the money comes from. Also, why would anyone ask you to audit when they know you will find the problem and they can hire others to give their book a nod? That stupid certificate, is a stamp of approval, worth so much money. You can lease it out to others and earn a good profit without doing anything. Of course, that is when everything goes well."
He added, "do you understand? You are probably too young to understand".
Actually, it is not such a complicated explanation. You hear that a lot when it comes to the explanation for financial crisis. The incentive for rating agencies.
I think there are a small number of problems that account for most of our troubles. Sometimes we do not know how to solve them. Sometimes, we understand how to solve them very well, but the political system will not allow us to solve them.
China's Zoomlion Tangles With Feisty Newspaper Over Fraud Claims
Arrested Reporter in Zoomlion Case Admits To Taking Bribes And Inaccurate Reporting
Underlying these two pieces of news is the real big challenge for Chinese Economic development---loose accounting standard and dishonest accounting practices. It is a epidemic haunting China's business community. The tax system is partly to blame together with the loose enforcement, which makes a prisoners' dilemma---no one can afford to be honest.
I happen to know some accountants in China and what they told me is shocking beyond the kind of news you will see. I will quote anonymously what one accountant told me.
"Accounting is a stressful job. I hesitated a lot before taking another accounting job after I retired. It is just you have to make up the books when you do the job, otherwise, no one will hire you. The trouble is the accounts will be kept for ten more years, and I do not want to be haunted by the possibility of getting arrested five years down the road....audition is not the way to go either. You are too naive. Audition is super lucrative, but that does not come from its legal pay. You are certain to find problems in the books you audit, and you need to look the other way. That is where the money comes from. Also, why would anyone ask you to audit when they know you will find the problem and they can hire others to give their book a nod? That stupid certificate, is a stamp of approval, worth so much money. You can lease it out to others and earn a good profit without doing anything. Of course, that is when everything goes well."
He added, "do you understand? You are probably too young to understand".
Actually, it is not such a complicated explanation. You hear that a lot when it comes to the explanation for financial crisis. The incentive for rating agencies.
I think there are a small number of problems that account for most of our troubles. Sometimes we do not know how to solve them. Sometimes, we understand how to solve them very well, but the political system will not allow us to solve them.
Chinese Romanticism or stupidity
China has undergone gigantic changes in recent years, especially in economic areas. The market economy reform has disrupted the old system, leaving many disgruntled. Many complaints about the current outcomes are unfair for three reasons. 1) The current outcomes compared to the old unsustainable outcomes 2)The comparison ignores the gains and focuses on the losses alone 3) The comparison is made with the rest of the world, partially imagined by the Chinese to justify their complaints.
I will give some examples to illustrate how ridiculous such perceptions can get.
Education. People in China have so much complaints about their education. They think it sucks. The American education, on the other hand, rocks. American high school kids are stress-free, they do not have to work hard, they still get into good colleges and earn good money. That is how a Chinese people think. Our school teaches Maths and Science that is useless. American never need to learn that, and they are fine. So our school system is so outdated. Mea culpa. I once thought so as well. Indeed even after I came to US and realize that the US education system is far worse than the ideal I imagined it to be, I still held a bias against my hometown education (though I was never aware of it). I was shocked when I read in NYT the other day that Shanghai public secondary schools topped the world charts in the 2009 PISA (Program for International Student Assessment). When I told this to my parents, their response is "really? that is weird!"
Health care. Chinese people think they have the most expensive health care. They feel doctors earn too much. Look at Americans, their health care is much more affordable and the poor do not have to worry about not being able to go to the hospital. Nothing could be farther from the truth. I am not sure how they would react if I tell them that it cost me \$364 to get a chest X-ray, which cost a bit above 100 RMB (\$20). It is revealing to observe that when Chinese students in US go back for vacation, they rush to the dentist to get their wisdom teeth out, regardless they are impacted or not.
Housing prices. People complain about housing prices. yes, no doubt the housing prices are high. But it is unfair comparison. Americans pay property tax, but there is no property tax in China. So when you compare property prices, you have to add in the present value of all future tax payment. Also, you cannot compare Shanghai to Philadelphia. NYC is probably the only legit comparison. Many so called cities in US are nothing but small towns in China in terms of population. The demand for housing is simply no comparison. Also there is enormous difference in infrastructure. Cities in China means unparalleled public school system, convenient public transportation, better health care, and safety. The cities in US tend to have lots of problems like crime-ridden slums, inner city schools. With the only exception of NYC, the public transportation is no comparison. Even for NYC, their public transportation is outdated when compared to most cities in China.
Automobile ownership. China does tax automobile ownership a lot---this is done via expensive motor registration. The right to register is auctioned off in places like Shanghai in limited quantity, and people just bid up the prices. In Shanghai at least, that right to register could be more expensive than the car itself. I actually applaud this. I think in cities in China, the sheer number of automobiles create huge negative externalities in terms of pollution and congestion. This externatlity is best internalized via taxation, not only in car ownership and oil tax. In US, car ownership in cities is usually taxed in another form---the difficulty to find a parking place. It is so prohibitive that some people think twice before driving to cities. This tax is prevalent---even in the tiny "city" of Seattle, you will need to drive around for quite a while to get a roadside parking place. Finally it is funny to note that while a shanghainese will complain that the government do so little to alleviate the congestion problem and the pollution problem and complain that it is so expensive for him to buy another car.
Subscribe to:
Posts (Atom)