I love NPR. I listen on my way to work every morning. My most frequent parking lot moments are on Wednesdays, when I arrive at work just about the time Shankar Vedantam is reporting on recent social science research. This week, the story was about a study investigating whether or not university professors exhibit racial and gender biases in their interactions with students. The researchers sent bogus emails to about 6500 faculty in highly rated universities around the country. The emails purportedly came from students, expressing interest in the professor’s research and asking for a meeting. The emails were identical except that the “student” names were varied to indicate various racial, ethnic, and gender demographics. The outcome measured was whether or not each faculty member responded and agreed to meet with the student. The headline findings were that faculty were less likely to agree to meet non-Caucasian (black, Hispanic, Indian, Chinese) and female students. White male students were most likely to receive a meeting, and Chinese females were least likely. The report went on to say that even same-ethnicty or gender faculty exhibited these disparities toward the inquiring students.
I am always looking for stories that use a simple analysis that my introductory statistics students can understand and might be interested in. So when I got in the office, I googled the author (her name is Milkman so it was easy to remember!). I did find the paper , and it included many more findings than Vedantam had reported in his short piece. But what I found even more intriguing was the volume of citations of the article in both academic and popular press about the ethics of conducting such an experiment. One of the most critical and insistent nay-sayers was a statistician at Columbia, Andrew Gelman, who was one of the sampled faculty, and who writes a fascinating blog about data, data collection and data analysis . He feels that the unwitting subjects should be compensated for their time, since they were not provided with the opportunity for informed consent. On the other side are those who point out that revealing these kinds of inequities requires deception, and is important enough to suspend a strict interpretation of this principle. A middle ground suggested by some is that after the de-briefing, a subject should be allowed to require that their data be removed from the database, if they so choose.
But one of the more interesting points made, to me, was that there was much less concern among academics about this ethical principle when similar experiments investigated discrimination in hiring. Instead of faculty receiving bogus emails, those studies entailed human resources departments receiving bogus applications for jobs, with similar manipulation in the ethnicity of the applicant’s name. Why did we not worry so much about that? One pair of researchers whose work in this area is well cited is Bertrand and Mullainathan from University of Chicago . If you google their names, you find many discussions of the article, none of which are about its ethics. Why is that?
I am not sure of the answer to this question, but perhaps it is that some subjects of that experiment in human resources departments felt just as wronged, but don’t have time or interest in writing about topics that don’t directly related to getting their job done. Those of us in academics are paid to think, talk, and write, and we especially like a good controversy.
Another explanation might be that we think the findings can’t possibly be true, and so are looking for something to be wrong with the study. I know that was my initial reaction. In my field, if we didn’t agree to talk to Asian students about our research, we wouldn’t have many students to talk to! So, as my rationalization went, perhaps what we are seeing is the rarity factor at work. But that really wouldn’t explain the lower meeting rate for black and Hispanic students. So I am left with a feeling of disbelief/discomfort with the findings.
I’d like to hear opinions on these two questions:
1. Are you surprised by the findings of the study; and
2. Do you think the study methods are unethical?
I’ll start. #1. Yes. #2. I’m not sure.