Saturday, March 31, 2012

On Whether the Rich Are Jerks

A recent article by Paul Piff and collaborators, purporting to show that rich people are jerks (more formally: "higher social class predicts increased unethical behavior"), has been getting attention in the popular media. Numerous people have sent me the article, correctly surmising that I'd be interested given my own research on the moral behavior of (generally high socioeconomic status) ethicists.

The article nicely displays some of the difficulties of researching moral behavior.

First, let me express a thought about the article's reception. If Piff and collaborators had found no differences in behavior, it seems a reasonable conjecture that the article would have received less attention. It might even have been difficult to publish at all. The same might be true even if Piff et al had found significant results but in the opposite direction, that is, if they had found the rich better behaved. Readers' and referees' critical acumen would probably have been activated, much more so than for a sexy result that tickles our fancy. Consider the many filters that a study must pass -- from approval by one's advisor, to design, to data collection, to analysis and write-up, to refereeing, to editorial acceptance, to public dissemination. At each step, the sexy study has an advantage over less-sexy competitors. The cumulative advantage in the marketplace of ideas should make us nervous about forming our opinions based on what we see in the news. (I recognize this applies to my own research on ethics professors too. So far, my most frequently mentioned study on ethicists is the one study that found ethicists behaving worse.)

Now let's consider the methods. The authors report the results of seven different studies.

Two studies examine the rudeness of drivers. Piff et al report that people driving fancy cars are less likely to wait their turn at a four-way stop and less likely to stop for a pedestrian entering a crosswalk. While I like the real-world naturalness of this study ("ecological validity"!), this particular measure seems very likely to be subject to experimenter effects -- that is, distortion in coding and results so as to favor the hypothesis of the experimenter. Experimenter effects can be large even when there is no obvious source of bias (hence medical research typically aims to be "double blind"). In this case the sources of possible coder bias seem obvious and very difficult to control. This is especially true of the crosswalk study. A confederate of the experimenter steps out into the crosswalk, and the experimenter codes both the perceived status of the car and whether it stops. Wisely, the status of the car is coded before the experimenter knows whether it has stopped. But anyone who has been a pedestrian in the San Francisco Bay Area (where the study was conducted) knows that the crosswalk is a place of subtle communication between ped and driver: You take a step out, you catch the driver's eye. How confidently you step, the look on your face, your reaction (or not) to the driver's glance and to the change (or not) of velocity -- all this has a big effect on what happens. The results might as easily reflect the expectations of the experimenter as any real difference in driving patterns. So this is exactly the sort of case in which one would expect large experimenter effects. Since the results have only mid-grade p values (.05 > p > .01), a small experimenter effect could vitiate them entirely.

(Piff et al state that the coders and confederates were "blind to the hypothesis of the study", but it is hard to imagine that the coders don't at least have strong suspicions, given that they are being asked to code the luxuriousness of the vehicles. At the very least, this should make vehicle status very salient to them, amping up any of the coders' prior expectations of a relationship between vehicle status and driving behavior.)

How about the other studies? Studies 3 and 5 asked participants to read scenarios and then describe how ethically or unethically they would act in those scenarios. Piff et al report that participants reporting higher social class also report that they would act less ethically in such scenarios. Would it seem too fussy of me to say that I don't fully trust self-report of moral behavior in hypothetical scenarios? I would like some evidence that this isn't, say, actually a measure of honesty and frankness instead of a measure of differences in how one would really act in such scenarios, with self-reports of less moral behavior revealing more honesty and frankness than do self-reports of moral perfection. That interpretation would completely flip the moral significance of Piff et al's results. Or maybe the measure is really something more like a measure of one's opinion about one's own moral character, which might have a zero correlation with real differences in moral character (as I suggest here)?

In Study 4, after completing filler tasks, participants were offered candy from a jar ostensibly for children in a nearby laboratory. Afterwards, they were asked how many candies they had taken from the jar. Participants who had been primed to think of themselves as relatively low class (by being asked to compare themselves to the rich, the well educated, and the prestigiously employed) reported having taken less candy than participants who had been primed to think of themselves as relatively high class [edited 5/28]. "Wait, what?" I hear you asking. They reported having taken more candy? But did they actually take more candy? If I'm reading the article correctly, the experimenters chose not to measure actual theft, relying on self-report instead, though the subjects' fingers were right there in the jar! Thus, honesty is confounded with immorality, as in Studies 3 and 5. Perhaps I can also mention the weirdness of coming into a psychology lab and then being offered candy ostensibly for children elsewhere. Are participants really buying this cover story? I participated in a few psychology studies as an undergrad, and I suspect I wouldn't have bought it for a minute. Educated undergrads expect to be lied to by psychologists.

Study 6 also has cover story problems (see also my discussion of a similar study by Gino and Ariely). Participants are set in front of a computer ostensibly presenting them with the outcome of random die rolls. Participants are asked to self-report the outcome -- without the experimenter checking -- and they are told they will have a higher chance of winning a prize if they self-report higher results. I ask you to imagine yourself as a participant in this experiment. What do you think is going on? Is there a moral obligation to tell the truth? Or is the whole thing just silly? The experimenters have brought you into this weird situation in which they seem, pretty much explicitly, to be asking you to lie to them. They, of course, are themselves lying to you, as you probably suspect. The connection between behavior in this setting and real-world honesty seems dubious at best.

In Study 7, participants were either asked to list three things about their day or three benefits of greed. They were then asked to self-report whether they would engage in immoral behavior in hypothetical scenarios. Participants who had been asked to list positive features of greed said that they would engage in more immoral behavior in the hypothetical scenarios, and this was especially the case for the lower socioeconomic status participants. Therefore...? In addition to the general types of concerns raised above, I might mention that an experimental context in which a researcher is asking you to list advantages of greed might encourage the respondent to entertain certain hypotheses about the experiment that influence her answers. It might also encourage the respondent to expect a more forgiving moral atmosphere in which self-report of selfish behavior would be viewed less negatively.

Real moral behavior is hard to measure. I appreciate the difficulty of the researchers' task. Three cheers for convergent measures! I think it's cool that this is being done, and I enjoyed reading the article and thinking about the issues. But I hope I will be forgiven for not buying it in this case.

Update, May 28: Readers of the post might also be interested in this critical reaction and response (HT Rolf Degen).

10 comments:

hagop sarkissian said...

Great comments, Eric! I find your criticisms of study 6 least convincing, though. Even if we grant what you say, why the difference between rich and poor in this case? Put another way, why wouldn't *everyone* lie to improve their chances at the gift certificate? DISCLAIMER: I haven't read the study.

Eric Schwitzgebel said...

I see your point, Hagop. It's hard to know exactly what's going on given how thinly the results are reported. Here are two possibilities: SES is measured in part by educational attainment. More educated participants might be more likely to think critically about the validity of the psychological study. Also SES relates with undergraduate major. Certain majors (e.g., business and econ) might take a very different attitude toward the experimental set up than other majors (e.g., English). Those are just two of the more obvious possibilities.

Baron P said...
This comment has been removed by the author.
Ben Bronner said...

I must be missing something. Putting aside Eric's cogent criticisms, studies 4 and 7 seem to support the opposite conclusion as is drawn by the researchers. The overall conclusion is that the rich act more immorally, but in study 4 those primed to think of themselves as low SES stole more candy (assuming self-report is accurate) and in study 7 the connection between listing positive features of greed and reporting immoral hypothetical behavior was stronger for low SES subjects. So even if there were no methodological problems, why would the researchers think that studies 4 and 7 support the conclusion that the rich are jerks?

Eric Schwitzgebel said...

Ben, thanks for your comment. In fact, I misstated the results of Study 4 in the original post, which I have now amended. Whoops! On Study 7, the researchers' hypothesis is that the opinion that greed is good is the mediating variable. When primed to think that the greed is good, the lower SES people start to act like higher SES people -- who, I guess, are assumed to already think that greed is good, since the greed-is-good prompt doesn't change their behavior. I'm pretty dubious about this, but that's the thinking, anyway.

Anonymous said...

Ok, now I see re: study 7. Thanks!

Anonymous said...

I haven't read the article but let me throw out a from the hip thought stimulated by reading your summary and critique.

The referenced studies seem to look for statistical behavioural differences between Rich/Non-Rich in moral test situations.

But to get to conclusions about morality (at least all things considered morality) another step seems to be at work: data that show no Rich/Non-Rich behavioural differences in the test situations are taken as evidence for the groups being equally moral.

But wait. How is that last part justified? Compare: in some scenarios it would seem to me more all things considered morally flawed of Rich (compared to Non-Rich) to not give resource aid to a needy person. More flawed because Rich has more surplus resources to spend.

So if a test on such situations show Rich/Non-Rich aiding to an equal degree then I'd take that as evidence for Rich being less moral.

Such reasoning might be extendable to any test situation where the behavioural cost to the subject can at least later be compensated for by the subject spending economic resources.

Eric Schwitzgebel said...

I agree with that point, Anon.

Jon Jermey said...

There's always the confounding variables of cause and effect, too, even if the findings are valid: it may not be that richer people have become less moral, but rather that less moral people become richer.

Eric Schwitzgebel said...

I totally agree with that, JonJ! Can't randomly assign people to wealth.