... or at least this is so for one narrative we've tried and one type of philosophical argument.
Update, Sep 9: In a follow-up study, we have replicated the main results described below using four different arguments and four different narratives.
Introduction
From 2009 to 2015, Eric published a series of studies that consistently found, using a wide variety of measures, that ethics professors do not behave any morally better than comparison groups of other professors (summarized here). One simple explanation (favored, for example, by Jon Haidt in his discussion of Eric's work) is that philosophical moral reflection has little to no influence on people's moral behavior. However, Eric and his frequent collaborator Joshua Rust have been hesitant to draw that conclusion without more empirical evidence (see here and here for possible alternative explanations).
Meanwhile, Eric's PhD student Chris was growing interested in the power of narratives to change behavior. So we decided to collaborate on a project to compare the influence of narrative and argument on charitable giving.
Design
For our argument, we chose an argument constructed by philosopher Peter Singer with the aim of convincing ordinary readers to donate to charity. The argument was originally written for a paper in draft by Luke Buckland, Matthew Lindauer, David Rodriguez-Arias, and Carissa Veliz. (Thanks to them and to Peter Singer for sharing this argument with us.) We supplemented Singer's argument with some additional clarificatory text. It's a typical Singer-style argument, emphasizing that people have a duty to prevent death and suffering due to extreme poverty by donating money that they would otherwise spend on luxuries, and that our physical distance from those who are suffering does not excuse us from the obligation to help.
For our narrative, we chose a true story, about the same word-length as the argument, about a family rescued from slavery by a charitable donation, which we adapted from the website of a charitable organization.
Participants were 918 MTurk workers in the United States. We divided them into four conditions: (1.) narrative only, (2.) argument only, (3.) narrative + argument, and (4.) control (who received a similar-length text from a middle-school physics textbook). Participants were then asked to rate, on a 7-point scale, how much they agreed or disagreed with statements about the ethics of donation and their own motivation to donate (e.g., "It is morally good to give money to charities that help those in extreme poverty", "Currently I feel motivated to give money to a charity that helps people in extreme poverty"). In Experiment 1, we then asked them a hypothetical question about how much they would give to a charity if we offered them a bonus of $10. In Experiment 2, we replaced the hypothetical-donation language with a lottery choice. We'll report only on Experiment 2 here, but the results of Experiment 1 were very similar.
Here is the exact language for the lottery choice:
Upon completion of this study, 10% of participants in this study will receive an additional $10. You have the option to donate some portion of this $10 to one of six well-known charities that have been shown to effectively fight suffering due to extreme poverty. If you are one of the recipients of the additional $10, the portion you decide to keep will appear as a bonus credited to your Mechanical Turk worker account, and the portion you decide to donate will be given to the charity you pick from the list below. If you are one of the recipients of the additional $10, how much of your additional $10 would you like to donate?
Participants then ticked a number on a scale from $0 to $10 in dollar intervals, indicating how much they would donate if they won the $10. (A big thanks to The Life You Can Save, especially Jon Behar, for funding this study!)
Finally, we asked questions designed to measure "narrative transportation" (i.e., how immersive and moving they found the narrative, or argument, or physics text), adapted from Green and Brock 2000.
We thought we'd share some preliminary results. We emphasize that these results are only preliminary. Later, we'll want to double-check our numbers, confirm that we got the exclusions exactly right, etc. (We have a preregistration at Open Science Foundation, but we haven't made it public yet.)
Results
Mean amount donated, by condition:
(Apologies for blurry image. Click to enhance. Error bars are 95% confidence intervals.)
Statistically, the conditions are different (ANOVA [913,3], F=4.99, p = .002). In planned, pre-registered one-tailed t-tests, we found that the narrative and narrative+argument conditions generated more donation than the control condition (p = .02, .001) but the argument condition did not (p = .48). Similarly in a multiple regression analysis we found that being in one of the narrative conditions (narrative only or narrative+argument) predicted donation amount but being in one of the argumentative conditions was not statistically significantly predictive. In other words, exposure to a Singer-style philosophical argument had no statistically detectable effect on charitable giving, but exposure to a narrative about a family rescued from slavery did have a detectable effect. This was also true in Experiment 1, concerning hypothetical donation.
The results were even clearer for the measure of ethical opinion and motivation to donate, which was scored on a scale from 5 to 35, based on 5 questions:
It's not a huge effect size -- about 2 points on the scale -- but it's easier to see how the results for argument and control are similar to each other and different from the results for narrative and narrative+argument, which are also similar to each other (ANOVA [913,3], F=10.8, p < .001; one-tailed t vs. control condition: argument p = .71, narrative p < .001, nar+arg p < .001). Similarly to the lottery donation results, a regression analysis of reported motivation and moral opinion finds being in a narrative condition predictive and being in an argument condition non-predictive.
As expected, we also found that participants reported more narrative transportation in the narrative and narrative+argument conditions than in the other conditions (ANOVA, p < .001) and that narrative transport, amount given, and motivation to give were mutually correlated (transport-amount given, r = .24; transport-motivation, r = .40, motivation-amount given, r = .50; all p's < .001).
Conclusion
We regard this as preliminary evidence that exposure to at least one type of narrative influences charitable giving, motivation, and opinion, while exposure to one common type of philosophical argument has little if any influence.
In our teaching, we also seem to find that Singer-style arguments have little effect on moral opinion or motivation to donate. In another series of studies (collaborative with Peter Singer and Brad Cokelet), which we hope to be able to report on soon, professors and TAs at UC Riverside taught lower-division philosophy students philosophical arguments for charitable donation and for vegetarianism. In subsequent questionnaires, students' self-reported attitudes toward charity don't appear to have been much influenced by the arguments, but their self-reported attitudes toward vegetarianism do appear to have been influenced.
This is not to deny that some people are influenced to donate by Singer's arguments. Eric believes that he has been, for example. But the effect might be weak or uncommon or not effectively produced by brief exposures.
In follow-up research, we plan to see if we find similar results with different types of arguments and narratives, and we hope to zero in on what underlying factors are at work behind these effects.
This actually fits with my own (admittedly anecdotal) observations, that stories convince people better than arguments. One thing I'd be interested to know is whether the narrative being fictional or a true story makes any difference. I suspect, unless the story is obviously allegorical, that it makes a minimal difference, if any.
ReplyDeleteHi, this is very interesting.
ReplyDeleteLet me do a few remarks:
The narrative dealt with slavery, which is not only a very emotion-arousing topic and especially strong due to America's history. So, I guess the results would be more reliable if both argument and narrative dealt with the same topics.
It is also impressive that the combination of argument and narrative scores higher than narrative only. This difference should be attributed to the argument? (or maybe you could use another group with two narratives, to see if things add up)
I also was impressed that the groups were already very prone to donation. And, related to that, the differences are really small (one hypothetical dollar). So that neither narrative nor arguments seem to do much difference.
I'd also love to see some study focused not on the increasing-effect, but on the changing-opinion-effect. To see if narratives could change the way people who do not believe in altruism see the world? Or if for that matter arguments are more effective.
I'm looking forward to the final version,
Best,
Celso
Thanks for the comments, Self-Aware and Celso! Celso, the N+A condition was not statistically different from N alone, but of course there's might be an effect that would show up in a study with more statistical power. Change of reported opinion shows up in some of the statements about ethics of donation, which also show the same pattern when analyzed individually.
ReplyDeleteI’ve found that when I talk about philosophical arguments for charitable giving (or altruistically motivated career choice), they generally become contentious. People feel judged, pressured, or hear it as arrogant.
ReplyDeleteOn the other hand, when I present the same conclusions in the context of a story or simple statement, it generates a more agreeable response. Pure anecdote, but very consistent in my experience. It would be interesting to see the argument vs narrative presented as a “personal statement” and to gauge people’s perceptions of the person’s character or personality.
For example, I was originally planning to become a doctor, but deviated toward studying policy and economics due to philosophical and marginal benefit analysis of the two professions. When I tell people that I feel I can do more good as a policy analyst or congressional staffer than I could as a doctor, they seem to hear it as arrogant, or as putting down doctors. On the other hand, when I say “the healthcare system is very robust already, and there are already lots of great doctors. I want to help fix our broken government,” people approve much more. For short, I just say “I got more interested in the policy side of things.”
Agreeable reaction is a different metric than immediate charitable giving, of course. I wonder if outcomes differ for amount of giving, whether that money comes out of your bank account or as a percentage of possible lottery earnings. What about willingness to do some volunteer labor or spread awareness? The former might be hard to test, but I’m sure there are small tests that would work for the latter. Perhaps ask respondents if they’re willing to write a note expressing their support for the cause for a petition.
Thanks for the comment, bskubi. What you say makes a lot of sense to me!
ReplyDeleteCan you please give CI's for the *difference* in the outcome between each treatment?
ReplyDeleteDaaronr -- not double-checked, but here's a quick output from Minitab 14, one way ANOVA with Tukey comparisons including CIs:
ReplyDeleteOne-way ANOVA: Money versus Condition
Source DF SS MS F P
Condition 3 157.5 52.5 4.99 0.002
Error 913 9601.6 10.5
Total 916 9759.2
S = 3.243 R-Sq = 1.61% R-Sq(adj) = 1.29%
Individual 95% CIs For Mean Based on
Pooled StDev
Level N Mean StDev ---------+---------+---------+---------+
1 210 3.967 3.058 (-------*--------)
2 217 4.700 3.204 (--------*--------)
3 276 4.174 3.316 (------*-------)
4 214 5.037 3.361 (--------*-------)
---------+---------+---------+---------+
4.00 4.50 5.00 5.50
Pooled StDev = 3.243
Tukey 95% Simultaneous Confidence Intervals
All Pairwise Comparisons among Levels of Condition
Individual confidence level = 98.96%
Condition = 1 subtracted from:
Condition Lower Center Upper ------+---------+---------+---------+---
2 -0.072 0.734 1.540 (-------*-------)
3 -0.555 0.207 0.969 (-------*-------)
4 0.262 1.071 1.879 (-------*-------)
------+---------+---------+---------+---
-1.0 0.0 1.0 2.0
Condition = 2 subtracted from:
Condition Lower Center Upper ------+---------+---------+---------+---
3 -1.282 -0.527 0.229 (-------*------)
4 -0.465 0.337 1.139 (-------*-------)
------+---------+---------+---------+---
-1.0 0.0 1.0 2.0
Condition = 3 subtracted from:
Condition Lower Center Upper ------+---------+---------+---------+---
4 0.105 0.863 1.622 (-------*------)
------+---------+---------+---------+---
-1.0 0.0 1.0 2.0