A Response to Critiques of Cushman's and My Work on Philosophers' Susceptibility to Order Effects

The order in which moral dilemmas are presented matters to people's judgments and can substantially influence later judgments about abstract moral principles. This is true even among professional ethicists with PhD's in philosophy. In 2012 and 2015, Fiery Cushman and I published empirical evidence supporting these claims. We invite a metaphilosophical conclusion: If even professional philosophers' expert judgments are easily swayed by order of presentation, then such judgments might not be stable enough to serve as secure grounds for philosophical theorizing.

Synthese has recently published two critiques of the literature on order effects in philosophy, which address Fiery's and my work (HT Wesley Buckwalter). Both critiques make valuable points. However both also admit of some clear replies.

To fix ideas, consider two versions of the famous Trolley Problem:

Push: A runaway boxcar is headed toward five people it will kill if nothing is done. Jane can stop the boxcar by pushing a hiker with a heavy backpack in front of the boxcar, killing him but saving the five.

Switch: A runaway boxcar is headed toward five people it will kill if nothing is done. Vicki can stop the boxcar by flipping a switch to divert it to a sidetrack where it will kill one person instead of the five.

Fiery and I presented Push-type and Switch-type scenarios (fleshed with a bit more detail) to professional philosophers and two comparison groups of non-philosophers. We found that when professional philosophers saw a Push-type scenario before a Switch-type scenario, 73% rated the two scenarios equivalently on a 7-point scale. Then later in the questionnaire when asked about the Doctrine of the Double Effect -- a moral principle often interpreted implying that Push-type cases are morally worse than Switch-type cases -- only a minority, 46%, endorsed that principle. In contrast, among philosophers who saw Switch before Push only 54% rated the two scenarios equivalently, and then later a majority, 62%, endorsed the Doctrine of the Double Effect. Endorsement of the principle thus seemed to shift, post-hoc, to rationalize philosophers' order-manipulated judgments about the scenarios.

We found similar effects for Action-Omission, Moral Luck, and "Asian disease" type cases (though not consistently for every measure across the board). Philosophers with PhDs and self-reported competence or specialization in ethics showed no smaller effects than other philosophers or than comparison groups of non-philosophers -- and in fact trended slightly (non-significantly) toward showing larger order effects.

In general, we found pretty substantial effect sizes, suggesting substantial instability of judgment even in philosophical respondents' areas of expertise. Hence the metaphilosophical worry.

----------------------------------------------

Critique by Zachary Horne and Jonathan Livengood.

Horne and Livengood make three main points about the literature on order effects in philosophy:

(A.) First, they helpfully distinguish between what they call "updating effects" and "genuine ordering effects". Genuine ordering effects, in their terminology, are effects measured only after all the stimuli have been presented. "Updating effects" are measures taken along the way, and might well reflect participants' learning. There is of course nothing irrational in judging Scenario B differently as a result of seeing Scenario A because one learned something by seeing Scenario A. Most philosophical research on order effects, they note, takes the measures along the way -- and thus might be measuring learning rather than true order effects.

(B.) Second, they point out that perceptual judgments also show order effects. Thus, if we are to reject any type of evidence that shows order effects, then we must reject perceptual evidence too, which would lead to radical skepticism.

(C.) Third, they point out that order can sometimes reasonably make a difference to the evaluation of evidence. For example, a smile followed by a frown, on the same person's face, is a different type of evidence than a frown followed by a smile.

On (A): I find the labels tendentious (since if we know there isn't learning-type updating going on, what we might want to call "genuine order effects" can plausibly be measured mid-stream), however it probably is correct that most studies do not sufficiently rule out the possibility of learning or updating in the course of the experiment, if they have novice participants and take the measurements after each scenario rather than after both scenarios. However, since our participants were experts, we think it unlikely that a significant number learned anything in the process of our brief experiment that would rationally justify shifting their judgment about the equivalency or non-equivalency of Push and Switch. And as Horne and Livengood note, our measure of endorsement of the Doctrine of the Double Effect is a measurement of a "genuine ordering effect" even by their own lights.

On (B): Yes, of course it would be silly to reject all means of learning that are subject to any order effects! The epistemic sting, as they note, depends not on the mere existence of an order effect in one case, but on how large and how prevalent the order effects are. This is an open empirical question. But the limited empirical evidence that exists suggests that order effects are substantial and prevalent in moral dilemma cases. So far, we have found order effects in all of the scenario types we've tried, with about a 10-20% shift in opinion on the moral equivalency of our scenario pairs and in preference for the risky option in the "Asian disease" cases.

On (C): It's interesting to consider cases in which earlier evidence rightly colors our reaction to later evidence, but trolley problems presented to disciplinary experts seems a different kind of case.

Finally, Horne and Livengood suggest that exposure to a pair of dilemmas in our study is unlikely to have a long-lasting impact on professional philosophers' beliefs. I agree. They continue, "But if there is no long-lasting impact, then we think the effect is unlikely to matter to actual philosophical practice outside of the laboratory" (p. 17). I don't think this follows. Fiery's and my view is not that philosophers' opinions are permanently influenced by the order in which the scenarios are presented on any single occasion, but rather that their opinions are unstable -- possibly influenced one direction on one occasion, in another direction on another occasion. This instability is what drives the metaphilosophical worry.

----------------------------------------------

Critique by Regina Rini:

Rini -- a recent guest blogger here at the Splintered Mind -- looks only at our 2012 study. (Our 2015 study wasn't published until after her paper was in press.) She finds it plausible that if professional philosophers were already familiar with these cases they would not exhibit order effects of the sort Fiery and I find. She suggests that perhaps respondents were not previously familiar with the cases -- or at least not familiar in the right sort of way. She calls this the "familiarity problem" and offers four possible explanations:

(1.) The respondents were not really experts. She wonders if our participants, recruited through the internet, really had the degrees they claimed to have.

(2.) The respondents didn't carefully attend to our scenarios. Maybe they breezed through them so quickly that they failed to notice relevant features.

(3.) The respondents might not have familiar responses to these types of scenarios. Perhaps they have so far refrained from forming judgments on such cases and principles.

(4.) The respondents might not have diachronically stable familiar responses. This is the explanation Fiery and I favor. However, Rini helpfully points out that as long as philosophers are aware that their responses are not diachronically stable, the metaphilosophical threat is reduced: Presumably philosophers who are aware that their responses are not stable would be reluctant to ground their theorizing on those responses.

On (1): I am not aware of a general problem in the survey literature of respondents' frequently misreporting their educational status -- though certainly a bit of misreporting is possible. One specific piece of evidence against this possibility in our own study is that we recruited philosophers mostly by asking department chairs to forward a recruitment email to faculty and graduate students in their departments. Most of our "philosopher" participants took the survey within just a few days of these emails.

On (2): The median response time on the first scenario was 40 seconds, on the second scenario was 34 seconds. While these are not huge response times, if you stop to count out 34 seconds now, you'll probably notice that it's a reasonable amount of time for a thoughtful response to a brief scenario.

On (3) and (4): These are potentially quite serious issues, and in fact our follow-up study in 2015 was designed specifically to address them, after we saw an early version of Rini's critique. In our 2015 study we specifically asked participants if they were previously familiar with the scenarios. We also asked whether they regarded themselves as "having had a stable opinion" about the issues before participating in the experiment, and whether they regarded themselves as experts on those very issues. We also added a "reflection" condition to help address concern (2). In the reflection condition we asked participants to reflect carefully before responding and enforced a minimum 15-second delay between when participants reported having finished reading the scenario and when their response options appeared.

We did not find that self-reported familiarity or stability reduced the size of order effects in two different types of scenario pairs (trolley problems and risky-choice "Asian disease"-type problems), nor did we find reduced order effects in the reflection condition compared to a normal control condition without special instructions to reflect.

For example, percentage rating the Push and Switch scenarios equivalently:

Thus, I am inclined to think that Rini's fourth suggestion is the most plausible -- that participants do not have diachronically stable familiar responses, despite high levels of expertise. But since those who report having stable responses were no less subject to order effects than were those who reported not having stable responses, self-knowledge of stability appears to be largely absent. Despite Rini's interesting suggestion that instability is metaphilosophically non-threatening if people are aware of it, Fiery's and my results suggest that we should not hasten to that comfort.

----------------------------------------------

Both Horne and Livengood and Rini emphasize that we only have very limited evidence about order effects on professional philosophers' judgments. I agree! Fiery's and my two studies are hardly decisive. Convergent evidence from several different labs would be necessary before drawing any confident conclusions, especially if those conclusions are at variance with what one feels one knows from personal experience. Rini also makes positive suggestions for follow-up experimental work that might be done, which I am inclined to support. Both critiques raise important methodological concerns that ought to help shape and direct future work on this topic.

The Los Angeles Times

... has been publishing philosophers' op-eds recently -- a couple by me (here and here), and this past week Harry Frankfurt on why inequality isn't immoral and an adaptation of Regina Rini's Splintered Mind guest post on microaggression.

The new op-ed editor Juliet Lapidos is behind this trend. Encourage Juliet by sharing the LA Times philosophy links widely and by sending the LA Times your best op-ed queries. It would be terrific if this trend could stick and we could have another major U.S. newspaper that regularly publishes philosophers!

Microaggression and the Culture of Solidarity

A guest post by Regina Rini

If you are on a college campus or read anxious thinkpieces, you’ve probably heard about ‘microaggression’. A microaggression is a relatively minor (hence ‘micro’) insult to a member of a marginalized group, perceived as damaging to that person’s full standing as social equal. Examples include acting especially suspicious toward people of color or saying to a Jewish student, "Since Hitler is dead, you don’t have to worry about being killed by him any more." A microaggression is not necessarily a deliberate insult, and any one instance might be an honest mistake. But over time a pattern of microaggression can cause macro harm, by continuously reminding members of marginalized groups of their precarious social position.

A recent paper by sociologists Bradley Campbell and Jason Manning claims that talk of microaggression signals the appearance of a new moral culture: a ‘culture of victimhood’. In the paper Campbell and Manning present a potted history of western morality. First there was a ‘culture of honor’, which prized physical bravery and took insults to demand an aggressive reply. Picture two medieval knights glowering at one another, swords drawn. Then, as legal institutions grew stronger, the culture of honor was displaced by a ‘culture of dignity’, in which individuals let minor insults slide, and reported more serious offenses to impartial authorities. Picture a 1950s businessman calmly telling the constable about a neighbor peeking in windows. Finally, there is now an emerging ‘culture of victimhood’, in which an individual publicly calls attention to having been insulted, in hopes of rallying support from others and inducing the authorities to act. Picture a queer Latina student tweeting about her professor’s perceived-to-be homophobic and racist comments.

There is a serious problem with Campbell and Manning’s moral history, and exposing this problem helps us to see that the ‘culture of victimhood’ label is misleading. The history they provide is a history of the dominant moral culture: it describes the mores of those social groups with greatest access to power. Think about the culture of honor, and notice how limited it must have been. If you were a woman in medieval Europe, you were not expected or permitted to respond to insults with aggression. Even if you were a man, but of low social class, you certainly would not draw your sword in response to insult from a social superior. The ‘culture of honor’ governed relations among a small part of society: white men of equally high social status.

Now think about the culture of dignity, which Campbell and Manning claim “existed perhaps in its purest form among respectable people in the homogenous town of mid-twentieth century America.” Another thing that existed among the ‘respectable people’ in those towns was approval of racial segregation; ‘homogenous towns’ did not arise by accident. People of color, women, queer people, immigrants – none could rely upon the authorities to respond fairly to reports of mistreatment by the dominant group. The culture of dignity embraced more people than had the culture of honor, but it certainly did not protect everyone.

The cultures of honor and dignity left many types of people formally powerless, with no recognized way of responding to moral mistreatment. But they did not stay quiet. What they did instead was whisper to one another and call one another to witness. They offered mutual recognition amid injustices they could not overcome. And sometimes, when the circumstances were right, they made sure that their mistreatment would be seen by everyone, even by the powerful. They sat in at lunch counters that refused to serve them. They went on hunger strike to demand the right to vote. They rose up and were beaten down at Stonewall when the police, agents of dignity, moved in.

The new so-called ‘culture of victimhood’ is not new, and it is not about victimhood. It is a culture of solidarity, and it has always been with us, an underground moral culture of the disempowered. In the culture of solidarity, individuals who cannot enforce their honor or dignity instead make claim on recognition of their simple humanity. They publicize mistreatment not because they enjoy the status of victim, but because they need the support of others to stand strong, and because ultimately public discomfort is the only route to redress possible. What is sought by a peaceful activist who allows herself to be beaten by a police officer in front of a television camera, other than our recognition? What is nonviolent civil disobedience, other than an expression of the culture of solidarity?

If the culture of solidarity is ancient, then what explains the very current fretting over its manifestation? One answer must be social media. Until very recently, marginalized people were reliant on word of mouth or the rare sympathetic journalist to document their suffering. Yet each microaggression is a single small act that might be brushed aside in isolation; its oppressive power is only visible in aggregate. No journalist could document all of the little pieces that add up to an oppressive whole. But Facebook and Twitter allow documentation to be crowdsourced. They have suddenly and decisively amplified the age-old tools of the culture of solidarity.

This is a development that we should welcome, not fear. It is good that disempowered people have new means of registering how they are mistreated, even when mistreatment is measured in micro-units. Some of the worries raised about ‘microaggression’ are misplaced. Campbell and Manning return repeatedly to false reporting of incidents that did not actually happen. Of course it is bad when people lie about mistreatment – but this is nothing special about the culture of solidarity. People have always abused the court of moral opinion, however it operated. An honor-focused feudal warlord could fabricate an insult to justify annexing his brother’s territory. A 1950s dignitarian might file a false police report to get revenge on a rival.

There are some more serious worries about the recent emergence of the culture of solidarity. Greg Lukianoff and Jonathan Haidt suggest that talk of microaggression is corrosive of public discourse; it encourages accusations and counter-accusations of bad faith, rather than critical thinking. This is a reasonable thing to worry about, but their solution, that “students should also be taught how to live in a world full of potential offenses” is not reasonable. The world is not static: what is taught to students now will help create the culture of the future. For instance, it is not an accident that popular support for marriage equality was achieved about 15 years after gay-straight alliances became a commonplace in American high schools and colleges. Teaching students that they must quietly accept racist and sexist abuse, even in micro units, is simply a recipe for allowing racist and sexist abuse to continue. A much more thoughtful solution, one that acknowledges the ongoing reality of oppression as more than an excuse for over-sensitive fussing, will be required if we are to integrate recognition of microaggression into productive public discourse.

There is also a genuine question about the moral blameworthiness of microaggressors. Some microaggressions are genuine accidents, with no ill intent on the part of the one who errs. Others are more complex psychological happenings, as with implicit bias. Still others are acts of full-blooded bigotry, hiding behind claims of misunderstanding. The problem is that outsiders often cannot tell which is which – nor, in many cases, can victims. And being accused of acting in micro-racist or micro-sexist ways is rarely something people receive without becoming defensive; it is painful to be accused of hurting others. We need a better way of understanding what sort of responsibility people have for their small, ambiguous contributions to oppression. And we need better ways of calling out mistakes, and of responding to being called out. These are all live problems for ethicists and public policy experts. Nothing is accomplished by ignoring the phenomenon or demanding its dismissal from polite conversation.

The culture of solidarity has always been with us – with some of us longer than others. It is a valuable form of moral community, and its recent amplification through social media is something we should welcome. The phenomena it brings to light – microaggression among them – are real problems, bringing with them all the difficulties of finding real solutions. But if we want our future moral culture to be just and equal, not merely quietly dignified, then we will have to struggle for those solutions.

Thanks to Kate Manne, Meena Krishnamurthy, and others for helping me think through the ideas of this post. Of course, they do not necessarily endorse everything I say.

image credit: ‘Hands in Solidarity, Hands of Freedom’ mural, Chicago IL. Photo by Terence Faircloth

Philosophical Conversations

a guest post by Regina Rini

You’re at a cocktail reception and find yourself talking to a stranger. She mentions a story she heard today on NPR, something about whether humans are naturally good or evil. Something like that. So far she’s just described the story; she hasn’t indicated her own view. There are a few ways you might respond. You might say, "Oh, that’s interesting. I wonder why this question is so important to people." Or you might say, "Here’s my view on the topic… What do you think?" Or maybe you could say "Here's the correct view on the topic… Anyone who thinks otherwise is confused."

It’s obvious that the last response is boorish cocktail party behavior. Saying that seems to be aimed at foreclosing any possible conversation. You’re claiming to have the definitive, correct view, and if you’re right then there’s no point in discussing it further. If this is how you act, you shouldn’t be surprised when the stranger appears disconcerted and politely avoids talking to you anymore. So why is it that most philosophy books and papers are written in exactly this way?

If we think about works of philosophy as contributing to a conversation, we can divide them up like this. There are conversation-starters: works that present a newish topic or question, perhaps with a suggestive limning of the possible answers, but without trying to come to a firm conclusion. There are conversation-extenders: works that react to an existing topic by explaining the author’s view, but don’t try to claim that this is the only possibly correct view and clearly invite response from those who disagree. And there are conversation-enders: works that try to resolve or settle an existing debate, by showing that one view is the correct view, or at least that an existing view is definitively wrong and must be abandoned.

Contemporary analytic philosophy seems to think that conversation-enders are the best type of work. Conversation-starters do get some attention, but usually trying to raise a new topic leads to dismissal by editors and referees. "This isn’t sufficiently rigorous", they will say. Or: "What’s the upshot? Which famous –ism does this support or destroy? It isn’t clear what the author is trying to accomplish." Opening a conversation, with no particular declared outcome, is generally regarded as something a dilettante might do, not what a professional philosopher does.

Conversation-extenders also have very little place in contemporary philosophy. If you merely describe your view, but don’t try to show that it is the only correct view, you will be asked "where is your argument?" Editors and referees expect to see muscularity and blood. A good paper is one that has "argumentative force". It shows that other views "fail" - that they are "inadequate", or "implausible", or are "fatally flawed". A good paper, by this standard, is not content to sit companionably alongside opposed views. It must aim to end the conversation: if its aspirations are fully met, there will no need to say anything more about the topic.

You might object here. You might say: the language of philosophy papers is brutal, yes, but this is misleading. Philosophers don’t really try to end conversations. They know their opponents will keep on holding their "untenable" views, that there will soon be a response paper in which the opponent says again the thing that they’ve been just shown they "cannot coherently say". Conversation-enders are really conversation-extenders in grandiose disguise. Boxers aren’t really trying to kill their opponents, and philosophers aren’t really trying to kill conversations.

But I think this objection misses something. It’s not just the surface language of philosophy that suggests a conversation-ending goal. That language is driven by an underlying conception of what philosophy is. Many contemporary analytic philosophers aspire to place philosophy among the ‘normal sciences’. Philosophy, on this view, aims at revealing the Truth – the objective and eternal Truth about Reality, Knowledge, Beauty, and Justice. There can be only one such Truth, so the aim of philosophy really must be to end conversations. If philosophical inquiry ever achieves what it aims at, then there is the Truth, and why bother saying any more?

For my part, I don’t know if there is an objective and eternal Truth about Reality, Knowledge, Beauty, and Justice. But if there is, I doubt we have much chance of finding it. We are locally clever primates, very good at thinking about some things and terrible at thinking about others. The expectation that we might uncover objective Truth strikes me as hubristic. And the ‘normal science’ conception of philosophy leads to written work that is plodding, narrow, and uncompanionably barbed. Because philosophy aims to end conversations, and because that is hard to do, most philosophy papers take on only tiny questions. They spin epicycles in long-established arguments; they smash familiar –isms together in hopes that one will display publishably novel cracks. If philosophy is a normal science, this makes perfect sense: to end the big conversation, many tiny sub-conversations must be ended first.

There is another model for philosophical inquiry, one which accepts the elusiveness of objective Truth. Philosophy might instead aim at interpretation and meaningfulness. We might aspire not to know the Truth with certainty, but instead to know ourselves and others a little bit better. Our views on Reality, Knowledge, Beauty, and Justice are the modes of our own self-understanding, and the means by which we make our selves understood to others. They have a purpose, but it is not to bring conversation to a halt. In fact, on this model, the ideal philosophical form is the conversation-opener: the work that shows the possibility of a new way of thinking, that casts fresh light down unfamiliar corridors. Conversation-openers are most valuable precisely because they don’t assume an end will ever be reached. But conversation-extenders are good too. What do you think?

The spirit of this post owes a lot to Robert Nozick, especially the introduction to his book Philosophical Explanations. Thanks to Eden Lin and Tim Waligore for helping me track down Nozick’s thoughts, and to several facebook philosophers for conversing about these ideas.

image credit: Not getting Involved by Tarik Browne

Ethics, Metaethics, and the Future of Morality

a guest post by Regina Rini

Moral attitudes change over generations. A century ago in America, temperance was a moral crusade, but you’d be hard-pressed to find any mark of it now among the Labor Day beer coolers. Majority views on interracial and same-sex relationships have swung from one pole to the other within the lifetimes of many people reading this. So given what we know of the past, we can say this about the people of the future: their moral attitudes will not be the same as ours. Some ethical efforts that now belong to a minority – vegetarianism, perhaps – will become as ubiquitously upheld as tolerance for interracial partnerships. Other moral matters that seem urgent now will fade from thought just as surely as the temperance movement. We can’t know which attitudes will change and how, but we do know that moral change will happen. Should we try to stop it – or even to control it?

Every generation exercises some control over the moral attitudes of its children, through the natural indoctrination of parenting and the socialization of school. But emerging technologies now give us unprecedented scope to tailor what the future will care about. From social psychology and behavioral economics we increasingly grasp how to design institutions so that the ‘easy’ or ‘default’ choices people tend to adopt coincide with the ones that are socially valuable. And as gene-editing eventually becomes a normal part of reproduction, we will be able to influence the moral attitudes of generations far beyond our own children. Of course, it is not as simple as ‘programming’ a particular moral belief; genetics does not work that way. But we might genetically tinker with brain receptivity for neurotransmitters that affect a person’s readiness to trust, or her preference for members of her own ethnic group. We won’t get to decide precisely how our descendants come to find their moral balance – but we could certainly put our thumb on the scale.

On one way of looking at it, it’s obvious that if we can do this, we should. We are talking about morality here, the stuff made out of ‘should’s. If we can make future generations more caring, more disposed to virtue, more respectful of rational agency, more attentive to achieving the best outcomes – however morality works, we should help future people to do what they should do. That is what ‘should’ means. This thought is especially compelling when we realize that some of our moral goals, like ending racism or addressing the injustices climate change will bring, are necessarily intergenerational projects. The people of the future are the ones who will have to complete the moral journey we have begun. Why not give them a head start?

But there are also reasons to think we should not interfere with whatever course the future of morality might take. For one thing, we ought to be extremely confident that our moral attitudes are the right ones before we risk crimping the possibilities for radical moral change. Perhaps we are that confident about some issues, but it would be unreasonable to be so sure across the board. Think, for example, about moral attitudes toward the idea of ownership of digital media: whether sampling, remixing, and curating count as forms of intellectual theft. Already there appear to be generational splits on this topic, driven by technology that emerged only in the last 30 years. Would you feel confident trying to preordain moral attitudes about forms of media that won’t be invented for a century?

More insidiously, there is the possibility that our existing moral attitudes already reflect the influence of problematic political ideologies and economic forces. If we decide to impose a shape on the moral attitudes of the future, then the technology that facilitates this patterning will likely be in the hands of those who benefit from existing power structures. We may end up creating generations of people less willing to question harmful or oppressive social norms. Finally, we should consider whether any attempt to direct the development of morality is disrespectful to the free agency of future people. They will see our thumbprint on their moral scale, and they may rightly resent our influence even as they cannot escape it.

One thing these reflections bring out is that philosophers are mistaken when they attempt to cleanly separate metaethical questions (what morality is) from normative ethical questions (what we should do). If this clean separation was ever tenable, our technologically expanded control over the future makes it implausible now. We face imminent questions about what we should do – which policies and technologies to employ, to which ends – that depend upon our answers to questions about what morality is. Are there objective moral facts? Can we know them? What is the relationship between morality and human freedom? The idea that metaethical inquiry is for dusty scholars, disconnected from our ordinary social and political lives, is an idea that fades entirely from view when we look to the moral future.

--------------------------------------

I’ve drawn on several philosophers’ work for some of the arguments above. For arguments in favor of using technology to direct the morality of future generations, see Thomas Douglas, “Moral Enhancement” in the Journal of Practical Ethics; and Ingmar Persson and Julian Savulescu, Unfit for the Future. For arguments against doing so, see Bernard Williams, Ethics and the Limits of Philosophy (chapter 9); and Jurgen Habermas, The Future of Human Nature.

image credit: ’Hope in a better future’ by Massimo Valiani