Thus inspired, Josh and I sent phony emails to several hundred professors -- emails designed to look like they came from an undergraduate. (Yes, we got human subjects ethics approval first; and yes we're aware that in spamming philosophers we are perhaps coming uncomfortably close to being a test case for our own thesis.) One of our emails asked about the professors' office hours; another expressed interest in declaring a major and asked for the name of the undergraduate advisor. Research question: Would ethicists be more likely than the other groups to respond to the emails?
No, it turns out. Here are the response rates:
Group: 1stemail , 2ndemailThis variation is well within chance (chi-squared, p = .51, .60).
Ethicists: 59.0% , 53.6%
Non-ethicist philosophers: 58.0% , 49.8%
Non-philosophers: 54.6% , 54.1%
Interesting enough, perhaps, as confirmation of our general finding (so far) that ethicists behave no better than non-ethicists. But this study had an additional twist: Many of these same professors also completed a survey we sent them -- a survey asking them, among other things, to rate the morality of "not consistently responding to student emails" on a nine point scale from "very morally bad" through "neutral" to "very morally good". We also asked: "About what percentage of student emails do you respond to?" followed by a blank for them to enter a percentage. Thus, we could compare normative attitude, self-described behavior, and actual behavior. (We hasten to add, here, that all identifying information was removed for analysis: We are not interested in the responses of particular individuals but only of groups.)
Our survey respondents said they nearly always responded to undergraduate emails. More than half estimated that they responded to 100% of undergraduate emails. More than 90% estimated that they responded to 90% or more of undergraduate emails. On the face of it, these appear to be gross overestimates -- I'm tempted even to say, in the aggregrate, borderline delusional (though I don't doubt that there are a few very conscientious email responders out there). When I reported these numbers recently in a talk to undergraduates, they laughed out loud. Ethicists reported neither more nor less responsiveness than did the other groups.
Those who reported responding to 100% of undergraduate emails were indeed somewhat more likely to respond to both emails: 47.2% versus 29.0% for those who claimed less than 100% responsiveness (chi square, p = .003).
Oddly, however, we found no relationship whatsoever between professors' expressed attitudes about the morality of consistently responding to undergraduate emails and their actual behavior. 83.0% of professors said it was morally bad not consistently to respond to undergraduate emails, but these professors were no more likely to respond to our emails than were the 17.0% who said it was morally okay not to respond. In fact, 65.5% of those who said it was okay not to respond consistently to undergraduate emails responded to our second email, compared to only 55.3% of those who said it was bad not to respond. (This was within the range of chance variation given the smallish numbers involved in this particular set of conditions, but the 95% confidence interval for the difference in response rates tops out at a 3.2% advantage for those who think it is morally bad not to respond -- so at best they're responding at practically the same rate.)
On none of these measures did ethicists appear to respond or behave any differently, or any more or less self-consistently, than the non-ethicist philosophers or the comparison group of non-philosophers.
Interesting. I've often been skeptical of the moral assumptions behind some of your measurements. This time, my initial thought as I started reading your post was, "Great idea! Responding to student emails is certainly obligatory."
ReplyDeleteBut once I got to the actual description of your phony emails, I came to the jolting realization: "Oh, no, one needn't respond to inappropriate inquiries (e.g. redundant info that's easily available on the department website)." So I guess I don't really think that responding to all student emails is obligatory after all. I only think that responding to appropriate emails is obligatory.
I wonder how common this sort of mix-up is. I'd guess those professors were implicitly just thinking of a certain (paradigmatic) kind of student email (e.g. asking a substantive philosophical question about the class content) when claiming that they respond to 100%. You could avoid this problem by building a reminder into the survey question, e.g. "About what percentage of student emails (e.g. asking about your office hours) do you respond to?"
Very ingenious experiment!
ReplyDeleteI think that this line of research is becoming a whole new branch within experimental philosophy (and experimental social psychology)
Thanks for the kind remark, Anibal.
ReplyDeleteRichard: Yes, that is a legitimate concern. I think the office hours and undergrad advisor emails are a little different in that respect. Office hours are not, at most schools, posted on the web in any obvious way, we've found, so we think that an email request for them is entirely appropriate. Most professors list their office hours on their syllabae or announce them in class, but office hours are supposed to be open to students who aren't currently taking a class.
Undergrad advisor information is generally more readily avaiable on the website, so here I think the problem you mention comes more into play. However, (1.) I think it's still the kind thing to do, at least, even if not obligatory, to give the name of the undergrad advisor in response to such a request. (2.) The response rates for the two emails are pretty similar, so probably the non-obligatoriness of the second (if it is non-obligatory) is not a big factor.
All that said, I do acknowledge that emails of the sort we sent, especially since they appeared to come from an unknown undergraduate, would probably be less likely to garner replies than would emails from known undergraduates. So we don't take the 50-60% response rates as indicative of overall response rate to undergraduate emails.
Just to reiterate Richard's point and apply it to the "hypocrisy" aspect of the study, I was one of the so-called ethicists who took the survey and I understood the question to mean "known undergraduates." I would not have said that it is obligatory to respond to those kinds of email, nor would I have put down "100%" if I thought the question referred to those kinds of emails. Here at UH anyway, we get a lot of spam from unknown students or people purporting to be unknown students...
ReplyDeleteAside from that reservation though, it's a very cool study.
In the real world of college instruction (i.e., among, adjuncts, visitors, and anyone who doesn't work in a "research-one" department with 2-2 load or less), answering student emails -- like most service activity -- is almost entirely a cover-your-rear matter, having precious little to do with genuine courtesy, kindness, or professionalism. Not to mention the actual response-worthiness of undergraduate email, about which the less said, the better.
ReplyDeleteAlso, a sufficient response can take the form of pulling a student aside just before or after the next class period.
Interesting experiment!
ReplyDeleteWere you at all worried about spam filters?
Thanks for the continuing comments, folks!
ReplyDeleteTamler: That's a substantial concern. If most respondents misinterpreted our questions as you say, then that limits the value of studying the relationship between the survey responses and the email responsiveness. I know not every respondent interpreted the survey question as you did, since one respondent reported noticing an inconsistency between her high rating of responsiveness and the fact that she didn't respond to one of the emails we sent.
I anticipate that a number of survey respondents will object roughly as you have done, however. One question is the extent to which this is post-hoc rationalization. It seems to me that at least at most research-orienting universities one is not so flooded with email from undergraduates that there is no obligation to respond to a query about one's office hours. I suspect this is also true at most teaching-oriented universities too, though "satellite campus drone"'s comment may reflect some professors' experience.
Furthermore, even if it is not an obligation to reply to such emails, it still may be morally better to do so than not to do so, so this is still an index of moral behavior; and it's probably reasonable to take answers to the question of whether it is bad not consistently to respond to undergraduate emails as an approximate (but not perfect) index of differences in normative view about the goodness of responding to such emails.
Perhaps also self-reported percentage of emails responded to might still be taken as a rough (not perfect) index of self-perception of general responsiveness and conscientiousness about undergraduate email, even if some respondents have misinterpreted as you said. But there are also concerns about ceiling effects here and whether 100% vs. 98% is a better index of self-perceived conscientiousness or of precision in the use of percentages.
Altogether, I would grant that these are substantial concerns about the study but not fatal flaws.
Satellite campus drone: That's useful to hear, thanks.
ReplyDeleteAnon 10:52: We tested our emails using Yahoo, Google, and a couple of university spam filters and all got through. One precaution we took was to spread out the emails over time so massive batches were not going out all at once.
Eric, one thing I'm unclear about is the precise philosophical problem to which this survey is designed to contribute new philosophical insight.
ReplyDeleteOne way to look at this would be to take the results and peel away their specific situational predicates in order to obtain a more universal truth. What we get then is something like the following: People tend to hold charitable opinions of themselves in contrast to their actual conformity with their own ideals. Fine. But I'm not sure what is gained by predicating "ethicist" to "people."
Are we to have expected that ethicists are less prone to human folly than non-ethicists? But since they start off human, shouldn't we rather have expected that human folly permeates even their attempts at compensation? In other words, they'll maintain charitable opinions of themselves even concerning their ability to compensate for their self-charity.
At least, that's why the results of this survey don't surprise me.
Eric, can you post the two versions of the email you sent out?
ReplyDeleteBadda: The main research question is whether ethicists behave any morally better than do other professors. A secondary question is whether they behave any more in accordance with their espoused norms. We take for granted that ethicists are human, imperfect, and flawed, so we are just looking for trends on average.
ReplyDeleteWe believe these are important philosophical and moral psychological questions! People seem to differ in their expectations about the moral behavior of ethicists -- as Josh Rust and I have explored in our "Peer Opinion" paper (forthcoming in Mind).
Sure thing, Tamler! Let me preface this by re-emphasizing that Josh and I have no idea whether you in particular responded to the emails, or even to the survey. I would also like to emphasize that on our view failing to respond to such emails is only a small moral failing. I am inclined to see ordinary life as permeated with moral choices: How kind are you to the cashier? How welcoming of and responsive to students? Every selection of a luxury for oneself over a gift to charity is a small act of selfishness. So everyone is pervasively short of perfection. The only question is by how much.
ReplyDeleteThe text of the emails was as follows. Some people will have received both, others only one, others neither.
1st email (spring 2008): from "JR" (address hi5university@yahoo.com), subject line "Office Hours?":
Dear Prof. [last name]:
Could you please let me know your office hours this term?
Thanks!
2nd email (spring 2009): from "Ryan Harrison" (from ryharrison89@gmail.com or raharrison89@gmail.com), suject line "declaring a major":
Dear Prof. [last name]:
I'm thinking about declaring a major in [field]. Do you know who the department's undergrad advisor is?
Thanks so much!
Ryan Harrison
Your thoughts and criticisms are welcome, Tamler (or anyone else reading this comment)!
I understand the research questions but I don't understand the philosophical or even psychological problem that answering these questions is meant to illuminate. The questions are questions of brute fact along the lines of asking whether dentists brush their teeth more frequently than other doctors, and whether they behave any more in accordance with their sense of good oral hygiene. Is the "dentist" predicate sufficient to characterize these other questions as medical questions? If not, then neither is the "ethicist" predicate sufficient to characterize your questions as philosophical. What is missing here is the element of philosophical praxis.
ReplyDeleteBadda, I regard questions about the moral consequences of studying (various sorts of) philosophy as philosophical questions. For example, they can be used in arguments about the moral value of certain ways of thinking. They are also key to moral psychology which I regard at the interdisciplinary nexus between ethics and psychology.
ReplyDeleteI am also intrinsically interested in the psychology of philosophy, regardless of whether psychology of philosophy is or is not a branch of philosophy. One reason to think psychology of philosophy might itself be a species of philosophy is the example of Nietzsche, who was an excellent intuitive psychologist of philosophy and who deployed his psychological judgments in genetic arguments regarding various philosophical positions. (James and Dewey also did some of this.)
Eric, I'm not sure either (about the email, I know I took the survey). I just wanted to see whether I thought the emails were ones that, in my view, should be answered. After seeing them, my first reaction is: maybe, probably, somewhere in between, we should respond to the undergraduate advisor query (although it's an annoying email, they should be able to figure that out) but the office hours email from someone you don't know who's not in your class sounds creepy. Of course I may be rationalizing since that one sounds vaguely familiar. But I just did a search for "office hours" and it didn't come up.
ReplyDeleteAny gender differences, Eric? I would guess that (regardless of speciality) women would feel more of an obligation to respond, and also would respond more.
ReplyDeleteThanks, Tamler, for the feedback about that. We'll be happy if readers agree that, even if it's not obligatory, at least morally better to respond than not to respond to at least one of the emails.
ReplyDeleteNeil: We found no differences by gender, university type (research vs teaching oriented), or rank (controlling for age). Younger professors were marginally more likely to respond and significantly more likely to say it's bad not to respond; but they were also less likely to claim to respond to 100% of undergraduate emails. Thus, it seems their self-portrayals were more accurate.
ReplyDeleteWe also compared self-described deontologists, consequentialists, virtue ethicists, skeptics, and "no settled position". Though the numbers were relatively small, we found no differences that approached statistical significance.
Hi Eric-
ReplyDeleteI very likely said I should respond to all undergrad emails, but I surely also read it to mean all undergrad emails written by my students or some other circumscribed subset. But this does convince me that I was too ready to say "all undergrad emails." I can now state with confidence that I would not necessarily reply to undergrad emails from (1) students that were asking rude, sexist, racist, or inappropriate questions, (2) students that I had reason to believe were dead, hospitalized, or in a vegetable state, (3) students curtly inquiring about things that that make no sense given my familiarity with students, local norms, and so on.
Emails about office hours from students I don't recognize in my small college, while on sabbatical, after having been away from regular teaching for two prior years fall into (3). :-)
But those might not be typical problems with what the study was picking up on. For what it is worth, I responded to the declaring a major email, but not the office hours email. Frankly, I was surprised to learn that I never responded to the office hours email, but then I looked at the email again I remembered why: I didn't think it was a student!
I thought it was one of those annoying people who come around my office several times a term and bombard me with emails about when they can come by and swipe unused books. Indeed, there is nothing in the email to mark that it is from a student, and many of the people I have in mind will contact me with ambiguous university-ish email addresses. That, I think, is a more serious thing for any faculty at universities where they are pestered by such people, or universities where the crank factor is high. When I was visiting at Berkeley, I sometimes felt besieged by the local cranks wanting to talk to me, sending me emails, and the like. So, perhaps, I'm more likely than most to not think an unknown email address was one of those folks.
I hereby encourage you to see what happens if the email makes it clear that it is an undergrad student at the university of the professor asking about office hours.
Anyway, fascinating stuff as usual!
Thanks for the helpful comment, Manuel. Your and others' comments here are helping me see that the interpretation of our emails was less clear than we anticipated. That's one reason it's good that we tried it with a couple different emails!
ReplyDeleteBut Eric, the answers to your questions are more illuminative of the practical as opposed to the moral value of certain ways of thinking vis-a-vis what you only assume, in line with the status quo, to be the moral value of certain ways of behaving. In other words, they are less suitable to be used, as you say, "in arguments about the moral value of certain ways of thinking" than in arguments about how to produce the behaviors you value without question. The latter is a problem for social engineering, not philosophy, very much akin to the promotion of good (m)oral hygiene. On the other hand, therein lies at least some kind of praxis.
ReplyDeleteEric,
ReplyDeleteI have similar worries to those that Manuel raised. I get emails from students who want help b/c they don't like their professors, people who want to buy my books, and religious nutjobs that want to find out when my office hours are so they can trap me in my office to tell me their views about evolution and homosexuality. If I ever receive an email asking about office hours, I'd probably assume that it's from someone in this group. That being said, I'm pretty good about responding to my students emails (even the less than fully reasonable ones), so I worry about this aspect of the study.
Interesting: the gap between people's overinflated opinions of themselves and their actual conformity with their own ideals is instrumental to avoiding the perils of overpatterned behaviors.
ReplyDeleteI second Richard's concerns, and don't find your response particularly compelling. At least, it makes it even less clear what the "big picture" is here. If responding to these e-mails is just a nice thing to do, then it's not obvious that you're studying moral behavior. (Cruel people can be nice at times.) Perhaps if the point is that responding to these emails is supererogatory, then you are. But I'm rather confused about what the moral take-home point is. (Aside from the fact that I'm likely overly generous in how I rate myself in various domains, but I already knew that.)
ReplyDeleteAnother thing is that you might want to think about what you mean by "ethicist": people working in metaethics are (sometimes) going to be ethicists in a different sense than people working in bioethics, etc. If the point is to expose an interesting and distinctive inconsistency, you might want to focus on the right kind of ethicist...(alas, I don't know which kind that is...)
Eric, I propose you devise another survey to assess the perceived mental health of those ethicists who do behave, in line with folk expectations, in rigid conformity with their ideals. I predict the results will show they are complete ass holes.
ReplyDeleteBadda: How much work is "rigid" doing in your comment? Acting in conformity with one's ideals doesn't sound nearly as creepy as *rigidly* acting in conformity with one's ideals. (Was Gandhi rigid? Not that I know much about his case in particular.)
ReplyDeleteClayton and Matthew: It sounds like you and some others are more beseiged by kooks than I. My inclination is to think, however, that if you think there's a decent chance it really is from someone with a legitimate claim to want to know your office hours you could reply, as a fair number of our respondents did, with a query asking who the person is. (This, or any other reply, would count as a reply in our coding system.)
ReplyDeleteMaybe you're right that it's nice or supererogatory to do so, rather than an obligation. I'm not sure why that would be a serious problem with the study, though: Maybe we're studying supererogatory behavior or a particular virtue (which is consistent with lacking other virtues). Fine! We draw no conclusion from this one study. We have multiple studies completed and ongoing; we're looking for convergent evidence, for patterns across a wide diversity of moral behaviors.
It doesn't have to do too much work. Let's just say rigid enough that the ethicists in question are found to be more responsive to undergraduate emails than other professors.
ReplyDeleteYour initial assumption that "it's morally better, generally speaking, to respond to undergraduate emails than to ignore them" strikes me as equivalent to saying that it's morally better, generally speaking, to perform your job duties than to neglect them, so unless you are cynical of the work ethic of other professors, anyone who overperforms in this area of their job is going to have other problems. Their overperformance or overpatterning is going to cut into the patterning of other behaviors, like, you know, not being an ass hole.
Then again, I don’t know what's on the job description of professors.
Eric (if I may): Fair enough. You may be right that I overblew the difference between niceness and (say) virtue in lodging my concern.
ReplyDeleteI am, however, still interested in what you think about the more general question/issue I raised: who are you counting as an ethicist? (Those who specialize in, for example, metaethics, may identify more closely with "M&E" than with normative ethics. It's also the case that there are a diversity of views on what "moral expertise" amounts to--or, for example, what the moral expertise is which "ethics professors" have which make them particularly qualified to teach courses in ethics. Many who chime in on these matters tend to focus on notions of "descriptive" expertise--that certain people know more about "descriptive ethics" (say, through study), as well as having (through philosophical training) more skill at identifying and evaluating arguments and justifications.)
I'm interested in that issue myself, and so I do think your studies can contribute to it. Particularly, if it does turn out that "ethics professors" act more or less as good or bad--ignoring disputes about details here--in various ways, then what exactly can ethics professors be teaching (or legitimately claim to teach) in ethics class?
With Aristotle in mind, I always warn my students (especially when we start talking about virtue ethics), that if they thought I'd be teaching them everything necessary for "being good," they signed up for the wrong class...
Badda: That's an interesting point. I do confess some very tentative attraction to the idea that putting energy into being morallly good in one domain may have a negative impact on your energy for other kinds of moral behavior. Or in terms of temporal trade-offs, every student email you answer is that much less time you could be spending being a good father. This is another reason, I think, to value convergent evidence from a variety of domains. But it would be absurd -- wouldn't it? -- to think the following: Oh she answers her undergraduate emails conscientiously, she must be a total jerk!
ReplyDeleteMatthew: We have used two criteria for counting someone as an "ethicist" in our studies. One is self-report: Does the person describe herself as a "specialist" in ethics? The other -- used in the analysis we report here -- is to look at the areas of specialization listed on the individual's website. (We haven't re-analyzed the data yet using the self-description measure; I doubt the results will differ much though.)
ReplyDeleteI suspect few ethicists would want to base their authority to teach ethics classes on having superior moral virtue, but rather as you suggest on their knowledge of the literature and their skill at philosophical argumentation. Of course, it's very much less than this to claim that philosophical moral reflection tends, on average, to have positive moral effects.
On Aristotle: Don't forget that even though he said that philosophical ethics cannot make a person good if that person hasn't had the proper upbringing to start with, he still said that the aim in studying ethics is to become good!
Oh, I'm only speaking hyperbolically. She might not be a jerk but she'll certainly be weird, maybe even dysfunctional in certain respects -- that is, compared to the status quo. Not that I find anything wrong with that, which is why, elsewhere, I said it was good news that ethicists don't consistently earn high marks for ethical behavior. Otherwise ethicists would be acting in conformity with what counts as ethical behavior only to untrained observers.
ReplyDeleteOn the other hand, if their ethical behavior is found to be on par with everyone else's instead of noticeably worse -- if they are not found to be militants of a cause, however trivial, like responding to or ignoring all emails -- then I would say they are still disengaged from any kind of interesting work.
Which, by the way, cuts to the real problem I have with this study: the standard for good behavior employed is simply that which is already accepted by the vast majority of readers, which in turn makes your implied social engineering goal one of boring us deeper into the status quo.
ReplyDeleteOn Aristotle: Don't forget that even though he said that philosophical ethics cannot make a person good if that person hasn't had the proper upbringing to start with, he still said that the aim in studying ethics is to become good!
ReplyDeleteRight. But perhaps he was obliged (from the perspective of self-interest) to say that so his students wouldn't ask for a tuition refund!
No doubt, Matthew, that was one of those lawyer-imposed lines!
ReplyDeleteBadda: Since we're also looking at the normative views of ethicists, we can at least judge whether the status quo is also approved by most ethicists or whether there is, instead, some deviation in opinion. (There is a marked deviation on some questions: Ethicists are much more likely to say that eating the meat of mammals is morally bad than are professors outside of philosophy.)
Is this work continuing, by any chance? Now every time I get a suspicious sounding email from student, I wonder whether it's fake.
ReplyDeleteAnon: I would encourage you not to get overly suspicious. We have not sent out any misleading email messages since winter. Nor is it clear that we will do any further attempts with this paradigm, especially since I've already spilled the beans on the blog.
ReplyDeleteSomeone on Leiter's 'do philosophers make the world a better place?' thread made this aside: "Isn't it interesting that the metaphysicians, logicians, philosophers of language, and epistemologists are often more politically engaged than the ethicists and political theorists?"
ReplyDeleteIs this a common perception among philosophers? Do you have any data on political engagement among ethicists vs. other philosophers?
Anon: "Often" is one of those funny words. If it were "on average" it would be a bold and interesting claim, but I don't know what to make of "often".
ReplyDeleteJosh Rust and I have a voting participation study in which we found that ethicists and political philosophers in the U.S. are, on average, no more likely to vote than are other professors (though Political Science professors are more likely to vote):
http://www.faculty.ucr.edu/~eschwitz/SchwitzAbs/EthicistsVote.htm