Philosophers' Honesty in Responding to Questionnaires

Last week, and in various previous posts, I've discussed a questionnaire Josh Rust and I sent to several hundred ethicists and non-ethicist professors (both inside and outside philosophy), soliticiting self-reports of their moral attitudes and moral behavior on a variety of issues, such as vegetarianism and voting. Our guiding question: Do ethicists behave any better, or any more in accord with their espoused principles, than do non-ethicists? Based on our analyses so far, it doesn't look like ethicists' behavior is any better.

You might wonder, though -- as I do -- how honestly our survey respondents are answering our questions. Are those who have behaved (at least by their own lights) less than ideally well really going to report that fact, even in an anonymous survey like ours? Maybe ethicists really do behave better than non-ethicists but don't look that way because they respond more honestly. Josh and I tried to get a handle on this, in part, by asking a few questions whose answers we could verify. Respondents' honesty on these questions might help us estimate the honesty of their responses overall. Since honesty, of course, is also a moral behavior, it merits examination in its own right.

We asked one question whose answer we could directly verify for all philosophy professors: whether they were dues-paying members of the American Philosophical Association. (The APA publishes an annual list of members, which includes people up to 10 months late with their dues.) Among the philosopher respondents, 138 non-ethicists and 128 ethicists were listed by the APA as members. Of the remaining 59 non-ethicist respondents -- that is, those not on the APA membership list -- 23 (39.0%) claimed to be members. Of the remaining 61 ethicist respondents, 27 (44.3%) claimed to be members. In other words, nearly half of the respondents with the arguably immoral behavior (free-riding by not belonging to the APA) denied that behavior.

The APA's list is not perfect, I'm sure, and people's memories are sometimes fallible for reasons entirely innocent, but it seems plausible to me that much of the effect here is due to culpable inaccuracies -- even if not deliberate lying, a blameworthy bias toward misremembering and misportraying oneself in a positive light. (More attributable, probably, to purely innocent error, either by the respondents or the APA, are the 4% of respondents -- 7 ethicists and 8 non-ethicists -- who were on the APA's lists but did not claim to be members.)

Of course, it's disputable whether philosophy professors should, morally speaking, belong to the APA. In the attitudinal part of the survey, a majority of philosophers (64.7%) said it was morally good to "regularly [pay] membership dues to support one's main academic disciplinary society (the APA, the MLA, etc., as appropriate)", but that left a substantial minority who said it was morally neutral (very few said it was bad). Non-members who claimed to be members may have been somewhat more likely to say it is morally good to support the APA through one's membership dues than were the non-members who truly stated that they were non-members, but if so, the trend was modest (62.0% vs. 52.9%), and not statistically significant, given our relatively small sample of APA non-members.

So the answer to our question about how accurately philosophers portrayed their negative behavior in our survey appears to be: not very accurately at all. Nor do ethicists seem any more honest; in fact the trend (not statistically significant) was toward less honesty. This also fits with professors' evident exaggeration, in our survey, of their responsiveness to undergraduate emails (with ethicists appearing just as prone to such exaggeration). Josh and I have some other tests of honesty, too, not all analyzed, which I'll discuss later.

Incidentally, near the the end of the questionnaire we asked about the morality of "responding dishonestly to survey questions such as the ones presented here" and also "Were you dishonest in your answers to any previous questions?" Those who appear to have falsely claimed APA membership trended, if anything, toward being more likely than those who truly stated their non-membership to say it is bad to respond to such questions dishonestly (93.2% vs. 85.3%, chi-square p = .17). Also, 2 of 49 in the first group (4.1%) and 3 of 65 in the second group (4.6%) admitted having answered a survey question dishonestly.