Monday, August 04, 2014

Citation Patterns in the Stanford Encyclopedia of Philosophy, Part 1

My fourteen-year-old son Davy is teaching himself to program, so for fun last week I had him pull all the reference sections out of the 1422 front-page entries in the Stanford Encyclopedia of Philosophy. In 2010-2011, I'd written a few posts analyzing SEP bibliographies. But my selection at the time had excluded entries on particular individuals and historical eras, and the SEP has grown and updated quite a bit, so I thought it might be interesting to revisit some of those analyses.

[The picture above shows Davy appreciating the SEP's clean html.]

Let's start with an easy analysis before I try to wrangle order out of the 137,000 rows of data Davy sent me. In 2009, Brian Leiter conducted an (unscientific) website poll of his readership, asking them to rank dozens of famous philosophers in order of "importance". The resulting top 20 were:

1. Plato
2. Aristotle
3. Kant
4. Hume
5. Descartes
6. Socrates
7. Wittgenstein
8. Locke
9. Frege
10. Aquinas
11. Hegel
12. Leibniz
13. Spinoza
14. Mill
15. Hobbes
16. Augustine
17. Marx
18. Nietzsche
19. Kierkegaard
20. Rousseau
For quick and dirty fun, we can toss these names in the SEP search engine and see how many different entries get hits, ranking the philosophers accordingly.

The results:

1. Aristotle
2. Plato
3. Kant
4. Hume
5. Descartes
6. Locke
7. Leibniz
8. Frege
9. Socrates
10. Wittgenstein
11. Mill
12. Aquinas
13. Hobbes
14. Augustine
15. Hegel
16. Spinoza
17. Marx
18. Rousseau
19. Nietzsche
20. Kierkegaard
Interestingly similar -- and thus partly validating each other, in a way. For example, the top five are the same, only with Aristotle and Plato swapping the two top spots. Leibniz shows the biggest difference in rank -- 7 in the SEP vs. 12 in the Leiter poll. Hegel is next, 15 in the SEP vs. 11 in the Leiter. All the others are within 3 places, most within 0-2. Mill is a bit higher in the SEP perhaps partly due to attracting more false positives than the others. Overall, the rank correlation is a solid .92.

Might there be others in the SEP top twenty that weren't in Leiter's top-twenty poll results? Well, actually things get messy outside the top 20. Berkeley is Leiter's 21, but he's hard to pull out of the SEP because of false positives from the university of the same name. Epicurus, 22nd in Leiter's poll, would slightly edge Kierkegaard, but not in a worrisome way (66 entries vs. 63). Russell is ranked 23rd in Leiter's poll -- also a lot of false positives there, plus possibly a recency bias. (See these old data on recency bias in the SEP, which I plan to update soon.) Then Rawls, Bacon, Smith (Adam), Quine, Kripke -- all also either too recent or drawing too many false positives. Thus, keeping comparisons within Leiter's top 20 seems the best plan.

Where are the Asian philosophers, you might ask. Well, Brian did include Confucius in his poll. Confucius pulled 32nd place, edging out Heidegger and Machiavelli, and just below Parmenides and Anselm. (Mencius didn't make it out of the qualifying heats.)

Now, Confucius is one of the most important figures in the history of Asia, both specifically in Asian philosophy but also more broadly in East Asian culture. Perhaps no philosopher in the history of the world has had more influence than he. So I find the poll result odd! But Confucius fares even less well in an SEP hit count than in the Leiter poll: Heidegger and Anselm both appear in many more entries than does Confucius.

This is my way of saying, I suppose, that these types of analyses capture a certain sociological phenomenon among the Leiter Reports readership and the SEP authors and editors.

But I happen to find the sociology of philosophy very interesting!


Eric Steinhart said...

I don't know that the SEP and Leiter rankings would partly validate each other. Their similarities just validate overlap in the people writing SEP articles and squawking to Leiter. One problem with all this is that you've really got no independent control groups or standards. It's all the same people, doing the same stuff, relative to each other. Of course, that doesn't mean that there won't be interesting patterns! There will be.

Eric Schwitzgebel said...

Eric, I think it is at least a weak validation of a sociological phenomenon. For example, one might wonder whether Nietzsche's much lower ranking than Wittgenstein in the Leiter poll is some kind of fluke. The fact that there's similar positioning in the SEP entries suggests that it's not *just* a fluke. I leave open the question of whether the best way to read it is as a stable consensus of expert opinion or as a merely sociological fact about the overlapping group of Leiter readers and SEP authors.