I have been asked to be an evaluator for the 2014-2015 edition of the Philosophical Gourmet Report. Contrary to what seems to be a widespread sentiment in the philosophical blogosphere, I support the rankings and will participate.
The PGR rankings have at least three related downsides:
1. They perpetuate privilege, including the privilege of people with social power in the discipline, the privilege of people in PhD-granting institutions over other types of institutions, and the general privilege of Anglophone philosophy and philosophers. 2. They reinforce mainstream ("Gourmet ecology") valuations of topics and approaches, in a discipline where the mainstream needs no help and it would probably be productive to push against the mainstream. 3. They risk blurring the distinction between second-hand impressions about reputation (especially outside evaluators' own subareas) and genuine quality.
In light of these downsides, I understand people's hesitation to support the enterprise.
I view the rankings as an exercise in the sociology of philosophy. The rankings are valuable insofar as they reveal sociological facts about how departments, and to some extent individuals (especially in the specialty rankings) are viewed by the social elite in Anglophone philosophy -- by the people who publish articles in journals like Nous and Philosophical Review, by the people who write and are written about in Stanford Encyclopedia of Philosophy entries, by the people who teach at renowned British and U.S. universities like Oxford, Harvard, and Berkeley. As a part-time sociologist of philosophy interested in patterns of esteem, I am curious how people in this social group view the field, and I regard the PGR as an important source of data.
The PGR is thus valuable in part because sociological and historical knowledge about academia in general is valuable. It is sociologically interesting, and of historical interest, to know what sort of esteem Australian universities have in mainstream Anglophone philosophy. It is sociologically interesting, and of historical interest, to see the shifting patterns of social power among Ivy League universities and large public U.S. universities that are able to hire renowned professors.
The PGR is also practically valuable because knowledge of the centers of social power is practically valuable. To the extent a student wishes to tap into the centers of social power to increase her likelihood of finding a research-oriented job, she should know where those centers of power are; and students and their advisors who are not currently near centers of power might not find it at all obvious where those centers are. By empowering outsiders with knowledge -- especially the knowledge that renowned universities like Harvard, Oxford, and Yale might not be the best universities in their subfield -- the PGR to some extent works against the perpetuation of privilege, despite the fact that it reinforces privilege in other ways. Also, to the extent one wishes to fight against mainstream perceptions of the discipline, it is of interest to track what those perceptions are and how they are changing over time -- though if this were one's primary motivation, one would probably oppose the PGR. Finally, to the extent one respects the judgment of philosophers in the Anglophone philosophy mainstream, one might infer differences in real quality from differences in reputation.
On the last point: If you think that Anglophone philosophy mainstream judgment is grossly erroneous in general, you might reasonably infer that the PGR does more harm than good; but I don't hold that view myself. In philosophy of mind, for example -- my own specialty -- I think that the best-regarded philosophers tend in fact to be excellent philosophers who deserve their good reputations.
One area in which I think mainstream philosophical judgment is ill-tuned is in its disregard of non-Western traditions. However, I believe that the PGR has the potential to be progressive on this issue. For example, in treating Chinese philosophy as an area worth special remark, despite the small number of PhD-granting philosophy departments in Anglophone countries who have specialists in the area, it gives the subarea more visibility than it otherwise would have. And were there sufficient hiring in other non-Western traditions, I suspect the PGR would adapt to reflect that.
Despite my support of the PGR rankings, I think it is important that the rankings be viewed critically, as a rough tool for revealing certain sociological patterns in the discipline. I would very much like to see other approaches to evaluation, which would help put the PGR rankings in context as only one way to think about the social structures that drive academic philosophy.