PageRanking Graduate Programs
Most of the current rankings for graduate programs rank institutions based on reputation surveys or citation records that assess the scholarly impact of faculty in a department. But there are issues with both of these measures for prospective graduate students deciding on which program to attend. Survey respondents are usually academics and administrators at competing universities, so they have an incentive to skew their assessments of other schools. Given the large number of universities, these respondents also rank and judge programs with which they might not be entirely familiar. And while the academic impact of professors is an important part of a department’s success, it does not tell us how much these professors actually contribute to the quality of a student’s education.
This 2007 paper proposes an alternative method for ranking graduate programs in political science based entirely on the placement record of graduate students. The authors argue that placement is a good proxy for department quality since it measures both the talents of incoming students and the value that the program adds to a student’s academic skill set. Their proposed ranking method works exactly like the basic PageRank that we covered in lecture. Universities all start out with equal weights, and then each placement counts as a vote for another university in the network. For example, if Yale hires a Cornell Government Ph.D., Yale casts a vote for Cornell the same way that one web page votes for another one by linking to it. Just like with the PageRank algorithm that we discussed in class, there are consecutive rounds where the value of each vote changes depending on how many incoming votes a department receives, and voting ends when the scores for each department stabilize to an equilibrium.
But what about department size? All things held equal, wouldn’t a political science department that graduates and places 40 Ph.D.s per year invariably get a higher ranking than a smaller department? The authors correct for this by normalizing the results based on size. The final scores are weighted with a number inversely proportional to the size of a department’s graduate pool, so smaller programs do not get penalized for having fewer graduates.
The authors do acknowledge several limitations of their rankings methodology. As stated earlier, placement rate is influenced by both the quality of training at the graduate program as well as the quality of students that the program is able to attract. For example, one famous professor may attract many talented students to a university, and their ultimate placement success will be high because of their intelligence and talents, regardless of the actual quality of instruction they receive. The authors admit that their measure is unable to distinguish between these two aspects of quality. They also acknowledge that their measure is a lagging indicator and does not account for large improvements or decreases in quality towards the end of the time period they study. Finally, using placement data as a proxy for quality may be questioned because not all graduate students aspire to go into academic teaching positions. This may not be that problematic for political science, though, since they cite a statistic that almost 80% of all political science graduate students do have aspirations to eventually obtain positions in academia.
This last point illustrates the limitations of using this novel PageRank method for other graduate programs and for undergraduate institutions. The analogy between websites/links is strong when comparing graduate departments where students primarily aspire to academic careers but weak in cases where students seek jobs in government and the private sector. Cornell’s Government Department places students at Columbia, and vice versa, but an investment bank’s research division does not have any reciprocal “links” to Cornell’s Economics Department. There are also few, if any, parallels to undergraduate institutions. While Cornell undergraduates create a lot of incoming “links” from law schools, medical schools, Wall Street and government jobs, there are no analogous outgoing “links” to these destinations.