By “external letter” we mean any letter written by an individual who is not a voting member of the candidate’s department. Note that by this definition an external letter can come from within Cornell, e.g., a letter from a graduate field member in a different unit. What the colleges have to say.
D1. What is the purpose of the external letters?
The purpose of the external letters is to determine outside perceptions of the candidate’s research. Collectively, the letters should inform the reader about the breadth, depth, impact, and trajectory of the candidate’s research program.
D2. Who should be involved in the production of the external reviewer list?
Because of the importance of this dossier component, the candidate should be able to suggest reviewers. The broader the engagement of the voting faculty the better. However, it is understood that in the larger units the task may be delegated to a subset of the voting faculty.
D3. What are some of the key factors to consider when thinking about a possible external reviewer?
Research Distance. The “distance” between the reviewer’s research area and the candidate’s research area is important. One purpose of the external letters is to assess the candidate’s radius of impact. This can only be ascertained by getting letters from reviewers who work around the fringe of the candidate’s proclaimed research area.
Collaboration History. The history of reviewer-candidate collaborations will affect how the letter is evaluated. Collaborations include co-authoring a paper, a book, or a research proposal as well as co-organizing a workshop, a conference, or a panel discussion. All of these activities concern the advancement of knowledge and as a group the collaborators are well equipped to offer important insights into the candidate’s ability to work with others. That said, claims like “we changed the world with this paper” need to be assessed for their self-promotion content.
Research Stature. The impact of an external letter is highly correlated with the research stature of the writer. Stature is a perception that depends on the actual research reputation of the reviewer, the reviewer’s local circle of colleagues, and the reviewer’s institution. It is not a simple summation. There are many examples of great researchers whose stature exceeds that of their academic home.
A one-paragraph synopsis of each reviewer should be provided in the dossier to help the reader assess the content of each external letter.
D4. What about having more than one reviewer from the same academic unit at a different school?
There is no reason to have a blanket rule against this; the reviewers involved may be able to write about widely separated components of the candidate’s research agenda. On the other hand, a pair of great letters from a pair of tight collaborators is likely to have diminished value because of redundancy.
D5. What about letters from Cornell faculty?
A letter from someone in the same graduate field can be valuable if it sheds light on collaborative skills or service qualities. An on-campus co-PI can certainly shed light on research contributions. Note that the act of including a letter from a Cornell faculty member disqualifies that individual from serving on an ad hoc committee.
D6. What about letters from individuals who are unfamiliar with the tenure system?
The research credentials of the reviewer are what is important. It doesn’t matter if they work in industry or a museum or haven’t a clue about the American tenure system. However, it may be harder to interpret the enthusiasm (or lack of enthusiasm) in a letter from such a reviewer. Care must be exercised.
D7. How many letters from external reviewers should there be in the dossier?
“10” is a typical number but it varies across the colleges: AAP(at least 10), CALS(9-13), CAS(at least 7), JCB(at least 10), COE(10-14), CHE(9-13), ILR(10), LAW(8-10), CVM(8-13). Of course, the anticipated return rate needs to be factored into the equation. Thus, it may be necessary to send out a dozen requests in order to get nine responses. Steps that are taken to make up for a shortfall need to be documented.
D8. How should the department go about actually producing the list of external reviewers?
The candidate should produce a complete list C and (separately) the department should produce a preliminary complete list D. The department then uses these two lists to produce the final list. The dossier indicates which of the reviewers appear only in C, only in D, or in both C and D. This approach to reviewer selection prompts the candidate to think more broadly about what is to be deduced from the external letters. It also minimizes gamesmanship, e.g., “if I do not include this obvious reviewer from my list then the department will most likely pick that individual”.
There are several numerical parameters that need to be publicized: the targeted length of the final list, the length of the candidate’s list, and the minimum and maximum number from candidate reviewers that are to be included in the final list.
D9. How should the candidate go about producing a list of potential external reviewers?
The candidate should take into consideration the points that are raised in D3-D6. Letters from fellow assistant professors are likely to be disregarded. A letter from the thesis advisor is often a nice gesture but there may be a down-weighting of its impact because of potential conflict of interest. The candidate should be encouraged throughout the probationary period to connect and share their work with others who could evaluate their reputation as a researcher. These individuals can turn into the crucial external reviewers who can assess that reputation.
D10. What about the candidate’s do-not-contact list?
Candidates should be allowed to submit such a list with a brief rationale next to each name. Several colleges mention the no-contact list in their documentation (AAP, CALS, CAS, JCB). If provided, the list should be part of the dossier. Legitimate reasons for including an individual on the do-not-contact list are (a) concern that the reviewer might “scoop” some partially developed research idea, (b) some kind of professional fight that occurred in the past, or (c) some kind of conflict of interest, e.g., both involved in competing proposals. It is not a sufficient grounds for elimination if the candidate believes an outside reviewer will not view his/her work favorably. The department is not bound to honor the do-not-contact list. However, it should provide an explanation in the dossier should it choose to use a reviewer from that list.
D11. Should the solicitation letter be public in template form and should its content be visible to the candidate and the tenured faculty?
As a service to its departments, several colleges provide advice on the form of the letter (CALS, CAS, JCB, CHE, ILR, CVM).
The solicitation letter frequently includes a high-minded paragraph or two that articulates how the department thinks about research and research trajectory. These encapsulations of department philosophy are important to share but they should not reflect a bias for or against the candidate’s research area.
Even though the chair (or equivalent) is ultimately responsible for the letter, the candidate and the tenured faculty should be given the opportunity to review its contents and suggest edits. The goal is to prompt the generation of a fair and professional assessment of the academic record.
D12. Are there certain questions that should or should not be posed to the external reviewer?
Three situations require care. (a) It is believed that female candidates are disadvantaged when the reviewer is asked to weigh in on direct one-to-one comparisons. (b) All too often the question “Would X get tenure in your department” leads to more of a discussion about departments rather than the candidate. (c) References to other researchers who are in the same “age group” is risky because family leaves may torque how the reviewer “computes” productivity.
D13. How much of the dossier should be made available to the external reviewer and in what form?
The candidate has the right to determine the exact subset of their scholarly work that is accessible to the external reviewer. Recommendations to the candidate:
- The chosen content should be discussed with the chair and/or mentor and it must include the CV.
- Mindful that external reviewers have limited time, the organization of the chosen content should be easy to navigate with important items highlighted accordingly. Providing a single pdf with navigation links is a widely used format.
- The research, teaching, and extension statements should be considered for inclusion as they provide handy overviews of your work and your philosophy.
- If the visibility of a publication requires payment of a fee or the purchase of a book, then steps must be taken to provide access.
- Do-not-share stipulations should attend those documents that are not (yet) intended for free public viewing.
- External reviewers are not in a position to interpret course evaluations so that data should never be included. On the other hand, course syllabi should be considered for inclusion especially if they reflect a measure of innovation or some novel way of bringing research ideas into the classroom.
