Letters of Evaluation

Letters of evaluation (also known as “referee letters”) are a valued component of the review process for appointments and promotions of faculty to advanced rank (i.e., Associate and full Professor on ladder, voluntary, or adjunct ranks; Research Scientist and Senior Research Scientist). They are used to assess how experts and leaders view the candidate’s accomplishments, expertise, and impact on a field. At the department’s discretion, letters of evaluation also may be considered in the reappointment process for faculty with term limits. The procedures described here do not apply to initial appointments at the rank of Instructor, Lecturer, Clinical Instructor, Associate Research Scientist, or Assistant Professor (on the ladder, adjunct, voluntary, or social work ranks).

Experts-in-the-Field

Referees should be recognized experts in research, education, and/or clinical service in the field of the candidate being evaluated. If the candidate has a very highly specialized focus, experts in more general areas of research, education, and/or clinical service may be sought in addition to those in the candidate’s specialized area of focus.

Requisite Rank

It is expected that referees are experts who hold positions equivalent to or higher than the rank proposed for appointment/promotion. In most cases, this means that letters for Associate Professor reviews (whether ladder, adjunct, or voluntary) come from Associate or full Professors, and letters for Professor reviews come from Professors. For research rank faculty, evaluations for Research Scientist reviews can come from ladder faculty who are at least Assistant Professor or from scientists with at least mid-level appointment/title at their institution. Senior Research Scientist letters can come from ladder faculty who are at least Associate Professor or from scientists with a senior-level appointment/title at their institution.

Although most letters of evaluation must come from faculty at or above the rank being considered, occasional exceptions are permitted. For all appointments and promotions, at the discretion of the Chair, a maximum of two (2) Arm’s-Length or Non-Arm’s-Length referees can be non-faculty senior experts or authorities in the candidate’s field, such as:

• Senior scientists at non-university affiliated institutes, industry, or Federal agencies
• Presidents or chief executives of health systems or national professional organizations
• Senior leaders of State or Federal government agencies highly relevant to academic medicine

In addition, at the discretion of the Chair, for Academic Clinician and Voluntary track reviews, a maximum of two (2) Arm’s-Length or Non-Arm’s-Length referees can be solicited who do not hold an academic appointment or senior leadership position (e.g., a referring physician), but who are experts able to provide valid evaluations of clinical or educational excellence and professionalism.

Department and Candidate Suggestions

Each candidate collaborates with their departmental leaders, mentors, and colleagues to develop a pool of potential referees to include on the Referee List Template, listed separately as Department-suggested and Candidate-suggested. Candidates are allowed to communicate to their department the names of faculty in their field who should not be contacted due to professional conflict that might undermine a fair evaluation. Notably, all Department-suggested referees should be Arm’s-Length while Candidate-suggested referees are a blend of Arm’s-Length and Non-Arm’s- Length (see section below on the distinction between these two categories of referees).

Referees who wrote for a prior appointment/promotion can be asked to write again for a subsequent promotion as long as they are at the requisite rank for the current review. With that said, inclusion of new referees is encouraged as well. Note that TACBS may have different requirements and should be consulted for those cases about the need for “new” or “fresh” letters.

Number of Letters

The required number and type of referees for each rank and track suggested by the Department and the Candidate are indicated in the Referee Letter Requirement Chart (at the bottom of this page). There are specific numbers required for solicitation and a specific subset that must be received for the academic review to proceed.

The number listed for initial solicitation is the minimum required by OAPD. Faculty and their Department are strongly advised to create a back-up list of referees for inclusion on the Referee List Template to ensure the required minimum number of letters are received in a timely manner.

Information about Referees

The list of referees is submitted on the Referee List Template. It must be completed fully with each referee’s name, titles, institution, department, email address, phone number, and field of expertise. Non-Arm’s-Length referee names require a brief description of the nature of the professional relationship that may affect the impartiality of review (see section below).

Interfolio

Once the Referee List Template is finalized and cross-checked with the CV Bibliography for Arm’s-Length referee confirmation, the department uploads this document to the candidate’s file on Interfolio. It should include the required minimum number of referees plus the names and contact information for the additional referees.

Responsibility

The department or dean’s office (depending upon the advanced rank under review) solicits letters of evaluation for faculty appointment and promotion reviews to advanced rank. Candidates for appointment or promotion to any advanced rank must not solicit letters of evaluation nor should they informally inquire about a potential referee’s availability, willingness, or follow-through in submitting an evaluation.

Forms Included

Each faculty rank and track have a specific template letter of solicitation which includes a brief description of expectations for appointment or promotion at the rank and track. Referee evaluations are requested by the Department Chair or Dean that address specific areas of accomplishment and the appropriateness of rank and track. Typically the evaluation is provided in the format of a letter. Also included with the Chair or Dean solicitation request is an Evaluator Relationship Form on which the referee documents their associations (if any) with the candidate being reviewed. This form helps the department and school confirm the Arm’s-Length or Non-Arm’s-Length status (see section below).

Interfolio

Depending on the rank and type of review, the Department or Dean’s office faculty affairs staff requests evaluations using Interfolio. Referees are asked to upload their evaluations directly to Interfolio or email the faculty affairs staff. Letters of evaluation are considered confidential and viewable only by current members of departmental or medical school appointment and promotion committees and faculty affairs staff, and they should never be released to the candidate or quoted in any document that is viewable by the candidate.

Follow-up with Referees

Tracking

Within the Referee List Template or in a separate document uploaded to Interfolio, the Department’s or Dean’s office faculty affairs staff track the date of solicitation and whether the referee agreed, declined, or did not respond to the invitation. This document also includes dates of any follow-up reminder requests made through Interfolio or emails sent or received, including any explanation provided by the referee for declining to review the candidate. Once the required number and type of letters of evaluation are received, the department may complete its formal review and vote or await receipt of additional letters.

Communication with Referees

In addition to faculty affairs staff communicating with referees, the Department Chair, Vice Chair, Section Chief, Dean, Deputy Dean for Academic Affairs or other designated senior faculty may follow-up after the initial solicitation about a referee’s ability to provide an evaluation. This communication should be limited to inquiring whether a letter of evaluation can be provided in a timely manner. Under no condition should this communication imply advocacy for the candidate or that a particular type of evaluation is being sought.

In addition, in tandem with Interfolio’s automated solicitation process, the Department Chair or Dean’s Office may send out individual or group emails providing notice of the automated invite and encouraging referees to indicate whether they will be able to provide a review or not.

Under no circumstance may the candidate communicate with a referee about the letter of evaluation process. If the candidate is contacted by the referee, they should communicate the strict school policy prohibiting communication about their promotion review and should not provide further information to the referee beyond what was included in the Interfolio case provided to the reviewer.

Guidance on Arm’s-Length and Non-Arm’s-Length Referees

Appointment or promotion to advanced ranks (i.e., Associate and full Professors on ladder, voluntary, or adjunct ranks; Research Scientist or Senior Research Scientist) involves Arm’s-Length and Non-Arm’s-Length letters of evaluation. Both types provide important information for the review process.

Non-Arm’s-Length

These types of referees have a current or recent professional and/or personal relationships with the candidate that could impact their actual or perceived ability to provide an impartial review. This includes referees whose professional activities or relationship with the candidate could be perceived as benefitting from the appointment or promotion (or its denial).

Non-Arm’s-Length referees may have no direct relationship with the candidate, or they may have experience or knowledge of their professional activities through working or training together in some capacity. They may be faculty from other departments at Yale, but with the exception noted below, should not be faculty from the same primary department as the candidate.

With the exceptions noted below, the following activities define a referee as being Non-Arm’s-Length:

• Co-author of a journal article, book chapter, or other scholarly product
• Co-presenter of a conference paper or poster
• Co-editor or Co-author of a book
• Co-PI, Co-I, or other named collaborator on a grant or contract
• Primary mentor/mentee, supervisor/supervisee, advisor/advisee
• Secondary mentor/mentee, supervisor/supervisee, advisor/advisee, or peer during training, if less than 10 years have passed.
• Current or former co-developer of a clinical or educational program or product
• Thesis or dissertation advisor or committee member
• Instructor, co-instructor, or student in a course
• Chair or close colleague from a previous department or school
• Current Yale faculty from another department
• Close personal and/or other professional relationship

OAPD’s 2012 guidelines prohibited Non-Arm’s-Length letters of evaluation from the candidate’s own department. This continues to be a relevant restriction for research-intensive faculty for whom impartial evaluation of national reputation by top scientists in the field outside of one’s department and school are essential. However, it may disadvantage clinically-intensive faculty for whom expert peer evaluation of clinical and educational excellence and professionalism is highly important.

At the discretion of the Department Chair, up to two of the Candidate-suggested Non-Arm’s-Length referees for Academic Clinician, Clinician Educator-Scholar, and Voluntary track reviews may be from faculty from the same department at or above the proposed rank who have direct knowledge of the candidate’s clinical or educational skills, expertise, leadership, or professionalism. If more than two departmental referees are received, only the first two received may be included in the Interfolio case. Departmental referees who provide a letter may not be listed on the Referee List Template as a faculty member who helped in the creation of the list or be involved in creating the Departmental A&P Narrative. In addition, they must declare themselves ‘ineligible to vote’ in the Departmental A&P committee review of this case and refrain from contributing to the discussion.

Arms-Length

An important goal of the academic review process is to obtain knowledgeable, unbiased, and objective evaluations of the candidate from established experts or leaders in the field. Arm's-Length referees provide an impartial evaluation that is free of actual or perceived bias, and they do not stand to benefit from a successful (or failed) appointment or promotion review. They typically have not worked closely or trained directly with the candidate or had any of the relationships listed above under Non-Arm’s-Length. They are not current faculty members at Yale.

Arm’s-Length referee objectivity and impartiality are sometimes the result of their never having met or known the candidate. In many cases, however, they have had direct professional interactions in one or more contexts and have become familiar with the candidate’s capabilities and contributions. The following associations, by themselves, do not disqualify a referee from being considered Arm’s-Length.

• Serving together on study sections or committees in a professional organization
• Editorial board appointments as member, Associate Editor, or Editor-in-Chief
• Editor of a book in which a chapter was published
• Peer reviewer of book manuscript, journal article, grant, or conference submission
• Co-participant or discussant in a conference panel or symposium
• Guest or invited speaker in a seminar, workshop, or Grand Rounds
• Former faculty colleague where there was no direct teaching or scholarly collaboration
• Referring patients to one another for consultation or treatment

Special Exceptions to Arm’s-Length Definition

At the discretion of the Department Chair or Dean, a small number of referees can be considered effectively Arm’s-Length who meet selective Non-Arm’s-Length criteria as listed above. These may be referees who have had previous collaborative, collegial, or educational relationships with the candidate, but for whom the outcome of the review has no discernible benefit and when the relationship is restricted in one of two ways.

• Distant-in-Time: Ten or more years have elapsed since the candidate and referee were co-authors or co-presenters of academic products, research grant collaborators, departmental colleagues, or training together as peers. This exception does not apply when there is evidence of a continued relationship or a past relationship (e.g., primary advisor or mentor during training) perceived as affecting impartiality or benefitting from the promotion.

• Limited-in-Scope: Participation as co-authors on large (e.g., >25 contributor) research consortium or multisite papers or co-contributors to national organization guidelines or consensus statements. The referee and candidate also may have served parallel roles as local site investigators or leaders of a large multi-site project but had no significant interaction with each other.

Clarifications for Arm’s-Length vs. Non-Arm’s-Length Status

Appointments and promotions committees make conclusions about referee status based on several sources of data:

1. Referee List Template completed by candidate and department
2. Evaluator Relationship Form in which the referee categorizes their relationship to the candidate
3. Statements made by the referee in their letter of evaluation.

Whenever there is discrepancy in these data sources, the department should ensure there is a sufficient number of other Arm’s-Length letters. Whenever there is concern or lack of clarity about whether a referee is Arm’s-Length, the Department is encouraged to consult with OAPD before moving forward with their departmental review and vote. (Email: faculty.affairs@yale.edu)

To minimize medical school review committee questions about the sufficiency of letters of evaluation, it is important for Departmental A&P Narratives submitted after successful reviews to include brief explanation for any referee who may be an exception to the traditional definitions. This would include those experts who do not hold an academic rank, whose professional relationship was ‘Distant-in-Time’ or ‘Limited-in-Scope,’ or are faculty from the same primary department.