Maintaining Accreditation Resources for Leaders

• Annual ADS Updates – The ADS Update window opens for one month between the months of July and September, depending on the ACGME schedule.
• Faculty and Resident scholarly activity – Scholarly activities should be collected from the previous academic year and reported in the Annual ADS Update. 
• Clinical Experience (e.g., Case Logs) – Trainees of participating programs enter cases in the Resident Case Log System throughout the year and must be monitored by program leadership, since many RRCs have case minimums and graduation requirements.
• Milestones reporting – Trainees are evaluated against the specialty-specific milestones on a semi-annual basis during the Clinical Competency Committee meeting. These milestones are grouped by competency and broken down into sub-competency levels.
• Resident Survey - The program’s annual Resident/Fellow ACGME Surveys are scheduled for one month duration between February and April. All survey results are released to the programs in May. The program is responsible for monitoring completion compliance.
• Faculty Survey – The program’s Faculty Surveys are also scheduled for the same month as the Resident Survey, between February and April. The Program Director or Program Coordinator is responsible for informing the relevant faculty and for maintaining compliance.
• Board Pass Rate (external) – The Specialty Boards will send the Board Pass Rates directly to the ACGME for inclusion with the Program’s annual data collection materials.
• Hospital accreditation data – The ACGME will monitor the continued accreditation of the hospital when determining the accreditation status of programs.

Subspecialty programs will have their annual reviews together with their respective core programs.

In order to maintain “Continued Accreditation”, programs must be diligent in successfully completing and submitting all of the required data elements. Following the ACGME annual review, existing programs and sponsoring institutions will be awarded one of the following accreditation status options in an annual notification letter:

• Continued Accreditation
• Continued Accreditation with Warning
• Probationary Accreditation
• Withdrawal of Accreditation
• Administrative Withdrawal

The Program must convene a Program Evaluation Committee at least annually consisting of at least two faculty members, one of whom is a core faculty member, and at least one trainee. The Program Evaluation Committee is responsible for rendering a written Annual Program Evaluation (APE) report, which must document initiatives for improvement in a plan of action. The annual review, including the action plan, must be distributed to and discussed with the members of the teaching faculty and trainees.

• APE Instructions AY2024
• High Value Data suggested for APE review
• APE Questions AY2024
• APE AP Multi-Year Template
• Microsoft Word versions of the APE documents can be found on the GME SharePoint site (login required).

Programs are advised to refer to the ACGME Common Program Requirements and their program specific requirements for a detailed description of the Program Evaluation Committee responsibilities and a complete list of program elements that should be included in the assessment of the program during the Annual Program Evaluation process. Programs may also consider incorporating the following materials in their program assessment: most recent internal and/or special review; Mass General Brigham GME Fall Program Survey; trainee compliance with mandatory training (e.g., HealthStream, etc.); and patient surveys.

The GME Office, in conjunction with the Annual Review of Programs Subcommittee of GMEC, will review all data elements submitted through the ACGME Accreditation Data System (ADS) as well as internally submitted materials. This review will start with an overview of all programs and will be used to determine which programs would benefit from a Special Review. Data reviewed may include the ACGME Resident and Faculty Surveys, recent ACGME accreditation letters (including any progress reports sent to the RRC), Annual ADS Update submitted by the program to the ACGME, Annual Program Evaluation (APE) along with the program’s multi-year action plan, recent Special and/or Internal Reviews (if applicable), GME Work Hour Program Survey (optional), Milestones Data, and Case Log Data (if applicable).

• Address the letter to the Executive Director of the RRC.
• Prior to the greeting, include a "Re:" section that states your program name as shown on the RRC letter (including the name of the sponsoring institution) and the program number.
• The first paragraph should reference the RRC letter to which you are responding: the date and the nature of letter (e.g., accreditation letter, request for additional information or for a progress report, etc.).
• Each citation, question, or comment to which you are responding should be included, verbatim and in quotes, followed by your response. Ex: Citation #1: “The RRC judged that resident call rooms are inadequate…” [include the full text of the citation]. Response: “The space for three additional call rooms has been identified. Since the time of the site visit, construction has begun and occupancy is expected on December 1, 2023.”
• In the final paragraph, you should thank the RRC for their careful evaluation of the program and offer to provide any further information that might be helpful. If there is some urgency to their assessment of your response (such as approval for program expansion that may hinge on their review), you can note that.
• Include this text as the last sentence: “The signature of the DIO below serves to attest that the GME Committee has reviewed and approved this report.”

• Responses should be clear and sufficiently detailed.
• Extraneous or irrelevant information should not be included—e.g., room location of conferences, etc. A historic perspective on the program is usually not appropriate, except (a) where you want to demonstrate progress relating to an issue of concern, or (b) where the program is facing a possible adverse outcome and you want to diplomatically include some “PR” (we can help with this).
• It is important to be clear in describing the time course and current status of program initiatives or improvements. For example, in addition to describing a new initiative, state whether it is “under consideration” vs. “planned” vs. “implementation anticipated on [date]” vs. “implemented as of [past date]”.
• It is also important to note when a problem was addressed with respect to the date of the site visit AND the date of the RRC meeting when the site visit information was considered. The RRC may consider more favorably improvements that were undertaken based on the program’s initiative, rather than in response to their concerns. (For example: “The department’s education committee eliminated the XYZ rotation as of June 30, 2022 based on input from residents and faculty, prior to the RRC’s consideration of this program at its meeting on April 15, 2022.”)
• Sometime citations don’t seem to correspond to a written RRC requirement. Please highlight this for us when you submit your draft and we will talk with you about whether to “rebut” the citation on that basis—which on rare occasion is a reasonable strategy.

• Attachments with detailed documentation addressing the specific issue at hand are helpful, such as: lecture schedules that illustrate coverage of all core curriculum topics; duty hours logs and summary reports documenting compliance; lists of resident-generated lectures and publications to document “scholarly activity”, etc.
• If data addressing the issue at hand was provided previously, such as part of the Program Application Form, that can be politely referenced in the letter and the data can be supplied again as an appendix.

• Whether or not the RRC’s concern seems reasonable, the tone of our response must be respectful and even appreciative. Placing blame on the RRC or site visitor, individuals within the program, or unreasonable requirements is counterproductive.

• If you absolutely cannot provide a response by the date requested, ask for an extension as soon as the letter is received. It is preferable to seek an extension by e-mail, rather than by phone, so that there is a written record of the extension being requested/granted.
• It is normally required that GMEC (Graduate Medical Education Committee) approve each program response to the RRC. To have your letter placed on the agenda requires advance notice: the GME accreditation manager for your program will coordinate this. Usually, the draft response should be submitted approximately three weeks prior to the Brigham and Women's Hospital or Mass General Hospital GMEC meeting for review and comment by the GME Director for your specialty.
• You may wish to respond to an accreditation letter even if no response is requested—especially if the RRC based any conclusions on incorrect or misinterpreted information. (Please discuss this with the GME Director prior to writing a draft.) Unsolicited responses may not be reviewed by the RRC upon receipt but would become part of the program’s ACGME file.

Mass General Brigham GME Office conducts Internal Reviews for all Mass General Brigham programs to assist them with maintaining the continuous accreditation status with the ACGME.

As part of the Next Accreditation System (NAS), the ACGME no longer requires Institutions to conduct Internal Reviews. However, the MGB Education Committee decided that the GME Office will continue conducting Internal Reviews for all Mass General Brigham programs to assist them with maintaining the continuous accreditation status with the ACGME.

The Internal Reviews will be scheduled approximately mid-point between the Self-Studies and will serve the following functions:

• assess the education provided by a program to their trainees;
• monitor quality improvement through recommendations and follow up; and
• help the program prepare for their next Self Study/Site Visit.

The Internal Review Committee (IRC) will consist of 1-2 faculty members from a Mass General Brigham ACGME-accredited program, 1-2 trainees from a MGB ACGME-accredited program, one of the GME directors, and the GME accreditation manager. As part of the review, the IRC will meet with the trainees, faculty, and program leadership.

The results of the Internal Review will be summarized in a report, reviewed and approved by the graduate medical education committee, then shared with the program. The GME Office will then follow up on items needing improvement and offer suggestions for implementing recommended changes. The ultimate goal of the review is to ensure the program is of the highest quality possible, and that it maintains continuous accreditation by the ACGME. The report is not shared with the program’s Residency Review Committee, and therefore would not directly affect the program’s accreditation.

Mass General Brigham GME Office conducts Special Reviews for programs identified by the Annual Review of Programs (ARP) process as underperforming and for those programs that request them.

The ACGME requires Institutions to monitor programs’ accreditation status and program performance on an annual basis. The Graduate Medical Education Committee demonstrates effective oversight of any underperforming programs through a Special Review process. To assist in meeting these ACGME Institutional and Common Program Requirements, the Annual Review of Programs (ARP) process was introduced. As part of this process, the GME Office, in conjunction with a newly-formed ARP subcommittee of GMEC, determines which programs would most benefit from a Special Review. 

The following materials may comprise the ARP review of the program:

• ACGME Accreditation Letter
• Annual ADS Update submitted by the program to the ACGME
• APE Form (including Multi-Year Action Plan) submitted by the program to the GME Office
• ACGME Resident and Faculty Surveys (if available)
• Most recent GME Internal of Special Review report and response
• ACGME Milestones Report
• ACGME Case Log Reports (if available)

Program identified as underperforming will be contacted to schedule the Special Review visit. The Special Review Committee (SRC) will be formed and will consist of 1-2 faculty members from a MGB ACGME-accredited program, 1-2 trainees from a MGB ACGME-accredited program, one of the GME directors, and the GME accreditation manager. As part of the review, the SRC will meet with the trainees, faculty, and program leadership.

The results of the Special Review will be summarized in a report, reviewed and approved by the graduate medical education committee, then shared with the program. The GME Office will then follow up on items needing improvement and offer suggestions for implementing recommended changes. The ultimate goal of the review is to ensure that the program is of the highest quality possible, and that it maintains continuous accreditation by the ACGME. The report is not shared with the program’s Residency Review Committee, and, therefore, would not directly affect the program’s accreditation.