Trainee Assessment
Assessing Surgical Trainees
Assessment methods in ISCP (http://www.iscp.ac.uk/Assessment/Intro.aspx)
Exams formed the mainstay of the assessment of surgical trainees in the past. With the Calman system of numbered surgical training posts, a system of review of trainees’ clinical and professional performance in the workplace began. Record of In-Training Assessment forms were completed by the consultant supervising each placement and these were reviewed by a local committee of consultants who interviewed each trainee at least yearly and confirmed progress or otherwise through the system. Calman style SpRs are still reviewed using this method. However there was no mechanism for regular assessment of trainees in the workplace.
The Intercollegiate Surgical Curriculum Programme (ISCP) lays considerable emphasis on Workplace Based Assessment (WBA). For each placement the trainee and Assigned Educational Supervisor agree a Learning Agreement which covers various clinical and professional syllabus topics. The Objective Setting for this Learning Agreement should be done within a few weeks of starting a placement and agreement should also be reached on the exams to be taken, courses and teaching to be attended and the research and audit projects to be done. During the course of the placement the trainee should undertake regular Workplace Based Assessments which will contribute to the evidence for the Assigned Educational Supervisor’s Summary Report at the end of the placement. The individual assessments are formative(assessment for learning), ie the main purpose is to provide constructive feedback and coaching for the trainee. However the final Summary Report is summative (assessment of learning) ie records the opinion of the educational supervisor about the trainee’s learning during the placement and level of competence at the end of the placement.
Five main types of Workplace Based Assessments are used in ISCP.
- Clinical Examination Exercise
MiniCEX (http://www.iscp.ac.uk/Assessment/WBA/MiniCEX.aspx) in which a trained observer (consultant or senior trainee) observes a trainee encounter with a patient. Typical scenarios would be an outpatient consultation or assessment of a patient admitted as an emergency. The abilities of the trainee in clinical care and communication can be observed and immediate feedback provided. This is of most use for junior trainees and it is advised that ST1 and ST2 trainees should record at least 6 of these assessments each year.
- Case Based Discussion
CBD (http://www.iscp.ac.uk/Assessment/WBA/CBD.aspx) in which the trainee discusses their own management of a patient with a consultant, usually the Educational Supervisor. The case notes are brought to the discussion and the trainee’s entries in the notes reviewed. Since the complexity of the case can vary significantly, this assessment is suitable for all grades of trainee. ST1 and ST2 trainees are advised to do a minimum of 6 CBDs each year and more senior trainees around 12 per year.
- Peer Assessment Tool
MiniPAT (http://www.iscp.ac.uk/Assessment/WBA/MiniPAT.aspx) is a 360 degree assessment tool in which the trainee asks at least 12 other health professional to rate his performance in a number of areas, including professional skills as well as clinical skills. The ISCP website regulates the raters to ensure that consultants, senior nurses and other allied health professionals are included as raters. The assessments are compared with the trainee’s self assessment and with the mean values for the peer group. This is mandated in ST1 and ST4 but additional assessments may be requested by the Programme Director.
- Direct Observation of Procedural Skills in Surgery
Surgical DOPS (http://www.iscp.ac.uk/Assessment/WBA/SurgicalDOPS.aspx) is a direct observation of a simple operation or procedure eg excision of skin lesion or rigid sigmoidoscopy, with the aim of providing immediate feedback to aid learning. The procedure should involve a limited number of steps and this assessment is aimed principally at more junior trainees. At least six surgical DOPS assessments are expected in ST1 and ST2.
- Procedure Based Assessments
PBA (http://www.iscp.ac.uk/Assessment/WBA/PBA.aspx) are used for assessment of more complex operations. In the UK, they were developed first in Trauma and Orthopaedic Surgery. The assessment combines a list of tasks which make up the procedure from consent to immediate postoperative care with a global summary of the trainee’s ability to perform the observed part of the procedure. Each task is to be assessed as “satisfactory to CCT level” or “development required”. However the level to be attained in the global summary is that listed in the appropriate stage of the syllabus. This means that a more junior trainee may record an assessment in which only a limited number of tasks is performed to a “satisfactory to CCT level” while obtaining a global summary score which matches or exceeds that expected for his stage. A limited number of relatively common operations have been chosen for PBAs – the aim is that these should be Index Procedures, typical of the specialty and encompassing the characteristic skills used in the specialty. The aim is that a PBA should be completed as often as practical when an Index Procedure is being performed under supervision and feedback given immediately afterwards. In colorectal surgery there are 4 operations that have a PBA. These are closure of ileostomy, right hemi colectomy, anterior resection of the rectum and lay open low fistula-in-ano. The relevant PBA can be down loaded from the ISCP website, using the above link.
These Workplace Based Assessments provide evidence on which the Assigned Educational Supervisor can base his Summary Report about the placement. Other evidence consists of:
- The logbook, which should be kept updated
- Exams
- Course and teaching attendance
- Research and Audit projects
- Clinical and professional performance
The assessment of professional skills can be challenging. The ISCP website has a syllabus section devoted to Professional Skills and Behaviour. The topics in this part of the syllabus should be allocated in the Learning Agreement in a similar manner to the clinical topics. Some areas can be assessed by some of the formal WBA eg communication with patients in miniCEX. However each topic includes a list of practical activities which the trainee might be expected to perform to demonstrate ability in the appropriate area.
The Summary Report of the Assigned Educational Supervisor remains key to the assessment of the trainee. Although the formal WBA and other evidence form the basis of this report the trainee’s overall clinical performance should also be assessed in consultation with the other consultants in the department, so that an adequate report may be recorded for each trainee. At the Annual Review of Competency Progression the Summary Reports of each of the Assigned Educational Supervisors are examined and the ability of the trainee to progress to the next level of training is reviewed by a committee of consultants in a similar way to the RITA system.
This page was last updated on 12-02-2009