Monday, 12 August 2013

A teacher's perspective and plea on the RCGP CSA (Clinical Skills Assessment) GP exit exam on the 50th anniversary of Dr Martin Luther King's speech

 
 

Hundreds of Asian, Chinese and Black ethnic minorities, predominantly male NHS GP trainees, often also fathers of young children, have been devastated by the CSA actor GP exit exam. Let not this one exam separate a father from the love of his family. Give him back his dignity and ability to provide for his family. Do not watch wives take their children away and leave the country. Do not separate a young child from her father. He wants to show his daughter how much he loves her but is unable to while he lives a constant psychological, financial and social nightmare, trying to pass the seemingly impossible CSA exam. He trained with real patients with real diseases. He did not train in medical school with actors faking illnesses. He does not know how to 'act' like a doctor treaing a faked disease, as all he knows is how to be a real doctor treating real medical conditions. Do not encourage masking the problem with SSRI antidepressants and anxiolytics. Please admit there is a problem and address the real problem as soon as possible. With each passing diet of this exam, more lives and families are shattered. The only difference in the photo depicted above to that of minority GP trainees is the colour of the father's skin. 

 

2013 marks the 50th anniversary of Martin Luther King's speech 'I have a dream' whereby people are judged by the content of their character and not the colour of their skin.

 
 
 

The CSA pass rate for white British GP trainees is 96% vs the CSA pass rate for black British GP trainees is 66%? Why!

British Medical Association Deputy Chair Dr Kailash Chand, an Indian IMG GP writes:

Why not increase diversity among the predominantly white CSA examiners?


I know of one asian female GP trainer who keeps asking the RCGP each year when they will take her as an examiner and she is told she is still on the waiting list and it may take 2 years. Surely if they wish to increase diversity among the examiners, ethnic minority GP trainers should have priority on the list.

 

Why not increase diversity among the predominantly white CSA actors?

 
 
Many GP trainees have shared with me that they either had all white examiners and all white actors or all white save one asian examiner or all white save one black or asian actress. This disadvantages the GP trainees who have been placed in a Polish NHS GP training practice with a majority of non English speaking patients or at an all Asian patient NHS GP training practice where they are often speaking only in their native tongue. Working in a multicultural city, it may be unsettling to find oneself only seeing white actors and white examiners in an exit GP exam when the local patient demographics are entirely different. Many GP trainees have had partnerships lined up at their local teaching practices which they have not been able to assume because of CSA failure.

 

Why is there a difference in pass rates among GP specialty training Deaneries?

 
 
If you look at the selection statistics, a handful of less popular Deaneries only fill on second round of applicants and these are more likely to be from the IMG pool. London Deanery is the most popular deanery, extremely competitive and matches mostly white local UK graduates.

 

Why is there such a huge disparity in CSA exam pass rates between white (96%) and nonwhite GP trainees (66% black British, 36% Indian IMG, 25% chinese IMG) when the difference was negligible 5% when assessed with videos of real patients in real GP surgeries?

 
 
 
Having been a private educator for the MRCGP exam, I have seen it evolve over the past 10 years and admit I miss the video exam assessment module. I enjoyed watching GP trainees show progress in consulting with real patients and learning patient back stories. Race, ethnicity and sex never played a factor in my teaching, as it was irrelevant as the patients filmed in the video were of different ethnicities but all spoke English well enough for the GPs to demonstrate competencies and all the patients had real physical or mental conditions. The patients were not reading a script or faking an illness for the camera. However, I was deeply struck by the huge difference in CSA pass rates based on colour and race, an exam employing actors to fake illnesses.
 
 
Prior to 2010 to pass CSA meant to pass 8 out of 12 CSA stations. This allowed for nerves, missed cues and any hidden extreme bias in the actors or examiners as one could fail up to 4 stations and still pass. No one is perfect. Although the difference in pass rates was present it was not as marked as post 2010 when marking changed to the borderline system.

With each of the 13 CSA stations now marked between 0-9/9 points, a total pass mark would be agreed upon at the end of the day, ie between 72 and 77. For a safe overall pass of 78, it meant that now GP trainees would have to concentrate on achieving 6/9 (pass) in all 13 CSA stations. A 5/9 is a fail. One would think that by raising the bar, all GP trainees pass rates would fall slightly but instead we saw a rise in white UK pass rates and a significant drop in non white UK and IMG (international medical graduate) pass rates.

As an educator, I now had to figure out how to help them achieve perfection! Now there was no room for error, nerves, missed cues, etc. Yet how did this explain the white UK pass rate going up, and for white local females it even went up to a staggering 98% pass rate!
 
 

Hidden bias

 
 
So I looked at hidden bias and what could trigger unconscious bias in stranger encounters. Simple things from an introduction as 'Dr Mohammed' vs 'Dr Mo' made a difference. One triggers hidden bias against male Muslims and the other no triggers. I noticed that female Muslim GP trainees in hijabs were more likely to pass. Why? It then became obvious that wearing a hijab triggered a hidden positive bias towards caring nuns in habits and seemed to negate foreigner bias. How does positive and negative hidden bias then affect exam outcomes?

An actor has all the answers. He or she does not have any physical symptoms or signs but only a script to relay. If bias is present, this may affect how easily or not the actor reveals clues or answers questions. Some ethnic trainees shared the actors would be silent for several minutes despite being asked question after question, no answer would be forthcoming. With video monitoring, we can assess whether a white actor was being too stubborn in revealing concerns or too forthcoming.

By applying a series of techniques to prevent triggering of unconscious bias, I was able to help ethnic resits jump from scores of 55 fail to 85 strong pass, from 66 fail to an outstanding pass of 91 and an Asian male GP trainee broke the 100 mark barrier to receive 101 marks. All this talk about having to improve English is irrelevant as under enormous psychological duress, 2nd language ethnic minorities facing a costly £1525 exit exam with a 64% failure rate were bound to be tongue tied and a nervous wreck. So I applied psychological techniques to give them back confidence, to make them believe 'brown eyes' were superior to 'blue eyes', how to relax under extreme duress to be able to speak naturally. After all they all passed the IELTS English test and conversed fluently when not in the 'hot seat'.
 
 

What is the solution?

 
I hope this helps give more insight into CSA. It doesn't need years of research but just simple application of Professor Greenwald and Banaji's Harvard Implicit Association test to screen actors and examiners for bias and remove actors or examiners who score high for hidden bias (bias can be both positive or negative with respect to age, sex, ethnicity, sexual orientation, weight, attractiveness etc), for shadow examiners to be present at each station to balance bias, more diversity among the actors and examiners and video accessibility for fair legal appeals. It has been very upsetting to listen to IMG GP trainees share how they have ended up on SSRIs over this CSA exam, to watch grown men weep on my course or over the phone, and all that is needed is better addressing, testing and controls for human bias in a subjective exam.

Local UK graduates will have practiced role plays from day 1 of UK medical school. IMGs have only treated real patients with real diseases from day 1 of medical school overseas and throughout their many years of NHS service and are not exposed to role playing with actors faking illnesses until late into their UK GP training programme. This would then give IMGs an unfair disadvantage when sitting an actor roleplay CSA exam as an exit GP exam. Better to go back to video assessments (as was previously used as an exit GP exam by NHS Deaneries) with real patients and real diseases as both UK and IMG doctors will have treated real patients equally in the NHS.


I leave you with one story. A CSA role play actor was faking right shoulder pain for an Asian IMG GP trainee on my course. The trainee who had years of NHS experience in surgery and had MRCS seemed baffled by the patient's history and being asked to examine a normal shoulder. The actor tried his best to fake shoulder pain but the role play seemed disingenuous. He wasn't really in pain. So the actor stopped role play and said well actually my left shoulder does hurt and I have been waiting ages for a GP appointment. So we interrupted the role play and I asked the IMG to examine the shoulder with real pain and real pathology. The IMG confidently and swiftly examined the left shoulder, made and shared the diagnosis and sat with the actor to share the management plan. The actor's eyes welled up with tears. He looked around the room and got it. He understood why so many IMGs/BMEs were sitting around the room, having failed the CSA exam umpteen times, not because they were not good doctors but because they were not good actors, they could not pretend to examine or make a diagnosis in a healthy patient who was faking pain.
 

Postscript