UNIVERSITY OF SÃO PAULO
SCHOOL OF MEDICINE
SELF EVALUATION POORLY
CORRELATED WITH
RESIDENCY ADMISSION
GRADES ON OSCE-BASED
TEST
Júlio César de Oliveira
Maria Patrocínio Tenório Nunes
Leonardo Borges de Barros e Silva
Cristina Helena Fonseca-Guedes
Itamar de Souza Santos
Joaquim Edson Vieira
São Paulo
BRAZIL
UNIVERSITY OF SÃO PAULO
SCHOOL OF MEDICINE
São Paulo
BRAZIL
OSCE Performance as a Residency Admission Test
Phase 1 - Writing test
3.000 candidates
-600 Graduated Doctors around
the country
-264 positions as a Resident
-21 Residency Training
Programs
From: 60 different Schools
21 different States
Phase 2 – OSCE Based Test
600 candidates
Observer
Actress patient
Candidate
PHASE 2 - OSCE Selection Process
OBJECTIVE
• Correlation between Self Evaluation score
and OSCE grade in a Residency
Admission Test
CASUISTIC and METHODS
• Self-evaluation questionnaire
• 314 recently graduated physicians
• Performance on Objective Structured
Clinical Examination – OSCE
• Correlation between OSCE final grade and
self-evaluation of skill levels
CASUISTIC and METHODS
Self evaluation questionnaire
0
1
You do not know the procedure
2
You have seen another person perform the procedure
3
You have done the procedure with guidance of another
person
4
You have done the procedure correctly on your own
5
You have done the procedure correctly so many times
that you feel both competent and confident about it
You have read about or been told how the procedure is
done
STATISTICAL ANALYSIS
Pearson Correlation’s test
Self evaluation scores X OSCE grade
RESULTS
RESULTS
Station 2
100
90
80
70
60
50
40
30
20
10
0
OSCE grade
OSCE grade
Station 1
2
R = 0,0336
0
1
2
3
4
5
100
90
80
70
60
50
40
30
20
10
0
6
2
R = 0,0271
0
1
Self evaluation
2
OSCE grade
OSCE grade
2
R = 0,0622
2
3
5
6
Station 4
100
90
80
70
60
50
40
30
20
10
0
1
4
Self evaluation score
Station 3
0
3
4
Sel- evaluation score
5
6
100
90
80
70
60
50
40
30
20
10
0
R2 = 0,0536
0
1
2
3
4
Self-evaluation score
5
6
RESULTS
Station 6
100
100
80
80
OSCE grade
OSCE grade
Station 5
60
40
20
40
2
20
2
R = 0,0568
R = 0,0341
0
0
0
1
2
3
4
5
0
6
1
2
3
4
Self-evaluation score
Self-evaluation score
Station 7
Station 8
100
100
80
80
60
40
R2 = 0,0238
20
0
OSCE grade
OSCE grade
60
5
6
60
40
R2 = 0,0258
20
0
0
1
2
3
4
Self-evaluation score
5
6
0
1
2
3
4
Self-evaluation score
5
6
RESULTS
Station 10
100
80
80
60
40
2
R = 0,0055
OSCE grade
100
60
40
2
20
20
0
0
0
1
2
3
4
5
6
R = 0,0114
0
1
Self-evaluation score
2
3
100
90
80
70
60
50
40
30
20
10
0
R2 = 0,0449
0,0
4
Self-evaluation score
All 10 stations
OSCE grade
OSCE grade
Station 9
1,0
2,0
3,0
Self-evaluation mean
4,0
5,0
5
6
DISCUSSION
• The OSCE provides a nearer real clinical
situation
• The self-evaluation questionnaire indicates how
secure and confident was the candidate about
the skill asked in each station
• It would be expected a good correlation between
OSCE grade and self evaluation scores
• But no correlation was found
CONCLUSION
Young physicians could not self-evaluate
OSCE grades during Residency
Admission Test.