TY - JOUR
T1 - The Long and Short of Summative Assessment in Competency Based Medical Education
T2 - Time to Raise the Bar?
AU - Lehl, Sarabmeet Singh
AU - Gupta, Monica
AU - Hande, Shyamala Handattu
AU - Hande, Manjunatha Handattu
N1 - Publisher Copyright:
© 2022 Journal of Association of Physicians of India. All rights reserved.
PY - 2022/2
Y1 - 2022/2
N2 - Background: The Competency-based medical education (CBME) has been introduced for MBBS programme in India from 2019. Reorganization of the assessment system is required to meet the challenges imposed by this new framework. Objectives: An evaluation of the university summative assessments held prior to the introduction of CBME-based curriculum was carried out to analyze the pattern, relevance and distribution of questions. Methods: Five sets of annual and supplementary summative examination papers from three universities, State (SU), Private (PU) and Medical (MU) were evaluated. The analysis included format i.e. Structured and Modified Essay questions, Short notes, other formats; marks distribution; terminology-based level of cognitive domain; subject-based relevance and topic-based distribution of questions. Results: A total of 352 questions were analyzed. The maximum number of questions were from the state university (140, 39.7%).The contribution of short notes in the theory papers was 65.8% (PU), 87.1% (SU) and 88.9% (MU).Only the PU had Modified-Essay Questions (10.5%) and Modified short notes(4.7%). Terminology addressing higher cognitive domains was low as the questions assessed mainly the knowledge level (80-96.8%), comprehension 3.1%-6.4%. Only the PU had problem solving questions comprising 11.7% of total questions. Majority of questions had moderate or high relevance and only 2.1%-8.2% were of low-relevance. Inter-university differences in the topics were observed for dermatology, psychiatry, and infections. Conclusions: The present evaluation of high-stakes assessment in three universities indicates minor differences in the format of questions. Questions were predominantly in the form of short notes, structured essays and modified essays were a rarity; few questions targeted higher cognitive levels although majority had high-moderate relevance.
AB - Background: The Competency-based medical education (CBME) has been introduced for MBBS programme in India from 2019. Reorganization of the assessment system is required to meet the challenges imposed by this new framework. Objectives: An evaluation of the university summative assessments held prior to the introduction of CBME-based curriculum was carried out to analyze the pattern, relevance and distribution of questions. Methods: Five sets of annual and supplementary summative examination papers from three universities, State (SU), Private (PU) and Medical (MU) were evaluated. The analysis included format i.e. Structured and Modified Essay questions, Short notes, other formats; marks distribution; terminology-based level of cognitive domain; subject-based relevance and topic-based distribution of questions. Results: A total of 352 questions were analyzed. The maximum number of questions were from the state university (140, 39.7%).The contribution of short notes in the theory papers was 65.8% (PU), 87.1% (SU) and 88.9% (MU).Only the PU had Modified-Essay Questions (10.5%) and Modified short notes(4.7%). Terminology addressing higher cognitive domains was low as the questions assessed mainly the knowledge level (80-96.8%), comprehension 3.1%-6.4%. Only the PU had problem solving questions comprising 11.7% of total questions. Majority of questions had moderate or high relevance and only 2.1%-8.2% were of low-relevance. Inter-university differences in the topics were observed for dermatology, psychiatry, and infections. Conclusions: The present evaluation of high-stakes assessment in three universities indicates minor differences in the format of questions. Questions were predominantly in the form of short notes, structured essays and modified essays were a rarity; few questions targeted higher cognitive levels although majority had high-moderate relevance.
UR - http://www.scopus.com/inward/record.url?scp=85125406286&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85125406286&partnerID=8YFLogxK
M3 - Article
AN - SCOPUS:85125406286
SN - 0004-5772
VL - 70
SP - 41
EP - 45
JO - Journal of Association of Physicians of India
JF - Journal of Association of Physicians of India
IS - 2
ER -