Item-analysis of multiple choice questions: A pilot attempt to analyze formative assessment in pharmacology

The aim of the study was to assess the quality of multiple choice questions, for creating a viable question bank for future use. The study was conducted in the department of Pharmacology. The marks obtained by the second year MBBS students in the MCQ tests during Pharmacology formative examinations were analyzed. Each correct response was awarded 1 mark and each incorrect response was awarded 0, with no negative marking. Post validation of the paper was done by item analysis. Each item was analyzed for difficulty index, discrimination index and distractor effectiveness. Difficulty index of 24 (40%) items was in the acceptable range (p value 30-70%), 21 (35%) items were too easy (p value >70%) and 15 (25%) items were too difficult (p value <30%). Discrimination index of 12 (20%) items was excellent (d value >0.35), 36 (60%) items was good (d value 0.20-0.34) and 12 (20%) items were poor (d value <0.2%). A total of sixty items had 180 distractors. Amongst these, 60 (33.3%) were nonfunctional distractors, 63 (35%) were functional distractors and 57 (31.7%) had nil response i.e. not attempted by any student. On the basis of non-functional distractors, distractor effectiveness of each item was assessed. This study inferred that there is a genuine need for regular item analysis of multiple choice questions to generate a question bank with an intention to achieve uniform, reliable and valid assessment. Questions with average difficulty and high discriminating power with functional distractors should be incorporated into future tests to improve the validity and reliability of MCQ as a mode of formative assessment.