Short Answer Open-Ended versus Multiple-Choice Questions: A Comparison of Objectivity
Keywords:Multiple choice questions, medical education, assessment, open-ended questions.
Objectives: We designed our study with the hypothesis that open ended Short Answer type Questions (SAQs), no matter how carefully framed, cannot be as objective as Multiple Choice type Questions (MCQs).
Methods: The study was conducted on 1st year MBBS students (n=99) studying at AIIMS, Jodhpur. Awritten test on 'Blood & Immunity' was conducted containing same questions in two formats; twelve MCQs (type E) in section Aand 12 SAQs in section B. Maximum marks for all questions in both sections were equal. All the answers of section B were evaluated separately by two different examiners to reduce the subjectivity and a model answer sheet for both the sections was prepared and provided to both the examiners.
Results: The difference in the scores in Section B SAQs that were evaluated by two different examiners was not statistically signiï¬cant. Mean of the marks awarded by the two examiners was taken as the ï¬nal score of each student in section B. The difference in the scores by the students in the two sections was also non-signiï¬cant (p=0.14). A signiï¬cant correlation (r=0.99, p<0.0001) was found in SAQ and MCQ scores. Bland- Altman analysis also showed no proportion of bias and the two methods of scoring were in agreement with each other.
Conclusion: The results suggest that meticulously-framed open-ended short answer type questions can be as objective as multiple choice type questions.
Keywords: Multiple choice questions, medical education, assessment, open-ended questions.
Schuwirth LWT, van der Vleuten CPM (2003). ABC of learning and teaching in medicine: Written assessment. BMJ 326 ( 7390 ) : 643â€“645.
Bhagat O, Bhandari B, Mehta B, Sircar S ( 2 0 1 4 ) . O b j e c t i v e Structured Practical Examination a n d C o n v e n t i o n a l P r a c t i c a l Examination: a Comparison of S c o r e s . M e d S c i E d u c 24(4):395â€“399.
Schuwirth LWT, van der Vleuten CPM (2004). Different written assessment methods: what can be said about their strengths and w e a k n e s s e s ? M e d E d u c 38(9):974â€“979.
Scoring rubric development: validity and reliability. Moskal, Barbara M & Jon A. Leydens [Internet]. [cited 2016 May 9]. A v a i l a b l e f r o m : http://pareonline.net/getvn.asp?v=7 &n=10
Pinckard RN, Mc Mahan CA, Prihoda TJ, Littleï¬eld JH, Jones AC (2009). Short-answer examinations improve student performance in an oral and maxillofacial pathology course. J Dent Educ 73(8):950â€“961.
Rabinowitz HK, Hojat M (1989). A comparison of the modiï¬ed essay question and multiple choice question formats: their relationship to clinical performance. Fam Med 21(5):364â€“367.
Anatol T, Hariharan S (2009). Reliability of the evaluation of students' answers to essay-type questions. West Indian Med J 58(1):13â€“16.
Wakeford RE, Roberts S (1979). A pilot experiment on the inter- examiner reliability of short essay q u e s t i o n s . M e d E d u c 13(5):342â€“344.
B l a n d J , A l t m a n D ( 1 9 8 6 ) . Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1(8476):307â€“310.
Pepple DJ, Young LE, Carroll RG (2010). A comparison of student performance in multiple-choice and long essay questions in the MBBS stage I physiology examination at the University of the West Indies (Mona Campus). Adv Physiol Educ 34(2):86â€“89.
M u j e e b A M , P a r d e s h i M L , Ghongane BB (2010). Comparative assessment of multiple choice questions versus short essay q u e s t i o n s i n p h a r m a c o l o g y examinations. Indian J Med Sci 64(3):118â€“124.
Rabinowitz HK ( 1986 ) . The modiï¬ed essay question: effect of
a u t h o r l o c a t i o n o n s t u d e n t p e r f o r m a n c e . M e d E d u c 20(4):318â€“320.
McCloskey DI, Holland RA (1976). A c o m p a r i s o n o f s t u d e n t performances in answering essay- type and multiple-choice questions. Med Educ 10(5):382â€“385.
Palmer EJ, Devitt PG (2007). Assessment of higher order cognitive skills in undergraduate education: modiï¬ed essay or m u l t i p l e c h o i c e q u e s t i o n s ? Research paper. BMC Med Educ 7:49.
Moeen-Uz-Zafar, Badr-Aljarallah (2011). Evaluation of mini-essay questions (MEQ) and multiple choice questions (MCQ) as a tool for assessing the cognitive skills of undergraduate students at the Department of Medicine. Int J Health Sci 5(2 Suppl 1):43â€“44.
Hift RJ (2014). Should essays and other â€œopen-endedâ€-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ 14:249.
Forsdyke DR (1978). A comparison of short and multiple choice questions in the evaluation of students of biochemistry. Med Educ
H e t t i a r a t c h i E S ( 1 9 7 8 ) . A comparison of student performance in two parallel physiology tests in multiple choice and short answer forms. Med Educ 12(4):290â€“296.
Adewoye EO, Oyebola OD, B a m g b o y e E A ( 2 0 0 0 ) . A comparative study of students' p e r f o r m a n c e i n p r e c l i n i c a l physiology assessed by short and long essays. Afr J Med Med Sci 29(2):155â€“159.
Oyebola DD, Adewoye OE, I y a n i w u r a J O , A l a d a A R , Fasanmade AA, Raji Y (2000). A comparative study of students' p e r f o r m a n c e i n p r e c l i n i c a l physiology assessed by multiple choice and short essay questions. A f r J M e d M e d S c i 2 9 ( 3 - 4):201â€“205.
Anyika EN, Anyika DI (2008). Essay, multiple-choice (MCQ) and combined (essay with MCQ) type examinations: the pharmacy students' perspective. Niger Q J Hosp Med 18(1):12â€“15.
Authors who publish with this journal agree to the following terms:
Copyright will be with Annals of the National Academy of Medical Sciences (India).
The work will be simultaneously licensed under Creative Commons Attribution Licence CC-BY 4.0Âhttp://creativecommons.org/licenses/by/4.0/