Short Answer Open-Ended versus Multiple-Choice Questions: A Comparison of Objectivity

Authors

  • Bharti Mehta Associate Professor, Physiology, All India Institute of Medical Sciences, Jodhpur-342005
  • Bharti Bhandari Assistant Professor, Department of Physiology, All India Institute of Medical Sciences, Jodhpur, Rajasthan, India. Mob. no. +91 8003996865, E-mail : drbhartibhandari@yahoo.co.in.
  • Parul Sharma RAK College of Medical Sciences, UAE
  • Rimplejeet Kaur Senior Resident, Pharmacology, All India Institute of Medical Sciences, Jodhpur

Keywords:

Multiple choice questions, medical education, assessment, open-ended questions.

Abstract

Objectives: We designed our study with the hypothesis that open ended Short Answer type Questions (SAQs), no matter how carefully framed, cannot be as objective as Multiple Choice type Questions (MCQs).

Methods: The study was conducted on 1st year MBBS students (n=99) studying at AIIMS, Jodhpur. Awritten test on 'Blood & Immunity' was conducted containing same questions in two formats; twelve MCQs (type E) in section Aand 12 SAQs in section B. Maximum marks for all questions in both sections were equal. All the answers of section B were evaluated separately by two different examiners to reduce the subjectivity and a model answer sheet for both the sections was prepared and provided to both the examiners.

Results: The difference in the scores in Section B SAQs that were evaluated by two different examiners was not statistically signiï¬cant. Mean of the marks awarded by the two examiners was taken as the ï¬nal score of each student in section B. The difference in the scores by the students in the two sections was also non-signiï¬cant (p=0.14). A signiï¬cant correlation (r=0.99, p<0.0001) was found in SAQ and MCQ scores. Bland- Altman analysis also showed no proportion of bias and the two methods of scoring were in agreement with each other.

Conclusion: The results suggest that meticulously-framed open-ended short answer type questions can be as objective as multiple choice type questions.

Keywords: Multiple choice questions, medical education, assessment, open-ended questions.

References

Schuwirth LWT, van der Vleuten CPM (2003). ABC of learning and teaching in medicine: Written assessment. BMJ 326 ( 7390 ) : 643–645.

Bhagat O, Bhandari B, Mehta B, Sircar S ( 2 0 1 4 ) . O b j e c t i v e Structured Practical Examination a n d C o n v e n t i o n a l P r a c t i c a l Examination: a Comparison of S c o r e s . M e d S c i E d u c 24(4):395–399.

Schuwirth LWT, van der Vleuten CPM (2004). Different written assessment methods: what can be said about their strengths and w e a k n e s s e s ? M e d E d u c 38(9):974–979.

Scoring rubric development: validity and reliability. Moskal, Barbara M & Jon A. Leydens [Internet]. [cited 2016 May 9]. A v a i l a b l e f r o m : http://pareonline.net/getvn.asp?v=7 &n=10

Pinckard RN, Mc Mahan CA, Prihoda TJ, Littleï¬eld JH, Jones AC (2009). Short-answer examinations improve student performance in an oral and maxillofacial pathology course. J Dent Educ 73(8):950–961.

Rabinowitz HK, Hojat M (1989). A comparison of the modiï¬ed essay question and multiple choice question formats: their relationship to clinical performance. Fam Med 21(5):364–367.

Anatol T, Hariharan S (2009). Reliability of the evaluation of students' answers to essay-type questions. West Indian Med J 58(1):13–16.

Wakeford RE, Roberts S (1979). A pilot experiment on the inter- examiner reliability of short essay q u e s t i o n s . M e d E d u c 13(5):342–344.

B l a n d J , A l t m a n D ( 1 9 8 6 ) . Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1(8476):307–310.

Pepple DJ, Young LE, Carroll RG (2010). A comparison of student performance in multiple-choice and long essay questions in the MBBS stage I physiology examination at the University of the West Indies (Mona Campus). Adv Physiol Educ 34(2):86–89.

M u j e e b A M , P a r d e s h i M L , Ghongane BB (2010). Comparative assessment of multiple choice questions versus short essay q u e s t i o n s i n p h a r m a c o l o g y examinations. Indian J Med Sci 64(3):118–124.

Rabinowitz HK ( 1986 ) . The modiï¬ed essay question: effect of

a u t h o r l o c a t i o n o n s t u d e n t p e r f o r m a n c e . M e d E d u c 20(4):318–320.

McCloskey DI, Holland RA (1976). A c o m p a r i s o n o f s t u d e n t performances in answering essay- type and multiple-choice questions. Med Educ 10(5):382–385.

Palmer EJ, Devitt PG (2007). Assessment of higher order cognitive skills in undergraduate education: modiï¬ed essay or m u l t i p l e c h o i c e q u e s t i o n s ? Research paper. BMC Med Educ 7:49.

Moeen-Uz-Zafar, Badr-Aljarallah (2011). Evaluation of mini-essay questions (MEQ) and multiple choice questions (MCQ) as a tool for assessing the cognitive skills of undergraduate students at the Department of Medicine. Int J Health Sci 5(2 Suppl 1):43–44.

Hift RJ (2014). Should essays and other “open-endedâ€-type questions retain a place in written summative assessment in clinical medicine? BMC Med Educ 14:249.

Forsdyke DR (1978). A comparison of short and multiple choice questions in the evaluation of students of biochemistry. Med Educ

(5):351–356.

H e t t i a r a t c h i E S ( 1 9 7 8 ) . A comparison of student performance in two parallel physiology tests in multiple choice and short answer forms. Med Educ 12(4):290–296.

Adewoye EO, Oyebola OD, B a m g b o y e E A ( 2 0 0 0 ) . A comparative study of students' p e r f o r m a n c e i n p r e c l i n i c a l physiology assessed by short and long essays. Afr J Med Med Sci 29(2):155–159.

Oyebola DD, Adewoye OE, I y a n i w u r a J O , A l a d a A R , Fasanmade AA, Raji Y (2000). A comparative study of students' p e r f o r m a n c e i n p r e c l i n i c a l physiology assessed by multiple choice and short essay questions. A f r J M e d M e d S c i 2 9 ( 3 - 4):201–205.

Anyika EN, Anyika DI (2008). Essay, multiple-choice (MCQ) and combined (essay with MCQ) type examinations: the pharmacy students' perspective. Niger Q J Hosp Med 18(1):12–15.

Published

2016-07-01

Issue

Section

ARTICLES