Back

MCTs: Good for Assessment, Good for Research, Good for Teaching

Let us consider some examples of using multiple-choice tests (MCTs) in the classroom and discuss the opportunities that they afford to the teacher.

A diagnostic test. Sometimes, the students have deeply held erroneous beliefs (commonly known as "misconceptions" or "preconceptions") which, if not confronted directly, may render instruction ineffective. Such misconceptions have been identified in the areas of astronomy1, physics2, mathematics3, and chemistry4 to name just a few. A well-constructed multiple-choice test allows the instructor to diagnose the prevalent misconceptions, address them directly during the instruction process, and then evaluate the effectiveness of instruction by administering an appropriate posttest. Since my specialty is physics, I am more familiar with diagnostic tests developed in this area—such as Force Concept Inventory5, Force and Motion Conceptual Evaluation6, and others—but, as far as I know, similar tests have been developed in other disciplines. The general effectiveness of MCTs in identifying misconceptions has been well established7: students tend to select the wrong answer options (known as "distractors") that correspond to the students' own beliefs. As a result, instructors and educational researchers can not only find out "who gets it" but also get fairly specific information about the state of mind of those who don't yet "get it."

A pretest before the new material is presented. This technique was pioneered by Eric Mazur from Harvard University8 and has been used increasingly widely at the college level.9 The students are assigned preliminary reading and are tested on the basic points of it in the beginning of class. Electronic technology allows the instructor to tally up the answers nearly instantly. The answers show if the class is, in general, ready for the presentation and what concepts in particular seem to have presented difficulty. In effect, the students ask questions by answering them. If, for instance, most students did not answer a certain question, the teacher may focus on the related concept in greater detail. If most students selected a particular distractor, it gives the teacher even more insight into the students' collective thinking (or the lack of preparation). This information allows for more effective lectures, making them more dynamic and interactive.

This method can be easily modified for instant "dip-sticking" at any point during the instruction: for instance, the teacher can ask a question in the middle of a class to make sure that the students follow the material. Low-tech solutions like color-coded cards raised by the students may serve as a good substitute for sophisticated equipment that many secondary schools may find difficult to afford.

Direct assessment. Since MCTs are easy to grade, they can serve as a way to administer a quick quiz at a point in the course when the teacher may not have time to administer and grade an open-response one. The results of such a quiz would provide another "data point" for the final grade and keep the students on their toes—which is not a bad thing. However, the ease with which the results can be analyzed would also serve to inform and enhance instruction—which is far more important. This "research" aspect of MCTs in classroom teaching is often overlooked. A typical teacher is overloaded with grading (it is not unusual for some of our colleagues to teach 120 students or more). Getting through such a pile of open-response tests is hard enough—who would blame the teacher for not conducting a thorough analysis of the results?

The use of MCTs, on the other hand, makes such an analysis quick and easy enough to be practical: the teacher can, in effect, become an educational researcher and inform his or her own instruction in fairly sophisticated ways. In addition to identifying misconceptions and pinpointing areas that need further study, it becomes possible to answer other important questions, for instance: Do my students find computational questions easier than qualitative ones? Are they capable of extracting certain types of information from a graph? Do my male students fall behind females on certain types of questions? Do the students who rush to complete the test fare better or worse than those who take their time? A creative teacher, armed with well-constructed MCTs, can significantly improve the quality of teaching and the feedback received by the students.

Contrary to what some opponents of MCTs may believe, multiple-choice questions can be quite challenging and require deep understanding of a particular concept. I once wrote a 10-question physics test, consisting entirely of conceptual questions on mechanics, and administered it to a large group of pre-AP physics students. Several questions were so hard that only 10 to 15 percent of the students got them right. Each item had four answer options—a group of hamsters trained to push buttons would have probably gotten close to 25 percent—but the distractors proved too juicy for most kids. Many questions on the AP Exams are also quite hard and are answered correctly by less than 25 percent of the students.

Combining MCTs with other kinds of assessment. I would never advocate MCTs as the sole (or even dominant) tool of assessing students' learning. However, multiple-choice questions complement other tools fairly well: since a typical MCT contains a relatively large number of questions, the teacher can test some "small" but important aspects of knowledge. If your course covers, say, 15 major concepts, it is impossible to create enough open-response problems to cover each of them, lest you want your students to spend the night at school finishing your test. Combining a few open-response problems with a larger number of multiple-choice ones can create a manageable and balanced test. A typical AP Exam, for instance, is a combination of multiple-choice and open-response questions, which makes it a more comprehensive and authentic assessment tool.10

Where does one get enough multiple-choice questions to use in the classroom?

It is hard to write even one good multiple-choice question. It is much harder, of course, to write a number of good questions. Imagine how difficult and time-consuming it is to write a number of good questions that actually complement each other well enough to form a meaningful test! A classroom teacher who wishes to write his or her own MCT may well possess sound knowledge of both content and pedagogy and still not be able to do the job due to the time constraints. Therefore, most of us must rely on already existing resources. What are they?

  • There are some established MCTs online (such as the already mentioned Force Concept Inventory). They tend to be "misconceptions-oriented." Read the accompanying comments to make sure that the test is suitable for the purpose that you have in mind.
  • Released AP questions are a great resource for teaching (not just for test-prepping). They are well crafted, carefully checked, and often accompanied by difficulty rating.
  • Many textbooks come with test banks. I find these questions to be, in general, more primitive and not as carefully written as the AP questions. Use with caution and be ready to find some errors.
  • AP test-prep books published by various companies all contain sample MCTs. I have seen a small number of such books (in general, I discourage my own students from using them), and I found many questions to be inadequate. If you choose to use an MCT from such books, use extreme care and be sure to look at each question closely before assigning it.
  • For my physics colleagues who want to challenge their students, check out the questions used in selecting the U.S. Physics Team:11 they are very hard and fun to work through.

Finally, you may opt to spend some time and write a few questions of your own for a specific topic. Not only will it make you appreciate the effort that goes into creating the AP Exams and other similar tests, you may actually enjoy the challenge as you try to predict the possible student errors and set your "evil traps." Some tips for writing good MCTs can be found online.12 In general, each question is expected to be unbiased and unambiguous and contain plausible distractors and only one unequivocally correct answer. For instance, the question that began this article is clearly not a good question: it is biased and confusing and has no answer that is definitely correct. Another example of a "bad" question, intentionally confusing, was written as a parody by Dr. Donald Simanek.13 Here it is, in all its delicious glory:

 

Which of the following is not an answer to this question?

  1. All of the responses below.
  2. This one.
  3. Some of the above.
  4. All of the above.
  5. None of these.

Okay, now that you had your chuckle (you did, didn't you?) and know what kinds of questions not to use, do consider incorporating MCTs in your everyday teaching. They are powerful tools for learning and teaching, good for much more than just "drilling for the test."

Notes

  1. www.planetarium.org/astronomy-survival/misconcp.html.
  2. www.physics.montana.edu/physed/misconceptions/. See also I. Halloun and D. Hestenes, "The Initial Knowledge State of College Physics Students" and "Common-Sense Concepts About Motion," American Journal of Physics 53 (1985): 1043-1065.
  3. See, for instance, www.counton.org/resources/misconceptions/index.shtml or www.amstat.org/publications/jse/v9n2/hirsch.html.
  4. The source http://tortoise.oise.utoronto.ca/~science/chemmisc.htm contains a comprehensive bibliography.
  5. The source http://modeling.asu.edu/R&E/FCIforw.html contains a detailed misconception analysis and related bibliography.
  6. R. Thornton and D. Sokoloff, Force and Motion Conceptual Evaluation (Medford, MA: Tufts University, 1999).
  7. See, for instance, P. Sadler, "Psychometric Models of Student Conceptions in Science: Reconciling Qualitative Studies and Distractor-Driven Assessment Instruments," Journal of Research in Science Teaching 35 (1998): 265-296; see also D. Treagust, "Development and Use of Diagnostic Tests to Evaluate Students' Misconceptions in Science," International Journal of Science Education 10 (1988): 159-169.
  8. E. Mazur, Peer Instruction: A User's Manual (Upper Saddle River, NJ: Prentice Hall, 1997). See also the Mazur Group Web site at: http://mazur-www.harvard.edu/education/educationmenu.php.
  9. See, for instance, M. Milner-Bolotin, "Tips for Using a Peer Response System in a Large Introductory Physics Class," The Physics Teacher 42 (2004): 253.
  10. I have only taught AP Chemistry, AP Physics B, and AP Physics C; my judgment is based on my own experience.
  11. www.compadre.org/psrc/evals/olympiad.cfm.
  12. My quick search rendered this source: www.psychwww.com/selfquiz/aboutq.htm.
  13. Originally published in Donald E. Simanek and John C. Holden, Science Askew: A Light-Hearted Look at the Scientific World (Institute of Physics Publishing, 2001).

Boris Korsunsky was born in Moscow, Russia. He has been living and teaching high school science in the U.S. since 1992. In his spare time, Boris wrote several thousand end-of-chapter problems for different textbooks and published two original collections of problems: Challenging Problems for Physics and Holt Physics Workbook. He has also written several articles on various aspects of teaching physics, plus many items for the national graduate admission tests and the state high school competency tests in physics and chemistry. A former coach of the U.S. Physics Team, Boris enjoys teaching, writing and conducting workshops for teachers. Boris holds graduate degrees in physics and physical chemistry from Moscow colleges and a doctorate from Harvard School of Education. He currently teaches freshman honors physics and AP Physics.

Authored by

Boris Korsunsky
Weston High School
Weston, Massachusetts