|
The purpose of a multiple choice item is to measure
candidate ability with regard to a specific content area. A multiple choice item has a stem which asks
a question, describes data or presents a situation. The responses include a keyed correct response
and three or four distractors or foils. The way the item is framed and the type
of response required determines whether the item is Recall, Interpretation, or
Problem Solving.
Recall items simply ask candidates to recall or
recognize a fact. Interpretive items
require candidates to use their base of knowledge to interpret data or other
information and come to some conclusion.
Problem solving items require the candidate to assess a situation,
synthesize with information from their base of knowledge, and then correctly solve
a problem or make a decision. The
distribution of items among the three item types is an issue that bears some
consideration.
First, most certification examinations are intended
to evaluate the capability of the candidate to practice. Is this measured better through asking
questions that test specific bits of knowledge or is it more useful to
structure items so that they require the candidate to apply their knowledge in
a specific scenario? Second, problem
solving items are often longer than recall items, consequently requiring more
time to read, interpret and answer.
Therefore, a test of predominately problem solving items could require
more testing time. Third, it appears
that problem solving items are more difficult to write because there is more
likely to be disagreement concerning the correct response, or there actually is
more than one way to go about solving that problem. Consequently, to insure clarity, problem
solving items take more time and effort to write. Fourth, there is a fine line between how much
is enough information and how much is too much.
Problem solving items must be written succinctly, yet provide sufficient
information to answer the questions.
This often involves a great deal of thought and input from more than one
person. Interpretation items are often easier to write
in association with a visual. For
example, many medical and dental specialties require that the candidates be
able to read x-rays of various types.
Thus, it is fairly easy to find a good x-ray and ask the candidate to
interpret the x-ray and reach a diagnosis.
If the treatment plan is requested in addition to interpreting the x-ray,
the item becomes problem solving.
The usual recommendation is that items be
divided as follows: 33% recall; 33% interpretation; and 33% problem
solving. There is no data to support the
validity of this distribution. However,
logic suggests that when items are distributed this way, candidates are measured
on more than just the ability to recall facts.
They also must apply these facts in appropriate circumstances. This
causes the items in the test to be viewed as closer to practice, but within the
context of the multiple choice exam. One
interesting observation is that problem solving items are usually no more
difficult statistically than recall items overall.
|