The question is designed
to test the ability to reason logically, and most people would choose
option number 1. People less able to reason logically might pick any one
of the options. Thus, the question appears to select against the less
able.
However, we are
accustomed to letters of the alphabet being arranged in linear series,
as in the text you are now reading. It is likely that both the person
who set the question, and most people who read it, would assume this,
probably quite unconsciously. However, if the letters were arranged in a
circle, then all five of the options would be correct.
A person with a
more subtle mind would see that the arrangement of A, B and C is not
specified. Instead of choosing the first option and quickly moving on to
the next question, he/she would have to exclude the circle arrangement
(because only one answer is correct) and perhaps other arrangements.
His/her progress through the text paper would be slowed. Thus the
question penalizes both
those
less able, and those more able, to reason logically.
This is not just
hypothetical. A study by the author of these web-pages found that,
indeed, multiple-choice questions select against
the more able students (Forsdyke 1978. A
comparison of short and multiple choice questions in the evaluation of
students of biochemistry. Medical Education
12, 351-356).
Another, albeit anecdotal,
example is by biochemist Jeffrey Toney in ASBMB Today (Dec
2017). The six year old boy was confronted with a pile of jig-saw pieces
and told to put them together. Many of the pieces in the pile had fallen
with the picture side down. The examiner did not tell him to be guided
by the picture. He had a quick eye for shape and assembled the puzzle as
a perfect rectangle with the picture face down. The teacher returned
when he had finished and clicked her timer. Next he knew, his mother was
in tears telling him he had failed the test!
Most individuals in systems
where research peer-review operates have already passed many selective
gates (e.g. admission to university, graduate and doctoral degrees,
appointment to a university or research institute). Thus, the less able
are already likely to have
been filtered out. All that is left for the peer-review gate to achieve
is discrimination between those of average ability and the more able.
This selection is against
the more able (i.e. those who can think subtly about difficult
problems). If the selection were merely by tossing a coin, the more able
would at least have the same chance as being selected as those of
average ability.
A Deep Integrity
So, if coin-tossing
will not give us at least a 50:50 chance of spotting researchers of high
ability, what will? Certainly, it is nothing to do with their
appearance. Imagine that you suddenly found yourself by chance on a
plane with your country's Olympic team. You would have little trouble
picking out the small petite teenager as a likely gymnast, the tall,
gangly, male as a likely high-jumper, and the heavy, thick-set male as a
likely shot-putter or weight-lifter.
If you were similarly on a plane
with your country's leading scientific researchers there would be no
similar yard-stick. Researchers of talent come in all shapes, sizes,
colours and sexes. The one common thread is a deep personal integrity
that makes it impossible
to engage in the marketing practices that the
peer review system, as it currently operates, requires. Biologist Leigh
Van Valen said it quite succinctly in 1976:
| "The norm of our science
remains dishonesty, because it is made necessary for the
survival of creative research. Often one may either be honest,
or continue in science, but not both." |
The
Olympians who share your flight did not get selected to represent their
country on the basis of what they proposed to do in the future. They
were selected because they had demonstrated that they were the agilest,
could jump the highest, and could lift the mostest! Similarly, if you
were on a plane with top musicians you would know that each musician
was there because he/she had repeatedly and consistently demonstrated
his/her musical excellence.
Yet, of our
researchers, while giving lip-service to track-record, we demand that
they market a project that will be assessed by "peers." This
requires that they discard projects that are difficult to market even
though they believe they are the best projects, and propose projects
that are easy to understand and are likely to receive matching funds
from the pharmaceutical industry. Scientists of deep integrity can no
more do this than the elfin gymnast could have lifted weights, or the
weight lifter could have high-jumped!
Fortunately,
there are some fine medical researchers of high integrity who somehow
make it in our strange peer review funding system. But if many medical
researchers are selected because of their skills in marketing, rather
than in research, should we be surprised that, as pointed out by
bioethicist, Jackie Smith (2003; Toronto
Globe & Mail. 14th Jan):
| "the
genetic era is seeing legitimate and rogue scientists lose sight
of human rights and dignity in a race for publicity, scientific
glory and lucrative contracts" |
And when these same people are called
upon as the acknowledged "experts" to advise on matters that
require high expertise, do we get the best advice?
Return to Peer Review Index
Return to HomePage
Established circa 2003 and last edited
13 Jan 2018
by Donald Forsdyke
| |