Proper Criticism

This brief guide by psychology professor Ray Hyman, a member of the CSICOP Executive Council from the beginning, has for many years been used by Skeptical Inquirer's editorial staff and widely distributed to authors and others. It was printed some years ago in the CSICOP newsletter Skeptical Briefs, and it appears in Hyman's book The Elusive Quarry, but it has never actually been published in SI. We thought our 25th anniversary would be a good time to do so.
-The Editors
Ray Hyman

--------------------------------------------------------------------------------

Since the founding of CSICOP in 1976, and with the growing number of localized skeptical groups, the skeptic finds more ways to state his or her case. The broadcast and print media, along with other forums, provide more opportunities for us to be heard. For some of these occasions, we have the luxury of carefully planning and crafting our response, but most of the time we have to formulate our response on the spot. But regardless of the circumstance, the critic's task, if it is to be carried out properly, is both challenging and loaded with unanticipated hazards.

Many well-intentioned critics have jumped into the fray without carefully thinking through the various implications of their statements. They have sometimes displayed more emotion than logic, made sweeping charges beyond what they can reasonably support, failed to adequately document their assertions, and, in general, failed to do the homework necessary to make their challenges credible.

Such ill-considered criticism can be counterproductive for the cause of serious skepticism. The author of such criticism may fail to achieve the desired effect, may lose credibility, and may even become vulnerable to lawsuits. But the unfavorable effects have consequences beyond the individual critics, and the entire cause of skepticism suffers as a result. Even when the individual critic takes pains to assert that he or she is expressing his or her own personal opinion, the public associates the assertions with all critics.

During CSICOP's first decade of existence, members of the Executive Council often found themselves devoting most of their available time to damage control-precipitated by the careless remarks of fellow skeptics-instead of toward the common cause of explaining the skeptical agenda.

Unfortunately, at this time, there are no courses on the proper way to criticize paranormal claims. So far as I know, no manuals or books or rules are currently available to guide us. Until such courses and guide books come into being, what can we do to ensure that our criticisms are both effective and responsible?

I would be irresponsible if I told you I had an easy solution. The problem is complicated and there are no quick fixes. But I do believe we all could improve our contributions to responsible criticism by keeping a few principles always in mind.

We can make enormous improvements in our collective and individual efforts by simply trying to adhere to those standards that we profess to admire and that we believe that many peddlers of the paranormal violate. If we envision ourselves as the champions of rationality, science, and objectivity, then we ought to display these very same qualities in our criticism. Just by trying to speak and write in the spirit of precision, science, logic, and rationality-those attributes we supposedly admire-we would raise the quality of our critiques by at least one order of magnitude.

The failure to consistently live up to these standards exposes us to a number of hazards. We can find ourselves going beyond the facts at hand. We may fail to communicate exactly what we intended. We can confuse the public about what skeptics are trying to achieve. We can unwittingly put paranormal proponents in the position of the underdogs and create sympathy for them. And, as I already mentioned, we can make the task much more difficult for other skeptics.

What, then, can skeptics do to upgrade the quality of their criticism? What follows are just a few suggestions. It is hoped they will stimulate further thought and discussion.

Be prepared. Good criticism is a skill that requires practice, work, and level-headedness. Your response to a sudden challenge is much more likely to be appropriate if you have already anticipated similar challenges. Try to prepare in advance effective and short answers to those questions you are most likely to be asked. Be ready to answer why skeptical activity is important, why people should listen to your views, why false beliefs can be harmful, and the many similar questions that invariably are raised. A useful project would be to compile a list of the most frequently occurring questions along with possible answers.

Whenever possible try your ideas out on friends and "enemies" before offering them in the public arena. An effective exercise is to rehearse your arguments with fellow skeptics. Some of you can take the role of the psychic claimants while others play the role of critics. And, for more general preparation, read books on critical thinking, effective writing, and argumentation.

Clarify your objectives. Before you try to cope with a paranormal claim, ask yourself what you are trying to accomplish. Are you trying to release pent-up resentment? Are you trying to belittle your opponent? Are you trying to gain publicity for your viewpoint? Do you want to demonstrate that the claim lacks reasonable justification? Do you hope to educate the public about what constitutes adequate evidence? Often our objectives, upon examination, turn out to be mixed. And, especially when we act impulsively, some of our objectives conflict with one another.

The difference between short-term and long-term objectives can be especially important. Most skeptics, I believe, would agree that our long-term goal is to educate the public so that it can more effectively cope with various claims. Sometimes this long-range goal is sacrificed because of the desire to expose or debunk a current claim.

Part of clarifying our objectives is to decide who our audience is. Hard-nosed, strident attacks on paranormal claims rarely change opinions, but they do stroke the egos of those who are already skeptics. Arguments that may persuade the readers of the National Enquirer may offend academics and important opinion-makers.

Try to make it clear that you are attacking the claim and not the claimant. Avoid, at all costs, creating the impression that you are trying to interfere with someone's civil liberties. Do not try to get someone fired from his or her job. Do not try to have courses dropped or otherwise be put in the position of advocating censorship. Being for rationality and reason should not force us into the position to seeming to be against academic freedom and civil liberties.

Do your homework. Again, this goes hand in hand with the advice about being prepared. Whenever possible, you should not try to counter a specific paranormal claim without getting as many of the relevant facts as possible. Along the way, you should carefully document your sources. Do not depend upon a report in the media either for what is being claimed or for facts relevant to the claim. Try to get the specifics of the claim directly from the claimant.

Do not go beyond your level of competence. No one, especially in our times, can credibly claim to be an expert on all subjects. Whenever possible, you should consult appropriate experts. We, understandably, are highly critical of paranormal claimants who make assertions that are obviously beyond their competence. We should be just as demanding on ourselves. A critic's worst sin is to go beyond the facts and the available evidence.

In this regard, always ask yourself if you really have something to say. Sometimes it is better to remain silent than to jump into an argument that involves aspects that are beyond your present competence. When it is appropriate, do not be afraid to say, "I don't know."

Let the facts speak for themselves. If you have done your homework and have collected an adequate supply of facts, the audience rarely will need your help in reaching an appropriate conclusion. Indeed, your case is made much stronger if the audience is allowed to draw its own conclusions from the facts. Say that Madame X claims to have psychically located Mrs. A's missing daughter and you have obtained a statement from the police to the effect that her contributions did not help. Under these circumstances it can be counterproductive to assert that Madame X lied about her contribution or that her claim was "fraudulent." For one thing, Madame X may sincerely, if mistakenly, believe that her contributions did in fact help. In addition, some listeners may be offended by the tone of the criticism and become sympathetic to Madame X. However, if you simply report what Madame X claimed along with the response of the police, you not only are sticking to the facts, but your listeners will more likely come the appropriate conclusion.

Be precise. Good criticism requires precision and care in the use of language. Because, in challenging psychic claims, we are appealing to objectivity and fairness, we have a special obligation to be as honest and accurate in our own statements as possible. We should take special pains to avoid making assertions about paranormal claims that cannot be backed up with hard evidence. We should be especially careful in this regard when being interviewed by the media. Every effort should be made to ensure that the media understand precisely what we are and are not saying.

Use the principle of charity. I know that many of my fellow critics will find this principle to be unpalatable. To some, the paranormalists are the "enemy," and it seems inconsistent to lean over backward to give them the benefit of the doubt. But being charitable to paranormal claims is simply the other side of being honest and fair. The principle of charity implies that, whenever there is doubt or ambiguity about a paranormal claim, we should try to resolve the ambiguity in favor of the claimant until we acquire strong reasons for not doing so. In this respect, we should carefully distinguish between being wrong and being dishonest.

We often can challenge the accuracy or validity of a given paranormal claim. But rarely are we in a position to know if the claimant is deliberately lying or is self-deceived. Furthermore, we often have a choice in how to interpret or represent an opponent's arguments. The principle tell us to convey the opponent's position in a fair, objective, and non-emotional manner.

Avoid loaded words and sensationalism. All these principles are interrelated. The ones previous stated imply that we should avoid using loaded and prejudicial words in our criticisms. If the proponents happen to resort to emotionally laden terms and sensationalism, we should avoid stooping to their level. We should not respond in kind.
This is not a matter of simply turning the other cheek. We want to gain credibility for our cause. In the short run, emotional charges and sensationalistic challenges might garner quickly publicity. But most of us see our mission as a long-run effort. We would like to persuade the media and the public that we have a serious and important message to get across. And we would like to earn their their trust as a credible and reliable source. Such a task requires always keeping in mind the scientific principles and standards of rationality and integrity that we would like to make universal.

About the Author

Ray Hyman, professor emeritus of psychology at the University of Oregon, is a Fellow and member of the Executive Council of CSICOP and a member of the Skeptical Inquirer editorial board.