INTRODUCTION TO BASIC CONCEPTS IN EXPERIMENTAL DESIGN

erroi is very bad and is to be avoided. In the present example, the decision
rule is biased in favor of deciding that the population mean is egual to 100
rather than, say, 103. In many research situations, the cost of a type I error
may be large relative to the cost of a type Il error. For example, to commit
a type | error in concluding that a particular medication arrests the pro-
duction of cancer cells and therefore can be used in place of other medical
procedures is a serious matter. On the other hand, falsely deciding that the
medication does not arrest the production of cancer cells (type II error)
would result in withholding the medication from the public and would
probably lead to further research. in such a context, a type II error is less
undesirable than a type | error. However, in another context, concluding
that an experimental effect js not significant may result in an experimenter
discontinuing a promising line of research whereas a type I error would
mean further exploration into a blind alley. Faced with these two alter-
natives, many experimenters might prefer to make a type | rather than a
type II error. It is apparent from the foregoing discussion that the loss
function associated with the two errofS must be known before a rational
choice concerning x can be made. However, experimenters in the behavioral
sciences are generally unable to specify the losses associated with the two
errors of inference. Therein lies the problem. The problem is resolved by
falling back on accepted conventions. The principal benefit of statistical
decision theory —that of using decision rules having optimum properties
for a given purpose—is seldom enjoyed by experimenters in the behavioral
sciences. A general introduction to the meaning of optimal solutions to
problems is given by Ackofi, Gupta, and Minas (1962).

- It is hoped that the preceding discussion helps to dispel the magic
that seems so inextricably tied to the 05 and .01 levels of significance.
The use of the .05 or .0l level of significance in hypothesis testing is a
convention. When elther level is achieved by a test, it signals that an im-
probable event has occurred or that the hypothesis under test has led to a
poor prediction. A test of significance provides information concerning the
probability of committing an error in rejecting the null hypothesis. lt is
one bit of information reguired in making a decision concerning a research
hypothesis. A test of significance embodies no information concerning
loss-values associated with the decision, the experimenter'S prior personal
convictions concerning the hypotheses, OT the importance or usefulness
of the obtained results. Various problems associated with the uncritical
use of significance tests in research have been examined in detail by Bakan
(1966). Bayesian statistical theory represents an attempt to incorporate
prior information into the decision process, information that is not utilized
within the classical theories of Neyman-Pearson and Fisher. A rapproche-
ment involving the best features of classical theory, decision theory, and
Bayesian theory is to be hoped for. Binder (1964) in a review mentioned one
modification of classical theory that incorporates Bayesian theory. A
general introduction to Bayesian theory can be found in Edwards, Lindman,
and Savage (1963) and additional references in Binder (1964).