Contingency

Primacy Effects in Clinical Judgments of Contingency SHAWN P. CURLEY, PhD, MARK J. YOUNG, MD, MARGARET J.

KINGRY, PhD, RN, J. FRANK YATES, PhD In contingency judgment a primacy effect exists when a conclusion about the relationship between clinical variables is inordinately influenced by cases seen earlier rather than later in a presentation sequence. In this study, medical and nursing trainees evidenced this behavior in a hypothetical clinical judgment situation. The behavior was tied to an attention decrement explanation, by which inattention to the later-presented cases leads to inaccurate recall of the relative frequencies of observed cases, which in turn induces a misjudgment of a disease-finding contingency. An explicit intervention based on this hypothesis, forcing attention to later cases by warning that recall of the case frequencies would be required, was effective in reducing primacy effects among medical students. A related, but less explicit, intervention was also tried. This intervention did not significantly reduce primacy effects among nursing students, but was somewhat effective among general undergraduate students performing a non-clinical contingency judgment task. Key words: clinical judgment; diagnosis; decision making. (Med Decis Making 8:216-222, 1988)

ordinately influenced by cases seen earlier rather than a presentation sequence. Yates and Curley19 proposed that primacy effects in personality impression formation are only one manifestation of more general judgmental mechanisms that apply to contingency judgments as well. The operation of such judg-

Many important decisions made by health professionals rest on their judgments of the relationships among clinical variables. For example, a physician’s reluctance to prescribe for an elderly patient a drug that is routinely recommended for younger patients might be the result of a belief that the effectiveness of the drug is age-dependent. Health professionals’ knowledge and beliefs about relationships among clinical variables arise from a variety of sources. One source consists of the results of careful scientific study, as communicated in formal medical training and professional journals. However, an especially important source, particularly for knowledge specific to their own practice, is personal clinical experience. Indeed, examination of physicians’ expertise has indicated that a primary characteristic of the expert is his or her greater content-domain knowledge, presumably com-

later in

mental mechanisms is consistent with the active nature of clinical reasoning, as described by several researchers. 8,9,12 The diagnostic process has been observed to be highly dynamic, with hypotheses generated early in the patient interview and revised as information is gathered. The primacy hypothesis indicates that early information, and the generated hypotheses, can receive undue weight relative to later evidence. Given the probabilistic nature of disease processes, any small sample could be unrepresentative of its parent population. Divergent judgment policies, drawn from such samples, could easily result. During early clinical training, when experience is first being obtained, the influence of primacy would be particularly visible, with implications that may persist throughout their later

piled through practice.99 Smedslund16 and Jenkins and Ward 11,18 demonstrated that

people have limited proficiency in judging between variables from experience.

contingencies

research has extended their initial findreason for individuals’ misjudgments of ings.4,s contingencies is the presence of a &dquo;primacy effect,&dquo; as suggested by the personality impression literature.2,10 A primacy effect is indicated when judgments are in-

practice.

Subsequent

Andersonl

proposed that at least some primacy efjudgment are due to &dquo;attention decrement.&dquo; According to this hypothesis, specialized to the present situation, people will decrease their attention to the incoming data after viewing initial data relevant to the contingency. In effect, they rapidly form an initial judgment of the contingency, and then reduce the attention they devote to processing subsequent information. This process is related to the observation by Tversky and Kahneman 17 that subjective judgments conform to a &dquo;law of small numbers,&dquo; whereby population judgments are formed from small samples with undue certainty. Attention to later-presented cases di-

One

fects in

Received April 21, 1987, from the University of Minnesota, MinMinnesota (SPC), the University of Michigan Medical Center, Ann Arbor, Michigan, (MJY), Michigan State University, East Lansing, Michigan (MJK), and the University of Michigan, Ann Arbor, Michigan (JFY). Revision accepted for publication January 8, 1988. Address correspondence and reprint requests to Dr. Curley: Department of Management Sciences, University of Minnesota, 271 19th Avenue S, Minneapolis, MN 55455.

neapolis,

216 Downloaded from mdm.sagepub.com at UNIV OF MICHIGAN on March 24, 2015

217

minishes because the person believes that such information is superfluous. The aim of the present research was to test for both the existence of a primacy effect and the attention decrement hypothesis for the effect. Of particular interest was the role of primacy in influencing physicians and nurses during training, and the possibility of reducing primacy effects through appropriate interventions based on the attention decrement hy-

pothesis.

Method The basic design to check the validity of the primacy and attention decrement hypotheses is adapted from an approach used by Anderson and Hubert.3 Subjects are sequentially presented with cases. Each case, displayed on a slide, shows one of two complementary events for each of two variables, e.g., a clinical finding variable (present or absent) and a disease variable (present or absent). The subject must determine whether, to what extent, and in what direction the variables are related. Overall, there is no relationship between the variables in the sequences that the subjects see. However, the case order is such that there is in fact a continand an gency in one direction earlier in the direction equally strong contingency in the opposite later in the series. A primacy effect would suggest sensitivity to the earlier contingency. In addition to judging any contingency, subjects are required to recall the frequency with which each of the four possible case types occurred. However, only some subjects are forewarned of the recall test; the others are not. As a general intervention procedure, we would expect the recall task to force attention to the later cases, reducing primacy effects in those who were forewarned. This technique is derived from the attention decrement hypothesis, which states that primacy is due to insufficient concern for these later

series,

cases.

The attention decrement hypothesis, as an explanation for observed primacy effects, argues two separable points. The propositions are that the subject inadequately attends to later cases, implying inaccurate recall of the cases seen, and that the subject uses the inaccurate data correctly. In terms of the present experiment, if the attention decrement hypothesis is sufficient as an explanation of any observed primacy effects, then two claims should be supported. First, we would expect foreknowledge of the recall task to force attention to the later cases and reduce primacy effects. Second, any primacy effect should be due solely to misremembering case frequencies, and not to inappropriately transforming the recalled frequencies into probability statements. That is, a subject’s judged contingency, as calculated directly from stated prob-

ability judgments, would not differ from the judged contingency implied by his or her recall frequencies. Measures

are

hypothesis

defined to test the attention decrement these bases.

on

SUBJECTS

Three groups of subjects were used in the study. Medical Students. These were 38 students in their fourth year of the six-year combined BS/MD curriculum at the University of Michigan Medical School. The students were gathered for a class about decision mak-

ing in medicine, and were not paid for their participation in the study. The content of the class was not related to the content of the task. Nursing Students. One hundred and five (105) junior and senior nursing students were recruited by announcement in several classes at the College of Nursing of Michigan State University. Each was paid a nominal amount for participation in one of eight large group sessions used in the study. General Undergraduates. One hundred and twentynine (129) subjects were recruited from a subject pool maintained by the Department of Psychology at the University of Michigan. The subjects in this group were mainly undergraduates in the liberal arts college. The subjects were paid for their participation in the task, which was completed along with several other unrelated tasks in small groups of three to five subjects each. The responses of these subjects have been described elsewhere by Yates and Curley.19 STIMULI z

Medical Students. The medical student subjects in a two-task session as described below. For both tasks, subjects considered a hypothetical clinical sign and a pair of diseases. For task 1, the sign was skin redness, either present or absent, and the diseases were labeled disease A and disease B. For task 2, the sign was fever, either present or absent, and the diseases were labeled disease X and disease Y. For each task, the subjects were shown a purported random sample of patients with either of the two appropriate diseases for that task. Each observation was displayed on a slide indicating the presence or absence of the clinical finding and the disease. Nursing Students. The stimuli presented to the nursing student subjects were the same as those used in task 1 for the medical student subjects. General Undergraduates. The general undergraduate student subjects considered a hypothetical plant called the &dquo;Rhododipsia.&dquo; Rhododipsia were from either region A or region B, and were either light or dark in color. Observations from a purported random sample of Rhododipsia from regions A and B were displayed on slides, each of which indicated the specimen’s region, either A or B, and color, either light or dark.

participated

Downloaded from mdm.sagepub.com at UNIV OF MICHIGAN on March 24, 2015

218

block 1 followed by block 2, labeled order 1 for this task. (Nursing and undergraduate students saw two orders; see below.) Following the data presentation, subjects judged the conditional probabilities of the sign &dquo;redness,&dquo; given each of disease A and disease B. They also directly estimated the strength and direction of the relationship between clinical finding and condition. For task 2, the medical students judged the covariation between a different clinical sign (fever or no fever) and patient condition (disease X or disease Y). For this task, block 1 suggested the co-occurrence of fever with disease X; block 2 suggested the co-occurrence of fever with disease Y. The students again saw slide order 1, defined as in task 1. Following the contingency judgments for this task, subjects were to recall the number of slides of each type that they saw in the slide presentation. Thus, the medical students’ procedure involved a

single within-subjects manipulation designed

FIGURE 1. Distributions of patient cases according to clinical finding and disease used for nursing students and for medical students in task 1: a, block 1; b, block 2; c, overall.

Medical Students.

In task 1, the medical students

extent and dicovariation between clinical potential condition or no and (redness redness) patient finding (disease A or disease B). The data instances were presented on 28 slides, each with a specified clinical sign and disease. The slides were arranged in two 14-slide blocks so that for block 1 of data instances, there was a relationship in one direction (redness-disease A); and for block 2, there was an equally strong relationship in the opposite direction (redness-disease B). The extents and directions of these relationships are summarized by the distributions for blocks 1 and 2 in figures la and lb, respectively. Overall, there was no relationship between clinical finding and condition in the series, as illustrated by figure ic. The medical students all saw the same slide order, to

determine, from the data, the

rection of the

light color-region B contingency. The design of the experiment was a 2 X 2 factorial. Both factors were manipulated between subjects, yielding four groups for both the nursing and the general undergraduate students. The first factor was the order of presentation of the two data halves. Half the subjects received order 1, seeing the block 1 slides followed by the block 2 slides; the remaining students received order 2, seeing the block 2 slides followed by the block 1 slides. Thus, for the nursing students receiving order 1, the relationship seen first was such a

PROCEDURE

were

to test

the warning factor as a direct educational intervention. In the first task, subjects were not warned of a memory test following the slide presentation; and, in fact, no memory test was given. In the second task, the subjects were forewarned, and there was a memory test. Nursing and General Undergraduate Students. These two groups engaged in an identical procedure, as described in detail by Yates and Curley19 for the stimuli seen by the general undergraduate students. The procedure was an adaptation of that used for the medical students, and the differences from the design just described are outlined here. The main task was the judgment of contingencies based on a presentation of 28 slides in two blocks. For the nursing students, blocks 1 and 2 were identical to those used for the medical students in task 1 (fig. 1). For the undergraduate students, block 1 showed a light color-region A contingency) and block 2 showed

that redness tended to co-occur with disease A, but not with disease B. For the remaining nursing students, the opposite relationship was seen first, with redness tending to co-occur with disease B. The second factor of the design was whether or not the subject was warned, before the data presentation, of the recall test that would be required later. Forewarning was an intervention upon the task, although,


219

for these subjects, an indirect one. To the extent that the attention decrement hypothesis obtained, forewarning should have tended to reduce any primacy effects. However, unlike for the medical student subjects, there was no explicit connection between the recall and contingency judgment tasks, and forewarned subjects may not have perceived the recall intervention as an aid in performing the judgment task. Thus, the judgments made by the nursing and undergraduate students were equivalent to those provided by the medical students in the warning condition. For example, in all four experimental conditions, the nursing students judged the conditional probabilities of the sign &dquo;redness,&dquo; given each of disease A and disease B. They also directly estimated the strength and direction of the relationship between clinical finding and condition, and recalled the number of slides of each type that they saw in the presentation. MEASURES

Two

separable contingency indices, A’ and 3J , were

defined. The

derived from different the students; and, consejudgments provided by measured different observable quently, they aspects of the process underlying the contingency judgments. First, consider Op as an indication of a judged contingency in the direction observed in block 1. For example, for a medical student subject in task 1 or a nursing student subject:

Ap

=

measures were

P’(redness~disease A) P’(rednessl disease

B)

(1)

where P’ was the probability judgment stated by the subject, and block 1 indicated a redness-disease A contingency. This probability-based contingency index properly compares the two conditional probabilities needed to judge co-occurrence of the clinical finding and diseases. To the extent that the finding and diseases covary, redness will be more greatly associated with one disease than the other, and Ap will differ from zero. The measure indicates the extent to which the subjects’ probability judgments evidenced the presence of a contingency between variables in the direction suggested by block 1. For example, suppose physician 1 says that there is a 35% chance that a patient with disease A has skin redness, but that there is only a 15% chance that a patient with disease B has the sign. In contrast, physician 2 believes that the probability of a disease A patient having red skin is 40%o, and that there is an even higher probability of 50% that a disease B patient has red skin. Since each physician feels that the chance of skin redness depends on the patient’s disease, both

believe that the sign is related to the illness. Since the absolute size of the difference in probability judgments is greater for physician 1 than for physician 2 (3§ 0.20 vs. 3) = - 0.10), physician 1 thinks that the relationship is stronger. And because the differences are opposite in sign, the physicians also disagree in what they perceive as the direction of the relationship. If an individual would correctly recognize that no contingency existed, he or she would judge ð.~ 0, regardless of slide order. If a primacy effect obtained, then Ap > 0 would hold for subjects seeing order 1, and Ap < 0 would hold for subjects seeing order 2. That is, a significant order effect would obtain. If the attention decrement hypothesis has explanatory power, then this order effect would hold only for those subjects receiving no warning of the recall task. According to the first claim of the attention decrement hypothesis, forewarned subjects would show no order effect. Thereby, an intervention for reducing primacy effects would also be determined. To test the second claim of the attention decrement explanation for observed primacy effects, recalled frequencies were obtained for the appropriate subject groups. Let f’ denote the subject’s estimated frequency for each of the four types of case information in an observed slide presentation (fig. 1), e.g., f’(red-A) was the number of patients stated to have redness and disease A. Notice that if a subject had accurate overall recall, then his or her reported frequencies f’ would sum to 28, the actual number of slides used. This overall accuracy was measured. Further, using the reported frequencies f’, a frequency-based contingency index, Af, was defined. The judged frequency-based contingency index A’was calculated from the subjects’ reported cell frequencies in the recall test, as opposed to their conditional probability judgments. For example, the definition for a nursing student subject: =

=

-

Equation 2 parallels equation 1. However, whereas the elicited probabilities were used in defining A’, the elicited frequency estimates f’ were used in defining 0’ . Similarly, the interpretation of the frequency-based contingency index 3gparallels that for the probabilitybased contingency index 3) ; however, the two indices need not correspond. For example, suppose physician 1, with ~~ 0.20, further judged f’ (red-A) 6, f’ (not =

=

red-A)

=

9, f’ (red-B)

=

4, and f’(not red-B)

=

9. These

frequencies indicate that physician 1 believes the relative frequency with which a disease A patient had


220

Results

.

Subjects in all groups were accurate in judging the total number of slides they saw. The overall mean sums of f’, over all four case types, were 28.11 in the warning conditions, and 27.42 in the no-warning conditions. These means did not differ significantly from each other for any of the groups in the study. Overall, the students’ recall was accurate. Figure 2 displays the mean value of Ap for each condition of the study. The index ð.~ is shown as a function of slide presentation order and presence of recall test forewarning. Recall that, for the medical students, only one order was used and the warning manipulation was within-subjects, not between-subjects. First, consider the existence of primacy effects, both with and without forewarning. The medical student subjects showed an improvement in performance when forewarned of the recall test as an explicit intervention. The mean of Ap significantly exceeded 0 when they were not forewarned (t(37) 2.87, p < 0.01); however, mean Ap did not significantly exceed 0 when they were told in advance of the recall test (t(37) 1.70, p = 0.1). The two conditions differed significantly (t(37) = 2.08, p < 0.05). These results are consistent with both the primacy hypothesis and the effectiveness of the more explicit recall intervention. Now, consider the general undergraduate students. For these subjects, a primacy effect would be indicated by a significant effect of presentation order. When they were not forewarned, a primacy effect was evident: The mean of ð.~ was higher when the block of slides containing a light-A contingency was presented first (t(62) = 3.70, p < 0.001). Also, the intervention was effective : there was not~ a primacy effect when subjects were forewarned (t(62) 1.29, p > 0.1). This result has been reported for these subjects by Yates and Curley.19 Unlike the medical and general undergraduate students, the nursing students showed a primacy effect both with and without forewarning (t(50) 3.95, p < 0.001; t(51) 2.18, p < 0.05, respectively). Thus, the primacy hypothesis was supported, but the less explicit intervention was not effective in reducing the primacy effect for these students. Further evidence regarding the attention decrement hypothesis was obtained by comparison of Op and 0 f . Figure 3 shows the mean value of Of for each condition of the study. The index 3J is shown as a function of slide presentation order and presence of recall test forewarning. Using this index, the undergraduate students showed a primacy effect when not forewarned (t(57), p < 0.05), but not when forewarned (t(60), p > 0.1); and the nursing students showed a primacy effect with and without forewarning (t(50) 3.76, p < 0.001; t(51) 2.36, p < 0.05, respectively). The pattern and statistical reliability of the indices A’derived from the subjects’ recall data parallel the pattern and statistical =

=

FIGURE 2. Mean judged contingency index At inferred from subjects’ conditional probability judgments, as a function of experimental group, presentation order, and recall test forewarning. Circles and thin lines identify the mean responses of the general undergraduates, squares and thick lines identify those of the nursing students, and triangles identify the medical students’ mean responses. Warning conditions are identified by open figures and no-warning conditions by closed figures.

red skin was 6/15 0.40, and the relative frequency with which a disease B patient had red skin was 4/ 13 0.31, implying 3§= 0.09. The discrepancy between ~~ and Ofindicates that physician 1 had difficulty in translating the recalled frequencies into conditional probability judgments. This difficulty was in addition to the subject’s difficulty in recalling the frequencies. If physician l’s recall were perfect, all frequencies f’ would be 7, implying Of 0. To the extent that Op and Of did correspond, the subjects’ judged contingencies Ap can be described as accurate with respect to their recall of the relative frequencies, as indexed by 3g . That is, if the measures corresponded, then the contingencies were correctly derived from the recalled frequencies, even though the students’ actual recall may have been inaccurate, as indexed by Af’. The attention decrement hypothesis would predict correspondence between the measures on this basis. To the extent that the two measures did not correspond, the attention decrement hypothesis is inadequate, although it may still be partially explanatory of any observed primacy effects. =

=

=

-

=

=

=

=

=


221

described for the indices Ap derived from their conditional probability judgments. Together, these data provide some, but not complete, support for the attention decrement hypothesis. Forewarning was not always sufficient to counteract primacy effects, as predicted by the attention decrement hypothesis. However, consistent with the hypothesis, it is apparent that the students falsely observed a contingency largely as a consequence of misremembering the cases they observed. There were no statistically significant differences between the indices Op and A’for any of the subject groups in the study. Thus, there was no detectable, statistically reliable effect that influenced the students’ contingency judgments beyond their inaccuracy in recalling the frequencies. Interestingly, the recall inaccuracy was relative, in that the subjects’ recall of the overall frequency of total

reliability

cases was

quite

accurate.

Discussion People have been shown to have limited proficiency contingency judgment tasks, perhaps because they use suboptimal rules in determining contingencies.4,S,lS,16 Judges are also influenced by the method at

in which cases

are

presented)4,14,18 the way the relevant

variables are described,5 and prior conceptions about the relationship. 6,1 A &dquo;primacy effect&dquo; is another factor that influences contingency judgments. It is indicative of a more general judgmental mechanism, and is particularly relevant in general medical training. In the clinical environment, physicians’ and nurses’ experiential learning of the co-occurrence of potentially clinically related variables can be unduly influenced by the earlier cases seen. In this study, medical and nursing trainees manifested this behavior in simulated clinical judgment situations. The present findings indicate that attention decrement may partially explain observed primacy effects. Inattention to later-presented cases was one factor leading to inaccurate recall of the relative frequencies of cases seen, which in turn led to inaccurate judgments of contingency. The attention decrement hypothesis suggested an instructional strategy that potentially would be effective in reducing primacy effects, i.e., instruction that forces attention to all relevant data could improve the judgment of medical and nursing trainees by overcoming the natural tendency to pay little attention to later case information. This instruction was effective with the medical students, for whom the instruction was an explicit intervention upon the contingency judgment task. The recall forewarning was necessarily perceived by these students as an aid for the contingency judgment task, and was used by them appropriately to improve their recall and contingency

judgments.

FIGURE 3. Mean judged contingency index 3§ inferred from subjects’ frequency recall judgments, as a function of experimental group, presentation order, and recall test forewarning. Circles and thin lines identify the mean responses of the general undergraduates, squares and thick lines identify those of the nursing students, and the triangle identifies the medical students’ mean response. Warning conditions are identified by open figures and no-warning conditions by closed figures.

For the

and

general undergraduate stuless explicit, with the inbeing indirectly tied to the contingency

nursing

dents, the intervention tervention

was

judgment task. The students who received the forewarning in these groups would not necessarily frame the instruction as an aid for the contingency judgment task. The more indirect intervention was successful for the general undergraduate students only. A primacy effect remained in evidence even with forewarning for the nursing students. Given the differences in stimuli and subject populations between these two student groups, the particular factors leading to the differential effectiveness of forewarning cannot be determined with certainty from the present study. The findings of the existence of a primacy effect in a clinical judgment task and of the effectiveness of interventions are of interest within the limitations of the research design. The design involved artificial exercises, and students early in their training. The cases were observed in a single sitting, and not over a prolonged period of time as is more characteristic of clinical practice. Future research should examine the hypotheses within situations more representative of


222

clinical practice and for clinical professionals with more advanced training. Also of interest is the connection between these judgments and medical and nursing

6.

Chapman LJ, Chapman JP: Genesis of popular but erroneous psychodiagnostic observations. J Abnorm Psychol 72:193-204,

7.

Chapman LJ, Chapman JP: Illusory correlation as an obstacle to the use of valid psychodiagnostic signs. J Abnorm Psychol

8.

Eddy DM, Clanton CH: The art of diagnosis: solving the clinicopathological exercise. N Engl J Med 306:1263-1268, 1982 Elstein AS, Shulman LS, Spraika SA: Medical Problem Solving: An Analysis of Clinical Reasoning. Cambridge, MA, Harvard Uni-

1967

decisions.&dquo; The dynamic nature of the clinical environment motivates clinical decision makers to actively evaluate the information they receive in light of their present knowledge. To the extent that their present knowledge is determined on the basis of small samples of cases, the early experiential knowledge may be uncharacteristic of the underlying population of interest. A primacy effect, as observed in the present study, indicates an overweighting of this earlier experience at the expense of subsequent experience. This tendency is both inappropriate and potentially avoidable with appropriate interventions.

74:271-280, 1969

9.

versity Press, 1978 10. Hendrick C, Costantini AF: Effects of varying trait

11.

12. 13. 14.

References 1.

2. 3.

4.

5.

Anderson NH: Foundations of Information Integration Theory. New York, Academic Press, 1981 Anderson NH, Barrios AA: Primacy effects in personality impression formation. J Abnorm Soc Psychol 63:346-350, 1961 Anderson NH, Hubert S: Effects of concomitant verbal recall on order effects in personality impression formation. J Verbal Learning Verbal Behav 2:379-391, 1963 Arkes HR, Harkness AR: Estimates of contingency between two dichotomous variables. J Exp Psychol [General] 112:117-135, 1983 Beyth-Marom R: Perception of correlation reexamined. Memory and Cognition 10:511-519, 1982

15. 16. 17. 18. 19.

inconsistency

and response requirements on the primacy effect in impression formation. J Pers Soc Psychol 15:158-164, 1970 Jenkins HM, Ward WC: Judgment of contingency between responses and outcomes. Psychol Monographs [General and Applied] 79(1, Whole No. 594), 1965 Kassirer JP, Gorry GA: Clinical problem solving: a behavioral analysis. Ann Intern Med 89:245-255, 1978 Seggie JL, Endersby H: The empirical implications of Piaget’s concept of correlation. Aust J Psychol 24:3-8, 1972 Shaklee H, Mims M: Sources of error in judging event covariations: effects of memory demands. J Exp Psychol [Learn] 8:208224, 1982 Shaklee H, Tucker D: A rule analysis of judgments of covariation between events. Memory Cogn 8:459-467, 1980 Smedslund J: The concept of correlation in adults. Scand J Psychol 4:165-173, 1963 Tversky A, Kahneman D: The belief in the "law of small numbers." Psychol Bull 76:105-110, 1971 Ward WC, Jenkins HM: The display of information and the judgment of contingency. Can J Psychol 19:231-241, 1965 Yates JF, Curley SP: Contingency judgment: primacy effects and attention decrement. Acta Psychol 62:293-302, 1986
