Don’t Know Responding… 1 RUNNING HEAD: “DON’T KNOW RESPONDING…”
“Don’t know” responding to answerable and unanswerable questions during misleading and hypnotic interviews.
Alan Scoboria University of Windsor Giuliana Mazzoni University of Hull Irving Kirsch University of Hull
Keywords: memory, metacognition, interview, don’t know response, misleading question, hypnosis
In Press: Journal of Experimental Psychology: Applied
Corresponding author: Alan Scoboria Department of Psychology 401 Sunset Windsor, Ontario, Canada, N9B 3P4
[email protected] 518-253-3000 x4090
Don’t Know Responding… 2 Abstract “Don’t know” (DK) responses to interview questions are conceptually heterogeneous, and may represent uncertainty or clear statements about the contents of memory. A study examined the subjective intent of DK responses in relation to the objective status of information queried, in the context of memory distorting procedures. Participants viewed a video and responded to answerable and unanswerable questions phrased in misleading or non-misleading formats, while hypnotized or not hypnotized. Subjective meanings of DK responses were queried, and a recognition measure assessed the contents of memory. Lower DK and accuracy rates were consistently associated with unanswerable and misleading questions. One-third of DK responses were statements that the information had not been presented. When these were recoded, accuracy estimates for answerable questions decreased and more so for hypnotized participants. These results demonstrate that DK responses convey different types of information, thus accuracy estimates in studies which permit DK responses may be mis-estimated. Robust risks associated with asking unanswerable questions and asking questions at all were observed. Implications for working with DK responses during interviews are discussed.
Don’t Know Responding… 3 Introduction Empirical work on the quality of responses given to interview questions has focused primarily upon accuracy of responding (e.g. Köhnken, Milne, Memon, & Bull, 1999), and discrimination between correct and incorrect (often misleading) information (e.g., Loftus & Zanni, 1975). Such work has documented numerous factors which influence the amount and quality of information obtained (see Gudjonsson, 2003; Wright & Davies, 2007, for reviews). Less well understood is another common answer to interview questions: the “I don’t know” (DK) response. DK responses are a frequently occurring, naturalistic aspect of responding. For example, in one study adults responded DK to 60% of questions on average (Roebers & Fernandez, 2002). Some researchers argue that DK responses reflect the outcome of metacognitive monitoring of the contents of memory, as they represent one way by which responding to questions is avoided when confidence in potential answers is low (Koriat & Goldsmith, 1996; Nelson & Narens, 1994). Disallowing DK responses and forcing responding promotes guessing and substantially undermines accuracy (Koriat, Goldsmith, Schneider & Nakash-Dura, 2001; Roebers, Moga & Schneider, 2001). Thus, the quality of information obtained during interviews depends upon whether or not DK responses are permitted, and the degree to which they are encouraged or discouraged. DK responses have been examined to a degree in the children’s interviewing literature. This work has found that young children do not make spontaneous DK responses as often as adults (Ceci & Bruck, 1993); that DK response rates vary by chronological age (Ackerman, 1993; Beuscher & Roebers, 2005); and that training in DK responding (Saywitz & Moan-Hardie, 1994; Nesbitt & Markham, 1999) or incentives for accurate responding (Roebers, Moga & Schneider, 2001; Roebers & Schneider, 2002) may result in increased use of DK responding, but
Don’t Know Responding… 4 at the risk of encouraging a DK response set (Moston, 1987). Furthermore, DK responding is substantially attenuated by misinformation (Howie & Dowd, 1996; Roebers & Fernandez, 2002). Finally, research with children has shown important developmental progressions in the ability to effectively use DK responses under conditions of uncertainty (Roebers, von der Linden, & Howie, 2007; Roebers, von der Linden, Schneider, & Howie, 2007). Research with adults demonstrates that various factors can affect DK response rates. Loftus and Zanni (1975) found that questions containing definite articles (e.g. “the”) resulted in fewer DK responses than questions containing indefinite articles (e.g. “a”). Gudjonsson and Hilton (1989) reported that instructions encouraging DK responses resulted in resistance to misinformation. Scoboria, Mazzoni, Kirsch and Milling (2002) found that both misleading questioning and hypnosis resulted in reduced DK rates upon presentation of misinformation and/or hypnotic induction (see also Scoboria, Mazzoni & Kirsch, 2006), and that the effects of misinformation persisted at subsequent non-misleading questioning. In numerous studies, lower DK rates have been observed following misinformation (e.g., Roebers & Fernandez, 2002; Roebers, von der Linden & Howie, 2007). These findings support the importance of understanding factors that influence DK responses and devising methods for encouraging appropriate use of DK responses, without fostering a DK response set. To date DK responses have been conceptualized in inconsistent and contradictory manners in the literature. In some instances, DK responses are thought to reflect subjective uncertainty about the availability of knowledge. This type of DK response is discussed by Koriat and Goldsmith (1996) in the context of a metacognitive model that articulates how individuals distinguish whether or not questions can be answered. They posit that individuals evaluate the quality of information in memory in relation to the question asked before deciding how to
Don’t Know Responding… 5 respond. Responding to a question is a function of three factors: regulation of output (do respondents answer all questions, or if they can choose to answer some and not others), memory quality (indexed by confidence in recalled material), and response cost (anticipated rewards or penalties for answering). Within this framework, DK responses represent the “option to… withhold specific items of information (i.e. to respond ‘don’t know’)” (Koriat, Goldsmith & Pansky, 2000, p. 515) under free report conditions (i.e., when individuals can opt not to respond). In other research, DK responses to questions about unknown information are thought to reflect correct responses (Waterman, Blades, & Spencer, 2001; Roebers & Fernandez, 2002) or resistance to speculation (Poole & White, 1991). This approach hinges on the distinction between answerable and unanswerable questions. Answerable questions query information that is potentially available (that could have been attended to and encoded), whereas unanswerable questions query information that cannot be present in memory, as it was never available. For example, “What color was the robber's car?” is answerable if a car was seen but unanswerable if no car was seen. DK responses to unanswerable questions can be correct statements of ignorance, much like saying “the information was not present.” Some studies distinguish answerable from unanswerable questions and classify DK responses to unanswerable questions as correct (Mulder & Vrij, 1996; Waterman, Blades & Spencer, 2001), while others discuss DK responses to unanswerable questions, without assuming the responses are correct (Poole & White, 1991; Roebers & Fernandez, 2002). Thus, at least two distinct types of DK statements occur. In some instances, DK responses result from recalling that information was not present. Such statements are correct when made to unanswerable questions and incorrect when made to answerable questions. At other times, DK responses indicate that a response cannot be provided, because insufficient
Don’t Know Responding… 6 information is available or because potential responses are inadequate (i.e., are held with low confidence). These are “true DK” responses, whether the question is answerable or unanswerable. Thus, the meaning of a DK response is ambiguous unless the reason for providing it is assessed, and its accuracy is unknown unless the question is objectively known to be answerable or unanswerable. This is important, as it suggests that some accuracy estimates may not be correct. If some DK responses are better understood as correct responses, accuracy rates may increase when coded as such. Precise estimates require understanding the communicative intent of DK responses. The present study is the first to examine the subjective meaning of DK responses and the objective status of information queried, in relation to accuracy and DK rates. As a variety of memory distorting procedures have been shown to influence DK rates, it is also interesting to examine the ways in which such procedures influence DK responses. Studies have shown that both misinformation and hypnosis negatively influence DK rates, as well as response accuracy (e.g. Loftus & Zanni, 1975; Roebers & Fernandez, 2002; Scoboria et al., 2002; Scoboria et al., 2006). Both procedures are of interest in their forensic application and are of interest to memory researchers, as they are thought to operate by different mechanisms. Misleading questioning has been consistently associated with substantial memory distortion (see Loftus, 2003). Such questions introduce new erroneous information that can be easily incorporated into memory, particularly when the questions are about information not originally present. When misinformation contradicts original information, it is thought to operate either by “overwriting” the original information or by co-existing in competition with source information (Ayers & Reder, 1998). Looking at DK responses to answerable and unanswerable questions, with and without misinformation, can potentially inform discussion of the mechanism(s) by which misinformation operates. The “overwriting” hypothesis implies that
Don’t Know Responding… 7 misinformation should lead to lower DK rates relative to no misinformation, with similar DK rates for answerable and unanswerable questions due to the uncontested presence of the misinformation in memory. Conversely, if the original information remains, DK rates should be higher after misinformation for answerable questions, due to competing memory traces. Hypnotic induction and suggestion for memory enhancement has long been thought to enhance recall, though this has been a point of contention as long as hypnotic procedures have been used for this purpose (see Scoboria et al, 2006). Hypnosis for memory refreshment typically takes one of several forms. Following a hypnotic induction, an individual either receives a suggestion for general memory improvement, or is specifically guided (“age regressed”) to mentally relive a past experience (Mazzoni & Lynn, 2007). It is widely held that such hypnotic procedures result in substantial risk for memory distortion. In numerous legal jurisdictions, posthypnotic testimony is subject to a per se exclusion; once hypnotized for the purpose of memory refreshment, any information acquired during or thereafter is inadmissible in court. Despite evidence that hypnosis is unlikely to produce memory distortions more severe than admissible procedures such as misleading questioning (Scoboria et al., 2002), jurisdictions continue to apply this extreme exclusionary rule. For example, the Supreme Court of Canada recently barred the admissibility of post-hypnotic testimony nation wide (R. v. Trochym, 2007). Research has typically found no reliable memory enhancing effect of hypnosis. In a comprehensive review of the topic, Erdelyi (1994) demonstrated that while hypnotic induction can result in enhanced memory reports, this effect is not due to hypnosis. Rather, hypnotized individuals appear to remember more because they increase their output (talk more); once output is controlled, any effect of hypnosis disappears. This increased output consists of both correct and erroneous information, accompanied by inflated confidence in the accuracy of recall (Orne,
Don’t Know Responding… 8 Whitehouse, Dinges & Orne, 1988; Kebbell & Wagstaff, 1998). This is reminiscent of Koriat and Goldsmith’s (1994) model: as confidence increases, response output also increases, potentially explaining lower DK responses during hypnosis. This argument is supported by evidence that hypnosis lowers response bias in memory tasks (Dinges, Whitehouse, Orne, & Powell, 1992; Murrey, Cross, & Whipple, 1992; but see Scoboria et al., 2006). Thus hypnosis appears to impact how responses are evaluated. Lowered response bias, enhanced confidence, and increased output suggests that hypnotized individuals are less careful when searching memory and/or evaluating possible responses. Under such conditions, a general decrease in DK responding would be expected, as careless responding leads hypnotized individuals to endorse some items as correct that non-hypnotized individuals reject. However, unlike misleading questions, hypnosis need not introduce new information. Therefore, DK response rates following hypnosis should be higher for unanswerable than answerable questions. The present study This study was designed to explore the conceptual complexity of DK responses. Toward this goal, we explored the communicative intent of DK responses by querying the meanings of DK responses to answerable and unanswerable questions, and recoding these as correct or erroneous as appropriate. We further examined why DK rates may have varied following misinformation or hypnosis in prior research. Thus our emphasis upon a particular metacognitive control process (DK responses), and not underlying metacognitive monitoring processes (e.g. confidence in responses; for more on the relationship between DK responses and confidence see the discussion below). Finally, we examined the effects of misinformation and hypnosis on recognition memory using signal detection methods. This is important in order to determine whether the manipulations alter memory for the original event (detectability), or if basic memory
Don’t Know Responding… 9 remains intact despite differences responding to questions (bias). The procedures employed by Scoboria et al. (2002, 2006) were adapted to examine the effects of misinformation and hypnosis upon DK responding and accuracy about answerable and unanswerable information, both prior to and following clarification of the subjective meaning of DK responses. The external validity of our original procedures was also enhanced. The prior work used an audio taped narrative, and is thus most applicable to overheard information (“earwitness testimony”; Philippon, Cherryman, Bull & Vrij, 2007). The videotaped scene used in the present study made it possible to examine events of greater perceptual complexity. We predicted that more DK responses would be made to unanswerable questions than to answerable questions. We also predicted that clarifying the meaning of DK responses would result in re-classification of an unknown proportion as correct or error. Specifically, we expected that some DK responses made to unanswerable questions would be reclassified as correct, as such responses likely reflect the correct subjective sense “the information was not there”. However, we also expected that some DK responses to unanswerable questions would not reflect this subjective sense, thus demonstrating the complexity of DK responses. We made further predictions about the influence of recoding DK responses upon accuracy. If some unanswerable questions are recoded as correct, this will result in increased response output consisting of correct responses, and accuracy for unanswerable questions will remain stable or increase. The utility of clarifying DK responses would be best supported if recoding results in differences in the effects of one or more of the other independent variables. Finally, we expected to replicate previous findings for recognition memory. Using similar methods and a signal detection analysis, Scoboria et al. (2006) found that misleading questioning negatively impacted discrimination of information which had been subjected to questioning, but
Don’t Know Responding… 10 not information which had not been questioned during the interview. The use of signal detection analysis in the present study makes it possible to assess whether simply asking a question of any kind impacts memory by affecting discrimination or bias. This remains an empirically important issue, as it suggests that responses change not only as a function of the misleading content of a question, but also of the use of questioning at all. Method Participants Participants were 83 undergraduates (60% female; ages 18 to 32, mean 19.16, SD 1.90) at the University of Connecticut. Participants received course credit for their involvement. Design For analyzing responses to questions, we employed a 2x2x2 mixed design, with misleading vs. no misleading, and hypnosis vs. no hypnosis between subjects, and type of question (unanswerable & answerable) within subjects. For analyzing signal detection statistics, we also used a 2x2x2 mixed design, with misleading questioning and hypnosis between subjects, and questioning status (questioned vs. not-questioned) within subjects. Measures Hypnotic suggestibility. Hypnotic suggestibility was measured using behavioral scores on the Comey and Kirsch (1999) modification of the Carleton University Responsiveness to Suggestibility Scale (CURSS; Spanos, Radtke, Hodgins, Bertrand, & Stam, 1981). Screening occurred during routine sessions held for ongoing studies involving hypnosis. Subjective hypnotic responsiveness scores were normally distributed, thus hypnotic highs, who are underrepresented in the general population, were over-represented in the present study. 74% were screened; the remainder were not screened due to time-constraints. Participants were not
Don’t Know Responding… 11 aware of the present study when screened. The effect of hypnotic responsiveness as a moderator of the relationship of independent variables upon dependent variables (Baron & Kenny, 1986) was tested using hierarchical regression analyses. No significant main effects for hypnosis or interactions were found (all p > .10), and this covariate was dropped from subsequent analyses. Video stimulus. The video used in the present study depicted a robbery and subsequent police chase, and was taken from the work of Zaragoza (i.e., Zaragoza & Mitchell, 1996). Questions. Sets of misleading and non-misleading questions were generated. Two parallel sets of 24 questions were created, labeled misleading and non-misleading. The first set consisted of misleading questions, divided into equal sub-sets: 12 querying information not present in the video (unanswerable questions), and 12 querying information present in the video (answerable questions). The second set consisted of objective non-misleading questions, again with equal numbers of answerable and unanswerable questions, paralleling the content of the first set, but without misleading cues. Each set was augmented with 12 relatively easy questions to make the misleading questions less obvious, bringing the total to 36. Analyses revealed no group differences for these common questions, which were not included in subsequent analyses. Questions were administered to participants at two time points: immediately (Time 1, when participants received either misleading or non-misleading questions, and hypnotic or no hypnotic induction) and upon delay (Time 2, when only non-misleading questions were asked; see Figure 1). This permits examination of the effect of misleading and hypnotic procedures at the time they are presented, and their influence after a delay. Responses were coded as correct, error, or “don’t know” by two raters. Unanswerable questions were coded as correct if an answer such as “was not present” or negation of the question was provided, and as error if a substantive answer was provided. Errors at Time 1 were also coded as yield if the misleading cues were
Don’t Know Responding… 12 accepted (note that yield responses are a sub-set of the total number of error responses). Responses to open ended questions frequently contain mixed information. Following a convention adopted in our previous work, half points were scored when mixed information was provided. For example, consider the question “What was the license plate number?” (answer “555 ASE”). The response, “Part was ASE, but I DK the rest”, was coded as .5 correct, .5 DK; the response “999 ASE”, was coded as .5 correct, .5 error. In each case, forcing a category would be arbitrary. We emphasize that coding rules were applied identically across conditions. Raters coded independently, blind to hypnotic and Time 2 misleading (but not Time 1 misleading) conditions. Correlations between raters at Time 1 and Time 2 were .92 and .94 for correct, .90 and .96 for DK, and .93 and .95 for errors; disagreements were resolved via discussion. To investigate the subjective meaning of DK responses, Time 2 DK responses were queried. Participants were asked if DK responses indicated that information was not in the video, that they could not say/did not remember, or if they intended something else by saying DK. Three response categories emerged: 1) unable to provide an answer (“true DK”); 2) information was not presented; and 3) stating that the information was presented but that the details needed to answer could not be recalled. The latter categories were recoded as correct or error, based upon statement accuracy. For example, a DK response was recoded as correct when a participant reported that DK meant that information was not in the video, and the information was not present; however, if the information was present, this was recoded as an error. Inter-rater correlations for recoded variables were .95 for correct, .95 for errors, and .96 for DK. Signal detection measure. A series of forced choice (true / false) recognition statements were used to assess the influence of procedures upon the contents of memory. Signal detection methods were employed to assess whether misleading information and hypnosis produced
Don’t Know Responding… 13 changes in discrimination or response bias. Four groups of 12 statements were designed, two probing for “hits” (stating information was present when it was) and two probing for “false alarms” (stating information was present when it was not). One of the “hit” and one of the “false alarm” groups of statements probed for recognition of information that was asked about in the interview questions (via non-misleading or misleading questioning); the two remaining groups of statements probed for information which was never queried during the interview. Thus, statements in Set 1 (in video, questioned) referred to information present in the video and about which participants had already been questioned. Statements in Set 2 (not in narrative, questioned) referred to information that was not present and was questioned. Statements in Sets 1 and 2 were utilized to calculate detectability and bias scores for information which were subjected to questioning during the interview. Statements in Set 3 (in video, not questioned) referred to information present in the video but for which no question was asked during the interview. Set 4 (not in video, not questioned) statements referred to information not presented in the video and which was not questioned, and were therefore false. Responses to Sets 3 and 4 were used to calculate detectability and bias scores for information not subjected to questioning during the interview. Questions answered consistently by over 90% of the sample were dropped, as they were too easy/difficult for inclusion and would result in ceiling/floor effects. Of 48 questions, 10 were removed, resulting in 11 questions in Set 1, 9 in Set 2, 9 in Set 3, and 9 in Set 4. Procedures for calculating signal detection statistics are in Scoboria et al. (2006). Procedure The following procedures were originally adapted from the Gudjonsson Suggestibility Scale (GSS; Gudjonsson, 1984) a widely used measure of interrogative suggestibility. This measure consist of presenting an audio taped narrative, free recall, delay, second free recall,
Don’t Know Responding… 14 questioning, negative feedback, and second questioning. The Register and Kihlstrom (1988) adaptation of the GSS1 for American participants was employed in our prior work absent negative feedback between questionings (Scoboria et al. 2002, 2006), and the procedural framework was retained here to facilitate comparison with previous research. The procedures are depicted graphically in Figure 1. Upon arrival, participants randomized to hypnotic conditions were informed that participation involved hypnosis. Participants in non-hypnotic conditions were unaware of the use of hypnosis in the study. Interviewers were undergraduate research assistants, trained in the administration of procedures but blind to the specific hypotheses under study. Participants viewed the video on a 21” monitor from 5 feet. Participants provided written free recall, followed by a 40-minute distractor (the International Personality Item Pool inventory, Goldberg et al., 2006). After a second written free recall, half of the participants were hypnotized and given a suggestion for memory enhancement via an audio taped hypnotic induction; the remaining participants continued the distractor task. Half of the participants were asked the misleading questions, the other half the non-misleading questions (Time 1, immediate questioning). Responses were transcribed verbatim. Participants were neither encouraged nor discouraged from providing DK responses. Hypnotized participants were deinducted. Following a 10-min. delay, all participants were asked the non-misleading questions (Time 2, delayed questioning). Participants were then asked to clarify any Time 2 DK responses (see above). Finally participants completed the recognition task. Results We present our results in four sections. First, we examined DK response rates as a function of question type and interview method. Second, we examined the subjective intent of DK responses to see if different types of DK responses were present, and whether such types of
Don’t Know Responding… 15 varied across groups. Third, we examined correct, error and accuracy rates prior to and following recoding, to determine if they varied as a function of the independent variables. Finally, we examined recognition memory using signal detection analysis. Mean DK responses, correct, errors, and accuracy rates for Time 1 and Time 2 are presented in Table 1. Unless otherwise noted, analyses involved 2x2x2 mixed ANOVAs (question type within subjects; misleading/non-misleading, hypnotic/non-hypnotic between subjects). Post-hoc analyses were conducted using t-tests with Bonferroni corrections. Don’t know responses At immediate questioning, 12.20% of questions received DK responses, and 77.1% of participants made one or more DK response (mean = 2.93, SD = 2.23, range 0 to 9). At delayed questioning, 16.57% of questions received DK responses, and 100% of participants made one or more DK response (mean = 3.97, SD = 2.15, range 1 to 10). Next, we assessed whether DK response rates varied by group. We expected that unanswerable questions would produce more DK responses than answerable questions; that misleading questioning would result in lower DK rates; that DK rates would be lower for unanswerable than answerable misleading questions; and that hypnosis would lower DK rates, but more for answerable than unanswerable questions. At Time 1 questioning, significantly fewer DK responses were provided when receiving misleading information or when responding to unanswerable questions, compared to nonmislead and answerable questions respectively, as shown by main effects of misleading, F (1,79) = 34.20, p < .01, d = 1.03, and question type, F (1,79) = 13.32, p < .01, d = .50. These effects were qualified by a significant question type by misleading interaction, F (1,79) = 26.68, p < .01. Post-hoc analyses showed that participants made more DK responses to non-misleading answerable than to non-misleading unanswerable questions, t (42) = 5.62, p < .01, while DK
Don’t Know Responding… 16 responses for misleading questions did not differ by question type, t (42) = 1.21, ns. At Time 2, the only significant main effect was for question type, F (1,79) = 57.79, p < .01, d = 1.07; fewer DK responses were made to unanswerable questions. A question type by hypnosis effect also emerged, F (1,79) = 5.16, p < .05. Post-hoc analysis revealed this was due to hypnotized participants giving significantly more DK responses than non-hypnotized participants for answerable, t (81) = 2.02, p < .05, but not for unanswerable questions (p > .10). To summarize, the predicted effect of misleading questions upon DK responses was observed, but only at Time 1 and only for answerable questions. The anticipated effects of hypnosis were not observed. At Time 2 (non-misleading questioning) the number of DK responses for answerable questions relative increased for hypnosis as compared to no hypnosis. Contrary to predictions, unanswerable questions produced fewer DK responses at both times. Clarifying subjective meaning of DK responses Recoding DK responses. Our second set of analyses examined the subjective meaning of DK responses. We expected that clarification would result in reclassification of a proportion of DK responses as correct or errors. Specifically, we expected that some (but not all) DK responses made to unanswerable questions would be reclassified as correct. Stating that information was not in the video was recoded as correct for unanswerable questions and as error for answerable questions. Statements that the information was not known were left as DK. All but 5.4% of all DK responses fit into one of these two categories; for these, participants said that the information had been present, but that they could not recall its content. As this unanticipated category was rare, it was dropped from subsequent analyses. In total, 38% of Time 2 DK responses were recoded, 51% of them given to unanswerable and 31% given to answerable questions. Forty-six percent of the “not in video” responses were given to unanswerable
Don’t Know Responding… 17 questions and were recoded as correct, and 55% were to answerable questions and were recoded as errors. Sixty-two percent of DK responses were “true don’t know”, indicating an inability to respond. A greater proportion of DK responses to answerable questions (69%) were recoded than to unanswerable questions (49%). Rates of recoding of DK responses are reported in Table 2. Analysis of recoded variables. We next examined whether this recoding resulted in any changes in “true DK” response rates. A 2x2x2 ANOVA on genuine DK responses provided the same results as before: a main effect of question type, F (1,79) = 68.97, p < .01, d = 1.16 (lower DK rates for unanswerable questions), and no effects of misleading questions or hypnosis. Further ANOVAs were conducted to assess the influence of misinformation and hypnosis on recoded responses. Separate 2x2 analyses (misleading by hypnosis) were conducted for responses recoded as errors (for answerable questions) and as correct (for unanswerable questions). Both variables were positively skewed, and were square root transformed. More DK were recoded as errors for hypnotized than non-hypnotized participants, as shown by a significant main effect of hypnosis, F (1,83) = 5.40, p < .05, d = .52. For DK recoded as corrects, a trend towards an effect of misinformation emerged, F (1,83) = 2.87, p < .10, d = .37, with fewer DK responses recoded as correct for mislead than non-mislead participants. Summary of DK recoding findings. Clarifying DK responses revealed that 38% reflected the subjective sense that the information was not presented. When “not in the video” statements were made to unanswerable questions they were correct, and when made to answerable questions, they were errors (of omission). Recoded DK responses were more likely to reflect errors in hypnotized than non-hypnotized individuals. The remaining DK responses reflected genuine lack of information, which occurred more frequently for answerable questions. Errors, correct responses, and response accuracy
Don’t Know Responding… 18 Next, we examined errors, correct responses, and accuracy prior to and following recoding of DK responses. Recall that we predicted that recoding of unanswerable DK responses would result in a significant increase in correct responses. Although further predictions were not made, we were most interested in instances in which effects varied as a function of recoding. Errors. At Time 1, significantly more errors were made after misleading questions, F (1,79) = 21.99, p < .01, d = .74, and to unanswerable questions , F (1,79) = 27.75, p < .01, d = .82. We also analyzed yield responses (i.e. accepting misleading cues), comparing question type and hypnosis. The 2x2 ANOVA revealed a significant effect of question type, F (1,81) = 6.49, p < .05, d = .19. Yield was greater for unanswerable questions. No effects of hypnosis were observed. At Time 2, significantly more errors were made to misleading, F (1,79) = 6.39, p < .05, d = .42, and unanswerable questions, F (1,79) = 56.63, p < .01, d = 1.15. The hypnosis by misleading interaction was also significant, F (1,79) = 5.90, p < .05, due to a higher error rate in the mislead only group as compared with the other groups, t (81) = 3.26, p < .01. Following DK recoding, participants made more errors to misleading and unanswerable questions, F (1,79) = 4.05, p < .05, d = .35, and F (1,79) = 25.06, p < .01, d = .74. A significant hypnosis by misleading interaction was also obtained, F (1,79) = 8.71, p < .01. Post-hoc analyses revealed that the non-hypnotic non-misleading question group made significantly fewer error responses, t (81) = 3.21, p < .01, than the remaining groups, which did not differ significantly (all p > .10). These results confirm that misleading questions increased the number of errors. They also show that more errors were made when responding to unanswerable than answerable questions, confirming the results of Roebers and Fernandez (2002). Correct responses. At Time 1, the ANOVA revealed a question type by misleading
Don’t Know Responding… 19 interaction for correct responses, F (1,79) = 10.11, p < .01. Mislead participants made more correct responses to answerable than unanswerable questions, t (40) = 3.37, p < .01. No group differences in correct responding were observed at Time 2 before DK recoding, whereas following DK recoding, a significant hypnosis by misleading interaction emerged, F (1,79) = 4.13, p < .05. Analyses revealed that the non-hypnotic non-misleading question group made significantly more correct responses, t (81) = 2.76, p < .01, than the remaining groups, which did not differ (all p > .10). Response accuracy. At Time 1, accuracy rates were lower for mislead participants, F (1,79) = 12.67, p < .05, d = .54, and for unanswerable questions, F (1,79) = 21.44, p < .01, d = .74. No interactions were observed, and no effect was found for hypnosis. At Time 2 (nonmisleading questioning), accuracy was lower for misleading, F (1,79) = 5.05, p < .05, d = .36, and unanswerable questions, F (1,79) = 38.85, p < .01, d = .94, when compared to nonmisleading and answerable questions, respectively. The misleading main effect was qualified by a hypnosis by misleading interaction, F (1,79) = 5.03, p < .05. Lower accuracy was found in the mislead only group, as compared with the remaining groups, t (81) = 2.85, p < .01. Accuracy after DK recoding. Analysis of recoded DK responses revealed an important difference in findings: the emergence of a negative effect of hypnosis. A significant hypnosis by misleading interaction, F (1,79) = 7.84, p < .01, revealed higher accuracy in the non-hypnotic non-misleading question group than in the remaining groups, t (81) = 3.22, p < .01, which did not differ (all p > .10). The difference in accuracy before and after DK recoding is represented by the fact that before recoding accuracy was significantly lower (and errors higher) in the misleading only group compared to the other groups, whereas after recoding accuracy was lower (errors higher)
Don’t Know Responding… 20 in all groups compared to the non-hypnotic non-mislead group. To explore this difference, a 2x2x2 mixed ANOVA (within subjects: question type; between subjects: misleading, hypnosis) was conducted on accuracy change scores (Time 1 less Time 2 scores). Noting statistical concerns with analysis of change scores (Keppel & Zedeck, 1989), we also conducted analyses adding a within subjects time factor. As results were invariant, we present the change score analysis. Main effects for question type, F (1,83) = 54.12, p < .01, and hypnosis, F (1,83) = 5.21, p < .05, indicated that change in accuracy was significantly larger (showing a greater decrease in accuracy after recoding) for answerable questions and hypnotized participants. The hypnosis main effect was qualified by a significant hypnosis by question type interaction, F (1,83) = 5.41, p < .05. Post-hoc analyses revealed that accuracy decreased significantly only for answerable questions (one-sample t-test, comparison with 0, t = 8.28, p < .001), and that this decrease was greater for hypnotized than non hypnotized participants (t (81) = 2.08, p < .01). To summarize the accuracy findings, misinformation consistently negatively impacted memory. More interesting, responses to unanswerable questions consistently demonstrated lower accuracy than those to answerable questions, suggesting that the simple act of querying nonpresented information is problematic. After DK recoding, accuracy was stable for unanswerable questions; and decreased for answerable questions, more so in hypnosis groups. Signal Detection analysis The signal detection analysis was planned to assess whether misleading information, hypnosis, or questioning at all affected detectability or response criterion. Our predictions were that misinformation would negatively impact memory for questioned items, but items about which questions had not been asked would not show such a negative influence.
Don’t Know Responding… 21 Recognition results were analyzed using 2x2x2 mixed ANOVAs (questioned / not questioned within subjects; hypnosis, misleading between subjects). Questioned refers to recognition items which were queried during the interview, and not questioned refers to recognition items not so queried. Note that both questioned and not questioned items could refer or not refer to the original information. This is not to be confused with the answerable / unanswerable distinction made in the preceding analyses. Although question type (answerable vs. unanswerable) cannot be considered separately in this analysis, an effect of questioning would help understand the effect of unanswerable questions, as it would show that the simple act of being asked a question affects recognition memory. Group means can be found in Table 3. Analysis of detection scores revealed a questioning main effect, F (1,79) = 49.41, p < .01, d = .94; and no other significant main effects or interactions. Participants discriminated information that had not been questioned (mean = 1.24, SD = .60) better than information that had been questioned (mean = .73, SD = .49). A questioning effect for bias was also found, F (1,79) = 11.17, p < .01, d = .47. Participants demonstrated no bias for questioned (mean B = .02, SD = .54) and a liberal bias for unquestioned information (mean B = -.23, SD = .63). Overall, the signal detection analyses indicated that questioning (either misleading or non-misleading) impacted participants’ ability to discriminate information and affected their response bias. The hypothesis regarding the negative impact of misinformation was not upheld. Free recall Finally, we examined the relationship between free recall and subsequent DK responses. We emphasize that participants had only 5 minutes for each recall and thus did not have time to fully express the contents of memory. Recalls were coded for total idea units, distortions, and confabulations; no group differences were observed at either time (all p > .10). We examined the
Don’t Know Responding… 22 correspondence between the content of free recall and DK responses. The vast majority of DK responses were made about content that was not reported during free recall (Time 1, 93.2%; Time 2, 92.3%). DK responses about material reported during free recall were infrequent (16.5 at Time 1, 25.5 at Time 2); statistical analyses failed to reveal any significant group differences. Discussion This study demonstrates that the communicative intent of “don’t know” responses to interview questions varies. About one-third of DK responses were statements that information was not presented. Recoding these resulted in increased output, stable accuracy for unanswerable questions, and decreased accuracy for answerable questions. Thus accuracy and output rates in research examining responses to questions when DK responses are permitted are likely misestimated. To fully understand DK responses, both the subjective communicative intent and the objective status of the information as potentially available or unavailable are required. These findings show an important aspect of memory monitoring. Adults can discriminate between at least three different DK states, one reflecting uncertainty (“true DK”), another indicating that information cannot be retrieved because it was never presented, and a third indicating awareness that the queried information was present, but sufficient detail cannot be recalled. The results also suggest that people often correctly distinguish when information had or had not been presented, even when presented information is not fully retrievable. By using answerable and unanswerable questions, the present study provides some insight into previously observed changes in DK responding following misleading questioning and hypnosis. The results indicated consistent lower DK rates and accuracy for misleading questioning and unanswerable questions. Although the misleading cues in unanswerable questions were associated with lower accuracy as compared with answerable questions, this did
Don’t Know Responding… 23 not translate into the anticipated interactions. This indicates that the impact of misinformation is substantial across question types. In both cases DK responses are consistently lower following misinformation, as the acceptance of misinformation increases. The lack of interaction appears to support the overwriting hypothesis, as lower DK rates for answerable questions would be expected if original and misleading information co-exist in competition in memory. However, the signal detection analyses did not show an associated impact of misinformation upon recognition memory, and other mechanisms seem more likely. The misinformation might have had a higher level of activation relative to original information during questioning (recall), but the original material remained in memory at a level sufficient for more basic recognition when specifically queried at a later time. We note that the recognition findings are inconsistent with our prior work (Scoboria et al., 2006), where an effect of misinformation upon recognition was observed. Further research is needed to examine this inconsistency, which may have been due to differences in stimuli (audio vs. video). The finding that questions about unanswerable information were associated with lower accuracy and DK response rates, independently of misinformation, is interesting. This finding is counterintuitive; it is expected that at least non-mislead individuals would say DK more frequently in regards to unknown information. One explanation is that participants were prone to erroneously state that the information queried had been presented, whether or not the question was misleading. This suggests that a risk lies in the very act of asking about information that had never been presented, and that the risk of embellishment for unknown details is robust. The results of the signal detection analyses, which showed a strong influence of questioning at all upon discrimination, support this assertion, and show that the effect of questioning operates to alter the contents of memory. Questioning thus encourages people to
Don’t Know Responding… 24 process material differently than they would have if not questioned, and reinforces the idea that inviting people to consider non-presented information promotes confabulation and acceptance of confabulated information (Ackil & Zaragoza, 1998). Recoding DK responses also revealed a negative effect of hypnosis on accuracy which was absent before recoding. We note that there was no observable benefit of hypnosis, consistent with prior research. After recoding hypnosis clearly negatively affected accuracy, in particular for responses to answerable questions. In fact, as DK responses given in hypnosis were mostly errors, a significant decrease in accuracy resulted. This is consistent with the idea that hypnosis leads individuals to engage in a less careful memory search, as the hypnotic suggestion indicates that information should come effortlessly to mind. Interestingly, rather than leading to uncertainty (“true DK” responses), participants in hypnosis were more likely to state that information was not presented, and no effects of hypnosis were observed during the recognition task. Taken together, these results suggest that the hypnotic procedure may have impacted metacognitive processes. They might have adopted a misleading memorability strategy (Ghetti, 2004), by which they might have thought: “Since I was hypnotized, I would remember if the info had been present. Since I cannot remember, the information must not have been presented”. However, prior findings that hypnosis increases the amount of information reported, and decreases DK responses and response bias, were not replicated. Clearly further work is needed to understand how hypnosis impacts metacognition. Future directions and applications It will be important to extend these findings to examine the relationships between DK responses, confidence in responses, and response accuracy. Confidence is an important marker for understanding the quality of response output (Koriat & Goldsmith, 1996). DK responses have
Don’t Know Responding… 25 been described as one type of metacognitive control process which reflects the outcome of monitoring processes such as confidence and judgments of learning (Nelson & Narens, 1994). It is thus important understand the metacognitive processes that underlie DK responding. The relationship between confidence and accuracy is complex (see Shaw, McClure, & Dykstra, 2007), however confidence ratings can provide useful information about accuracy (Busey, Tunnicliff, Loftus & Loftus, 2000; Brewer & Wells, 2006). One approach to the confidence-accuracy relationship considers whether confidence ratings usefully discriminate between accurate and inaccurate occurrences. Under many circumstances, confidence for correct responses is in fact greater on average than for incorrect responses (Juslin, Olsson & Winman, 1996; Roebers, 2002; Roebers & Howie, 2003). Furthermore, von der Linden & Roebers (2007) have shown that children make higher confidence ratings for responses to answerable than to unanswerable questions. Another approach to the confidence / accuracy relationship looks at the consistency between confidence ratings and accuracy of responses for the same question (Howie & Roebers, 2007). It is well established that confidence tends to be inflated relative to performance (Bornstein & Zickafoose, 1999; Robinson, Johnson, & Robertson, 2000), a problem termed ‘miscalibration’. Recent work with children upon calibration of responses which examines the correspondence between confidence and accuracy reports that from at least the age of 8 onwards children effectively report uncertainty, under the correct conditions (ie., when questions are unbiased, and are not overly difficult; Roebers, von der Linden, & Howie, 2007) It is clear that DK responses must be taken into consideration when examining the confidence / accuracy relationship. To illustrate, Roebers, von der Linden and Howie (2007) recently hypothesized that children avoid using the lower end of confidence scales because low
Don’t Know Responding… 26 confidence responses are withheld in favor of DK responses. When they contrasted children who were able to provide DK responses with children who were forced to answer all questions, they found that confidence was lower in the forced response group. Thus, children’s tendency to avoid the lower end of confidence scales appears to be related to the use of DK responses: allowing DK responses inflates confidence estimates because uncertain responses are withheld. The present work suggests a novel approach to studying this trade-off between DK responding and confidence. Prior research has not collected confidence ratings for DK responses, yet this study suggests that DK responses are held with different levels of conviction. Future research can examine confidence ratings for DK responses (“How confident are you that you do not know?”). Different types of DK statements will presumably have different average levels of confidence. For example, confidence for “not present” responses should be greater on average than “uncertain” responses. This will help to clarify when DK responses do represent the lower end of the confidence spectrum. Of particular importance is reconciling the robust risk for providing erroneous responses to unanswerable questions with prior findings that confidence is lower for unanswerable than answerable questions (von der Linden & Roebers, 2007). Second, the variable meanings of DK responses raise the question of what interviewers should do when DK responses are given. If such responses are accepted outright, information that is available but not volunteered will not be obtained. If DK responses are challenged, speculation is encouraged. While discouraging DK responses would likely lead to increased response output and decreased accuracy, encouraging them would have the opposite effect. It is apparent, then, that the handling of DK responses has implications for both the amount and quality of information gathered and querying DK responses as to their communicative intent might result in improvements in the information obtained. This
Don’t Know Responding… 27 investigation provides a glimpse of how this might be done (e.g. distinguishing between “true DK responses”, and statements that the item was never witnessed), but does not inform how best to conduct an interview to do so. Further research is needed to understand how best to establish the DK set during an interview, and how to understand them once made. As a third issue, the finding that any questioning lowers memory discrimination is worrisome, and, in conjunction with the finding regarding the risk for error in response to unanswerable questions, may prove useful in designing techniques which minimize suggestion via questioning. It reinforces a well-established best practice in interviewing: relying upon free recall to the greatest extent possible. Typically, an interviewer’s job is to ascertain what occurred during a situation for which no “objective” account exists. As the original information is unknown, it is impossible to judge what questions are objectively answerable and unanswerable. The best approach may be to use questioning techniques which build upon answers which individuals provide during free recall (see Wells, Memon, & Penrod, 2006). By linking questioning to information provided during free recall, questions are restricted to domains of information which are endorsed as having been observed, and the risk of introducing new information via questioning may then be reduced. At the very least, any intruding information is then produced by the witness rather than via suggestion inherent to question type and content. We note that the two free recalls prior to questioning may have allowed for rehearsal of material prior to questioning. Different results might have been obtained without previous recall sessions. Thus, it is best to keep in mind that these results reflect responding after several prior rehearsals. This also applies to the signal detection analyses, as recognition was measured after free recall and questioning. These results are thus most relevant to questioning situations in which questioning follows several free recalls, analogous to interviewing situations, where
Don’t Know Responding… 28 several statements are provided prior to questioning. However, future work which examines the subjective intent of DK responses without such preceding procedures therefore is warranted. In summary, the present study emphasizes the importance of examining responding to interview questions in manners other than accuracy rates. Exploring naturally occurring DK responses is a promising approach for understanding how memory distorting techniques impact different types of information. These results emphasize the need to understanding people’s intention when they make DK responses to interview questions, as they reveal that not all DK response imply the same type of judgment about the contents of memory.
Don’t Know Responding… 29 References Ackerman, B.P. (1993). Children's understanding of the relation between referential knowledge and referential behavior. Journal of Experimental Child Psychology, 56(3), 385-411. Ackil, J.K., & Zaragoza, M.S. (1998). Memorial consequences of forced confabulation: Age differences in susceptibility to false memories. Developmental Psychology, 34, 1358-1372. Ayers, M.S. & Reder, L.M. (1998). A theoretical review of the misinformation effect: Predictions from an activation-based memory model. Psychonomic Bulletin and Review, 5, 1-21. Baron, R.M., & Kenny, D.A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51. 1173-1182. Beuscher, E., & Roebers, C.M. (2005). Does a warning help children to more accurately remember an event, to resist misleading questions, and to identify unanswerable questions? Experimental Psychology, 52(3), 232-241. Bornstein, B.H., & Zickafoose, D. J. (1999). "I know I know it, I know I saw it": The stability of the confidence-accuracy relationship across domains. Journal of Experimental Psychology: Applied, 5(1), 76-88. Brewer, N. & Wells, G. L. (2006). The confidence-accuracy relationship in eyewitness identification: Effects of lineup instructions, foil similarity and target-absent base rates. Journal of Experimental Psychology: Applied, 12, 11-30. Busey, T. A., Tunnicliff, J., Loftus, G. R. & Loftus, E. F. (2000). Accounts for the confidenceaccuracy relation in recognition memory. Psychonomic Bulletin & Review, 7, 26-48. Ceci, S.J., & Bruck, M. (1993). Suggestibility of the child witness: A historical review and synthesis. Psychological Bulletin. 113(3), 403-439.
Don’t Know Responding… 30 Comey, G., & Kirsch, I. (1999). Intentional and spontaneous imagery in hypnosis: The phenomenology of hypnotic responding. International Journal of Clinical and Experimental Hypnosis, 47, 65-85. Dinges, D. F., Whitehouse, W. G., Orne, E. C., & Powell, J. W. (1992). Evaluating hypnotic memory enhancement (hypermnesia and reminiscence) using multitrial forced recall. Journal of Experimental Psychology: Learning, Memory, & Cognition, 18, 1139–1147. Erdelyi, M. H. (1994). Hypnotic hypermnesia: The empty set of hypermnesia. International Journal of Clinical and Experimental Hypnosis, 42, 379–390. Ghetti, S. (2004). Memory for nonoccurrences: The role of metacognition. Journal of Memory and Language, 48, 722-739. Goldberg, L.R., Johnson, J.A., Eber, H.W., Hogan, R., Ashton, M.C., Cloninger, C.R., & Harrison, G.G. (2006). The international personality item pool and the future of publicdomain personality measures. Journal of Research in Personality 40, 84–96. Gudjonsson, G. (1984). A new scale of interrogative suggestibility. Personality and Individual Differences, 7, 195–199. Gudjonsson, G.H. (2003). The psychology of interrogations and confessions. Chichester, UK: Wiley. Gudjonsson, G.H., & Hilton, M. (1989). The effects of instructional manipulation on interrogative suggestibility. Social Behaviour, 4(3), 189-193. Howie, P.M., & Dowd, H.J. (1996). Self-esteem and the perceived obligation to respond: Effects on children’s testimony. Legal and Criminological Psychology, 1, 197-209. Howie, P. & Roebers, C.M. (2007). Developmental progression in the confidence-accuracy relationship in event recall: Insights provided by a calibration perspective. Applied Cognitive Psychology, 21, 871-894.
Don’t Know Responding… 31 Juslin, P., Olson, N., & Winman, A. (1996). Calibration and diagnosticity of confidence in eyewitness identification: Comments on what can and cannot be inferred from the low confidence-accuracy correlation. Journal of Experimental Psychology: Learning, Memory and Cognition, 5, 1304-1316. Kebbell, M.R. & Wagstaff, G.F. (1998). Hypnotic interviewing: The best way to interview eyewitnesses? Behavioral Sciences and the Law, 16, 115-129. Keppel, G., & Zedeck, Z. (1989). Data analysis for research designs. W.H. Freeman: New York. Köhnken, G., Milne, R., Memon, A., & Bull, R. (1999). The cognitive interview: A metaanalysis. Psychology, Crime & Law, 5(1-2), 3-27. Koriat A., & Goldsmith, M. (1996). Monitoring and control processes in the strategic regulation of memory accuracy. Psychological Review, 103, 490-517. Koriat, A., Goldsmith, M., & Pansky, A. (2000). Toward a psychology of memory accuracy. Annual Review of Psychology, 51, 481-537. Koriat, A., Goldsmith, M., Schneider, W., & Nakash-Dura, M. (2001). The credibility of children’s testimony: Can children control the accuracy of their memory reports? Journal of Experimental Child Psychology, 79, 405-437. Loftus, E. F. (2003). Make-believe memories. American Psychologist, 58, 867–873. Loftus, E.F., & Zanni, G. (1975). Eyewitness testimony: The influence of the wording of a question. Bulletin of the Psychonomic Society, 5(1), 86-88. Mazzoni, G., & Lynn, S.J. (2007). Using hypnosis in eyewitness memory: Past and current issues. In M.P Toglia, J.D. Read, D.F. Ross, R.C.L. Lindsay (Eds), The handbook of eyewitness psychology, Vol I: Memory for events, (pp. 321-338). Mahwah, NJ: Lawrence Erlbaum.
Don’t Know Responding… 32 Moston, S. (1987). The suggestibility of children in interview studies. First Language, 7, 67-78. Mulder, M.R., & Vrij, A. (1996). Explaining conversation rules to children: An intervention study to facilitate children's accurate responses. Child Abuse & Neglect, 20, 623-631. Murrey, G.J., Cross, H.J., Whipple, J. (1992). Hypnotically created pseudomemories: Further investigation into the "memory distortion or response bias" question. Journal of Abnormal Psychology. 101, 75-77. Nelson, T.O., & Narens, L. (1994). Why investigate metacognition? In J. Metcalfe, & A.P. Shimamura (Eds.), Metacognition: Knowing about knowing (pp. 1-25). Cambridge, MA: MIT Press. Nesbitt, M., & Markham, R. (1999). Improving young children's accuracy of recall for an eyewitness event. Journal of Applied Developmental Psychology, 20(3), 449-459 Orne, E.C., Whitehouse, W.G., Dinges, D.F., & Orne, M.T. (1996). Memory liabilities associated with hypnosis: Does low hypnotizability confer immunity? International Journal of Clinical and Experimental Hypnosis, 44, 354-369. Phillipon, A.C., Cherryman, J., Bull, R., & Vrij, A. Earwitness identification performance: The effect of language, target, deliberate strategies, and indirect measures. Applied Cognitive Psychology, 21(4), 417-556. Poole, D.A., & White, L.T. (1991). Effects of Question Repetition on the Eyewitness Testimony of Children and Adults. Developmental Psychology, 27, 975-986. R. v. Trochym, 2007 SCC 6, [2007] 1 S.C.R. 239. Register, P. A., & Kihlstrom, J. F. (1988). Hypnosis and interrogative suggestibility. Personality and Individual Differences, 9, 549–558. Robinson, M.D., Johnson, J.T., Robertson, D.A. (2000). Process versus content in eyewitness
Don’t Know Responding… 33 metamemory monitoring. Journal of Experimental Psychology: Applied, 6, 207-221. Roebers, C. M. (2002). Confidence judgments in children’s and adults‘ event recall and suggestibility. Developmental Psychology, 38, 1052-1067. Roebers, C.M., & Fernandez, O. (2002). The effects of accuracy motivation on children’s and adults’ event recall, suggestibility, and their answers to unanswerable questions. Journal of Cognition and Development, 3(4), 415-443. Roebers, C.M., Moga, N., & Schneider, W. (2001). The role of accuracy motivation on children's and adults' event recall. Journal of Experimental Child Psychology, 78(4), 313-329. Roebers, C.M., & Schneider, W. (2002). Stability and consistency of children's event recall. Cognitive Development, 17, 1085-1103. Roebers, C.M., von der Linden, N., & Howie, P. (2007). Favourable and unfavourable conditions for children’s confidence judgments. British Journal of Developmental Psychology, 25, 109134. Roebers, C.M., von der Linden, N., Schneider, W., & Howie, P. (2007). Children’s metamemorial judgments in an event recall task. Journal of Experimental Child Psychology, 97, 117-137. Saywitz, K.J., & Moan-Hardie, S. (1994). Reducing the potential for distortion of childhood memories. Consciousness and Cognition: An International Journal, 3(3-4), 408-425. Scoboria, A., Mazzoni, G., Kirsch, I., & Milling, L.S. (2002). Immediate and persisting effects of misleading questions and hypnosis on memory reports. Journal of Experimental Psychology: Applied, 8(1), 26-32. Scoboria, A., Mazzoni, G., & Kirsch, I. (2006). Effects of misleading questions and hypnotic memory refreshment on memory reports: A signal detection analysis. International Journal
Don’t Know Responding… 34 of Clinical and Experimental Hypnosis, 54(3), 340-359. Shaw, J.S., McClure, K.A., & Dykstra, J.A. (2007). Eyewitness confidence from the witnessed event through trial. In Toglia, M.P., Read, J.D., Ross, D.F., Lindsay, R.C.L. (Eds.), The handbook of eyewitness psychology, Vol I: Memory for events (pp. 371-397). Lawrence Erlbaum: Mahwah, NJ. Spanos, N.P., Radtke, H.L., Hodgins, D.C., Bertrand, L.D., Stam, H.J., & Dubreuil, D.L. (1983). The Carleton University Responsiveness to Suggestion Scale: Stability, reliability and relationships with expectancy and hypnotic experiences. Psychological Reports, 53, 555-563. von der Linden, N., & Roebers, C.M. (2006). Developmental changes in uncertainty monitoring during an event recall task. Metacognition Learning, 1, 213-228. Waterman, A.H., Blades, M., & Spencer, C. (2001). Interviewing children and adults: The effect of question format on the tendency to speculate. Applied Cognitive Psychology, 15, 521-531. Wells, G. L., Memon, A., & Penrod, S.D. (2006). Eyewitness evidence: Improving its probative value. Psychological Science in the Public Interest, 7(2), 45-75. Wright, D., & Davies, G.M. (2007). Eyewitness testimony. In F. Durso (Ed.), Handbook of Applied Cognition, 2nd ed. (763 – 788). John Wiley & Sons: Chichester, UK. Zaragoza, M.S., & Mitchell, K.J. (year) Repeated exposure to suggestion and the creation of false memories. Psychological Science, 7(5), 294-300.
Don’t Know Responding… 35 Author note Correspondence regarding this article should be directed to the first author at: Department of Psychology, 401 Sunset, Windsor, ON, N9B 3P4, Canada;
[email protected]. This work was partially supported by a dissertation award to the first author from the Council of Graduate Departments of Psychology / American Psychological Foundation and a Discovery grant from the Natural Science and Engineering Research Council of Canada. Our thanks to Melissa Burgos, Jarita Dunbar and Eileen Horsman for assistance with data collection.
Don’t Know Responding… 36
Table 1 Mean Correct, DK, Error, and Accuracy rates by Treatment Group and Time Treatment group Unanswerable questions Non-Hyp Non-ML
Non-Hyp ML
Hyp Non-ML
Hyp ML
Non-Hyp Non-ML
Non-Hyp ML
Hyp Non-ML
Hyp ML
21
21
21
20
21
21
21
20
Mean
8.07
7.41
7.60
7.53
7.60
8.59
7.40
8.50
SD
1.43
1.37
1.47
1.35
1.41
1.60
1.31
1.42
Mean
1.64
1.05
1.05
0.89
2.57
0.68
3.00
0.76
SD
1.41
1.46
1.07
1.05
1.09
1.01
1.40
1.01
Mean
2.29
3.55
3.36
3.58
1.83
2.73
1.60
2.74
SD
1.16
1.26
0.92
1.31
0.98
1.45
1.27
1.10
Mean
0.78
0.68
0.69
0.68
0.80
0.76
0.83
0.75
SD
0.10
0.10
0.09
0.11
0.10
0.12
0.12
0.11
Mean
7.88
7.02
7.48
7.79
8.05
7.59
7.62
7.37
SD
1.75
1.86
1.39
1.51
1.56
1.24
1.64
1.67
Mean
1.69
1.07
1.19
1.11
2.55
2.27
3.05
3.03
SD
1.62
1.53
0.89
1.33
1.18
1.36
1.45
1.53
Mean
2.43
3.91
3.33
3.11
1.40
2.14
1.33
1.61
SD
1.44
2.00
1.16
1.45
1.16
1.17
1.14
1.37
Mean
0.77
0.65
0.69
0.72
0.85
0.79
0.85
0.82
SD
0.14
0.16
0.11
0.12
0.12
0.11
0.12
0.15
Mean
8.64
7.25
7.76
8.11
8.19
7.59
7.62
7.37
SD
1.41
2.03
1.44
1.45
1.74
1.24
1.64
1.67
Mean
0.79
0.57
0.62
0.63
1.93
1.73
1.95
2.24
SD
0.98
1.16
0.72
1.07
1.33
1.25
1.24
1.30
Mean
2.57
4.18
3.62
3.26
1.88
2.68
2.43
2.39
SD
1.43
1.94
1.28
1.37
1.34
1.06
1.46
1.54
Mean
0.77
0.63
0.68
0.71
0.81
0.74
0.76
0.76
N Time 1
Correct
DK
Error
Accuracy
Time 2
Correct
DK
Error
Accuracy
Time 2
Correct
Modifieda DK
Error
Accuracy
Answerable questions
Don’t Know Responding… 37 SD
0.12
0.16
Note. Hyp – hypnosis; ML – misleading questions. a
Time 2 responses adjusted based upon recoding of DK responses.
0.11
0.11
0.13
0.10
0.15
0.16
Don’t Know Responding… 38 Table 2 Average Time 2 DK Responses Coded as “True DK”, Correct, and Error, by Question Type and Treatment Group Treatment group Unanswerable Subjective status of DK response True DK
Answerable
Non-Hyp Non-ML
Non-Hyp ML
Hyp Non-ML
M
0.71
0.41
0.57
SD
1.01
1.14
0.75
“Not present”, coded as:
Hyp ML
Non-Hyp Non-ML
Non-Hyp ML
Hyp Non-ML
0.68
1.86
1.75
1.83
1.92
0.82
1.43
1.23
1.08
1.25
Correct (for unanswerable)
Error (for answerable)
M
0.81
0.59
0.57
0.42
0.48
0.55
1.14
0.74
SD
0.93
0.85
0.6
0.61
0.68
0.51
0.96
0.73
“Present, not remembered”, coded as:
Correct / DKa (for answerable)
Error (for unanswerable)
M
0.05
0.05
0
0
0.19
0
0.05
0.21
SD
0.22
0.21
0
0
0.4
0
0.22
0.54
Note. Hyp – hypnosis; ML – misleading questions. a
Hyp ML
Note that the subjective statement of “Present, not remembered” to an answerable question is an expression of inability to
answer the question, which is best conceptualized as partially correct and partially DK (stating it was present is incorrect, however at finer grain a substantive response is not provided).
Don’t Know Responding… 39 Table 3 Recognition Signal Detection Statistics by treatment group
Treatment group
d'
Questioned
Not questioned
B
Questioned
Not questioned
Non-Hyp Non-ML
Non-Hyp ML
Mean
0.75
0.64
0.80
0.70
SD
0.54
0.48
0.52
0.38
Mean
1.18
1.22
1.21
1.39
SD
0.48
0.67
0.50
0.56
Mean
0.08
0.21
-0.21
0.05
SD
0.56
0.56
0.47
0.53
-0.12
-0.02
-0.23
-0.13
0.60
0.62
0.46
0.60
Mean SD
Note. Hyp – hypnosis; ML – misleading questions.
Hyp Non-ML
Hyp ML
Don’t Know Responding… 40 Figure 1. Graphical Depiction of Procedures.
Don’t Know Responding… 41
1) Video 2) Free recall 3) Distractor 4) Delayed recall
5a) Hypnotic induction
6a) Time 1 misleading questions
7) Deinduction and delay
6b) Time 1 non-misleading questions 5b) No hypnosis
6a) Time 1 misleading questions 6b) Time 1 non-misleading questions
8) Time 2 questions (same questions as 6b)
9) Time 2 DK responses queried 7) Delay 10) Recognition measure
