The On-line Study of Sentence Comprehension - Semantic Scholar

J Psycholinguist Res (2006) 35:215–231 DOI 10.1007/s10936-006-9012-0 ORIGINAL ARTICLE

The On-line Study of Sentence Comprehension: An Examination of Dual Task Paradigms Janet Nicol · David Swinney · Tracy Love · Lea Hald

Published online: 16 May 2006 © Springer Science+Business Media, Inc. 2006

Abstract This paper presents three studies which examine the susceptibility of sentence comprehension to intrusion by extra-sentential probe words in two on-line dual-task techniques commonly used to study sentence processing: the cross-modal lexical priming paradigm and the unimodal all-visual lexical priming paradigm. It provides both a general review and a direct empirical examination of the effects of task-demand in the on-line study of sentence comprehension. In all three studies, sentential materials were presented to participants together with a target probe word which constituted either a better or a worse continuation of the sentence at a point at which it was presented. Materials were identical for all three studies. The manner of presentation of the sentence materials was, however, manipulated; presentation was either visual, auditory (normal rate) or auditory (slow rate). The results demonstrate that a technique in which a visual target probe interrupts ongoing sentence processing (such as occurs in unimodal visual presentation and in very slow auditory sentence presentation) encourages the integration of the probe word into the on-going sentence. Thus, when using such ‘sentence interrupting’ techniques, additional care to equate probes is necessary. Importantly, however, the results provide strong evidence that the standard use of fluent cross-modality sentence investigation methods are immune from such external probe word intrusions into ongoing sentence processing and are thus accurately reflect underlying comprehension processes. Keywords Cross-modal lexical priming · Dual task · Sentence comprehension

J. Nicol (B) Department of Psychology, University of Arizona, Tucson, AZ, USA e-mail: [email protected] D. Swinney · T. Love · L. Hald University of California, San Diego, CA, USA T. Love San Diego State University, San Diego, CA, USA

216

J Psycholinguist Res (2006) 35:215–231

Introduction The study of sentence processing is, of necessity, inferential, because there is no direct methodological access to human mental processes. Given this fact, language researchers have evolved methodological procedures that attempt to maximally reflect underlying sentence processing details while minimizing effects introduced by the task (task demand effects such as problem solving, specialized strategies, metalinguistic reflection, etc.) When the goal of studying sentence processing is to characterize the moment-by-moment operations involved in computing the structure and meaning of a sentence as it unfolds, the optimal technique is one that is termed on-line. This then refers to a well-defined match of task time and task sensitivity to some aspect of ongoing comprehension (see e.g. Swinney, Prather, & Love, 2000). One on-line task that has appeared to be relatively immune from task-based effects is crossmodal lexical priming (CMLP), which uses semantic/associative priming obtained during ongoing sentence comprehension to illuminate sentence processing in real time (Swinney, Onifer, Prather, & Hirshkowitz, 1979). This task has proven to be particularly sensitive to moment-by-moment sentence processing. In what follows we describe this task in some detail and then raise a potentially significant problem in the use of this (and variants of this) task. We then present three empirical studies which directly examine this potential problem, and which compare CMLP to other relevant dual task methodologies. CMLP comes in different varieties, but all involve the following conditions and properties: First, the sentential material under study is presented auditorily to participants, who are instructed that their major job is to understand the sentence or discourse they hear.1 Second, participants are told that they have an additional task to perform: at some point while they are listening to each sentence, a visual target item will appear on a screen in front of them and they will have to make a decision about that visual item. This item may be a letter string (to which they may be required to make a lexical decision, or a ‘naming’ response), or a picture (to which they make a binary classification response—e.g., ‘animal vs. non-animal’ or ‘living vs. non-living’). Additional important characteristics of appropriate use of CMLP include: (1) Presentation of the auditory sentence always continues throughout and beyond presentation of the visual item (to the end of the sentence or paragraph). That is, the sentence is never ended with presentation of the visual probe, a factor which has been assumed to prevent the probe from being integrated into the ongoing sentential material or from reflecting end-of-sentence wrap-up effects; (2) Participants never make metalinguistic judgments about the sentential material they hear. Such judgments engage conscious (and hence largely non-automatic) processes; and (3) Processing of the sentence is uninterrupted up to the point of the visual target probe presentation. Thus participants are not consciously evaluating each successive word or phrase as they hear it (as required in other on-line techniques such as monitoring or word-by-word reading, etc.). In CMLP there is a planned relation between the tasks the participant performs (auditory sentence comprehension and visual target decision). On experimental trials, the visual target is associatively/semantically related to a critical word (or phrase) in the sentence. The occurrence of this critical auditory word (the prime) just prior to presentation of a related visual target probe results in speeded processing of the target probe (compared to an unrelated control probe), a result that is termed priming (see, e.g., Meyer, Schvaneveldt, & Ruddy, 1974; Neely, 1991). The CMLP task exploits the fact that this priming occurs between associatively 1 Participants are standardly tested for comprehension throughout the entire experiment to ensure attention

to the materials.


217

related materials across modalities (including words and pictures; Kroll & Potter, 1984; McKee, Nicol, & McDaniel, 1993; Swinney & Prather, 1989). The flexibility afforded by being able to place the visual target probe anywhere during the course of a sentence (and to test for priming at every such point) allows the CMLP technique to sensitively map out the moment-by-moment activations of word meanings in ongoing sentence processing. To further elucidate characteristics of the CMLP task, consider an example in which participants are presented auditorily with the sentence: The policeman saw1 the boy 2 who the crowd at the party 3 accused 4 of the crime This is an object-relative construction, in which the direct object (‘boy’) is not in its canonical post-verb position (following ‘accused’), but has been moved to a fronted position. One issue concerning the understanding of such sentences has been whether the fronted direct object must be reactivated in its S_V_O canonical post-verb position during comprehension for correct interpretation. The CMLP technique is ideally suited to examination of this issue in that it allows examination of the time course of activation of this fronted direct object throughout the course of the sentence. Thus, visual target probes presented at each of the numbered positions in the example can reveal activation (or lack thereof) at each such point during comprehension. Such activation (priming) is determined by comparing RT to a target probe that is semantically/associatively related to the fronted direct-object ‘boy’ (i.e., GIRL) and an unrelated control probe word (matched to the ‘related probe for a priori lexical decision RT as well as length and frequency) at each of the test points. Only one such probe word is seen by any subject for any particular sentence. What has been standardly found in studies such as this is that: (1) there is no priming for ‘boy’ at test point ‘1’; (2) there is, however, significant priming found at test point ‘2’ (indicating that the word ‘boy’ has been activated at that point); and (3) there is no longer any priming at test point ‘3’; but there is again significant priming found at test point ‘4’ (indicating that the fronted direct object ‘boy’ is reactivated at that point in the sentence). Thus, the CMLP technique appears to provide a sensitive, on-line, reflection of structural processes during ongoing sentence comprehension. There exists a large array of CMLP studies demonstrating such reactivation of a ‘fronted’ direct object NP following the matrix verb in object-relative constructions (as in the example above) as well as at theoretically similar sentence sites in English and in other languages such as German, Japanese, and Bulgarian (see, e.g., Clahsen & Featherston, 1999; Hickok, Canseco-Gonzalez, Zurif, & Grimshaw, 1992; Love & Swinney, 1996; Nagel, Shapiro, & Nawy, 1994; Nakano, Felser, & Clahsen, 2002; Nicol & Pickering, 1993; Nicol & Swinney, 1989; Stamenov & Andonova, 1998; Swinney, Ford, Frauenfelder, & Bresnan, 1987,2 as reported in Nicol & Swinney, 1989; Swinney, Zurif, Prather, & Love, 1996, among others). Although there is evidence from reading studies (most notably, those that explore the “filled-gap effect”, e.g. Crain & Fodor, 1985; Stowe, 1986) and ERP work (e.g. Garnsey, Tanenhaus, & Chapman, 1989) that is compatible with these results, only the CMLP studies provide the time-sensitive manipulations that allow determination that reactivation (as opposed to continued activation) is the mechanism by which interpretation of preposed NPs takes place. A potential problem Despite the seeming advantages of this technique, it is important to assess (as it is with any methodology) whether or not the tasks that participants are asked to perform reflect only 2 Data presented by Swinney, Ford, Frauenfelder, and Bresnan in 1987, was actually run in 1982 and had been intended at that time for a paper in a book that never appeared. The authors of that original unpublished paper were: Ford, Frauenfelder, Bresnan and Swinney.

218


the cognitive processes of interest or whether they also reflect processes introduced by the demands of the task(s). One concern about CMLP is that the task involved in responding to a visual probe target might involve some integration of the target into the ongoing sentence. It is standardly been assumed that this is not the case, but there are some sentence processing studies that do introduce these effects. For example, there are numerous studies reporting “syntactic priming” effects: a word is responded to relatively quickly if it is syntactically congruent with the sentential material that just precedes it; e.g., naming times to ‘are’ are faster than to ‘is’ following contexts such as “As they glide gracefully over the city, flying kites ____. . .” (Tyler & Marslen-Wilson, 1977; see also Boland, 1997; Cole & Segui, 1994; Cowart & Cairns, 1987; Friederici & Kilborn, 1989; Goodman, McClelland, & Gibbs, 1981; Gorrell, 1989; Gurjanov, Lukatela, Moskoljevic, Savic, & Turey, 1985; Lukatela, Kostic, Todoroic, Carello, & Turvey, 1987; McKoon, Ratcliff, & Ward, 1994; Seidenberg, Waters, Sanders, & Langer, 1984; West & Stanovich, 1986; Wright & Garrett, 1984). Not surprisingly, semantic congruence also facilitates lexical decision: a noun that makes sense in a particular sentence context is responded to more quickly than is a noun that does not make sense (e.g. Fischler & Bloom, 1979, 1980, 1985). All of these studies, while not following standard CMLP protocols, show that a sentence external target word can indeed be fitted in to an on-going sentence in some paradigms. These results are potentially problematic for findings from CMLP studies because congruence between a probe target and fragment of the sentence heard up to the point of probe presentation have not been controlled in all CMLP studies (although there are certainly a number of cases in which such control has been utilized ; see, e.g., Love & Swinney, 1996; Nagel et al., 1994; Nicol, 1993; Nicol & Pickering, 1993.) Thus, in the example sentence: The policeman saw 1 the boy 2 who the crowd at the party 3 accused 4 of the crime If a probe such as GIRL appears at point ‘1’, the sentence fragment “The policeman saw_” could potentially be integrated with the probe at that point. We believe that a control factor that dictates whether or not participants attempt to incorporate target probes into an ongoing sentence is whether or not the sentence is paused in some fashion when the target probe is presented. If the sentence continues normally and un-paused during the presentation of the visual target; then the target will be treated by the participant as independent from the sentence. But if the sentence stops mid-stream and a probe word appears, participants are likely to try to put the parts together, even if the task (e.g. lexical decision or naming) does not require it. This suggests that the continuous presentation feature of CMLP is crucial. But even the cross-modal variant may be at risk for congruence effects if the sentences are presented in a way that allows penetrability by external stimuli. For example, if the rate of presentation is particularly slow (see, e.g., McKoon, Allbritton, & Ratcliff, 1996) or otherwise disfluent (as with some synthetic speech or word by word presentation) the participants may attempt to fit the probe words into the sentence (possibly because they have the leisure to do so). The experiments The goal of this paper is to critically examine the conditions under which integration of a sentence-external target probe into an ongoing sentence may occur. In doing so, we explicitly test the assumption that is adopted by those using the cross-modal technique to probe lexical activation in sentences: when participants are understanding a fluent, auditory sentence presented at a normal rate of speech and also making a judgment about a visually presented target word which appears concurrently with some portion of the auditory sentence (but


219

before the end of that sentence), they will not integrate the visual word into the auditory sentence, and, all else being equal, will not respond more rapidly to ‘good fit’ targets than to ‘poor fit’ targets. In the current experiments, we examine the integration of target probe words into sentences—in the absence of semantic priming per se—with a set of structurally simple sentences designed to allow us to make a direct comparison between the unimodal visual lexical priming technique (such as that used by a number of investigators; e.g. McKoon et al., 1994) and two variants of the cross-modal continuous-sentence technique: one with a fluent, normal rate of sentence presentation (which we expect will not allow the concurrentlypresented visual word to intrude) and one with slower-than-normal presentation, which may allow such intrusion. Materials and materials pre-tests for experiments 1, 2, and 3 Identical materials were created for all three experiments—the unimodal visual study (Experiment 1), the cross-modal study (Experiment 2) and the slow speech cross-modal study (Experiment 3). In order to make certain that the congruent targets did, in fact, ‘fit’ with the preceding context better than the incongruent targets, these experimental stimuli were subjected to two pretests to assess congruence of the visual targets with the sentences up to the point the target was presented. The first of these, a continuation-judgment task, required participants to judge to what extent the target word formed a good continuation of the sentence; the second, a relatedness judgment task, required participants to judge how related the target word was to the sentence fragment overall. Materials Forty-eight experimental sentences were created. All sentences contained the following sequence of constituents: noun phrase, prepositional phrase, verb, noun phrase, prepositional phrase, as in example (2) below (target probe point is indicated with an asterisk). Each sentence was paired with two target words, one of which was intended to be more congruent with the sentence fragment (to provide a better fit with it, and a better continuation of it) than the other target although neither target was meant to be predictable from the prior context. Members of each target pair were matched in length and frequency, and, most importantly, a priori lexical decision times (taken from a lexical decision test performed on the words presented in isolation). Thus, in (1), the word ‘technique’ is more congruent with the sentence initial fragment than the word ‘newspaper’ on the grounds that a technique is more likely than a newspaper to be something that one masters. Clearly, both targets would constitute better sentence continuations if the sentence fragment ended with the determiner; however, our interest here was in simply to determine that there was a difference in ratings between the two types of target probe words. We chose a post-verb target probe point because that is the point at which many existing of the cross-modal lexical priming studies have probed (see object-relative example given above). (1)

The butler in the mansion mastered ∗ the difficult recipe in moments. Congruent target : TECHNIQUE Incongruent target : NEWSPAPER

In addition, the target words were paired with a second sentence that provided a different control. As the example in (2) shows, the incongruent target for sentence (1) is the congruent target in sentence (2).

220

(2)


The attendant in the hotel folded ∗ the clean towels for the new guests. Congruent target : NEWSPAPER Incongruent target : TECHNIQUE

In sum, any given target-sentence pairing had two different controls: one control was the matched control word that appeared with the same sentence (‘newspaper’ vs. ‘technique’) (we will refer to this as the matched-target procedure, which holds the sentence constant and contrasts matched target items); the other control was the same target paired with a different sentence with which it differed in congruence (‘newspaper’-congruent vs. ‘newspaper’-incongruent) (we call this the switched-target procedure, which holds the target word constant and switches the sentence with which it is paired). (See Appendix for a list of experimental materials.) Materials pretests Two pretests were conducted with the materials to determine congruence of the target with the sentence (up to the point of presentation): a continuation-judgment pretest and a relatednessjudgment pretest. Continuation-judgment pretest In the continuation judgment study, 20 participants3 were provided with booklets containing written versions of the experimental sentence fragments up to the point where the targets were to be presented (the verb). Participants were asked to rate the degree to which each probe word constituted a good or plausible continuation of the sentence at that particular point. A five-point scale was used: a score of 1 was to be used to indicate a poor continuation and a score of 5 a good continuation. If participants rated a sentence with a 2, 3, 4 or 5, they were instructed to complete the sentence using the target word as the next word, so that the basis for their judgments could be evaluated. Relatedness-judgment pretest In the relatedness judgment study, 22 participants were asked to rate the degree to which the target word was related to the sentence fragment as a whole. Results The mean ratings for both congruency judgment pretest tasks are shown in Table 1. These show that, by both measures, the pre-designated ‘congruent’ targets were significantly more congruent than the incongruent targets. (For the continuation judgments, t47 =−4.1, p=.0001; for the relatedness judgments, t47 = −16.3, p