Ultrafast Cognition - PURE

Sebastian Wallot and Guy Van Orden

Ultrafast Cognition Abstract: Observations of ultrafast cognition in human performance challenge intuitive information processing and computation metaphors of cognitive processing. Instances of ultrafast cognition are marked by ultrafast response times of reliable, accurate responses to a relatively complex stimulus. Ultrafast means response times that are as fast as a single feedforward burst of activity across the nervous system connecting eye to hand. Thus the information processing and computation metaphors in question are those in which some amount of time is required to decide and initiate a response, over and above the minimum time required, physiologically, for the eye-hand chain of action potentials — these are metaphors in which the brain does work that has a measurable duration in time. Ultrafast cognition can be explained by synergies spanning the mind and body. Synergies are temporary dynamical structures that anticipate context-appropriate behaviour. An anticipatory state poises the mind and body in symmetry among equivalent options for behaviour, and only a minimal change in context, favouring one option over any other, is sufficient to break symmetry and enact an ultrafast cognitive response. Keywords: ultrafast response time, mind–body synergy, symmetry breaking, complexity If cognition is conceived as computation then cognitive processes are akin to component processes arranged in flow charts of information processing. Thinking in terms of computation, the accuracy and speed of responding evaluates the psychological complexity of cognitive Correspondence: S. Wallot or G. Van Orden, CAP Center for Cognition, Action, & Perception, Dept. of Psychology, University of Cincinnati, Cincinnati, OH 45221-0376, USA Email: [email protected] or [email protected]

Journal of Consciousness Studies, 19, No. 5–6, 2012, pp. ??–??

2

S. WALLOT & G. VAN ORDEN

processing (all else equal). Furthermore, through systematic variation of psychologically relevant factors, contrasts among response times could allow inferences about the mental operations involved (e.g. Donders, 1868/1969; Sternberg, 1969). Speed and accuracy are equally central to the information processing perspective in cognitive neuroscience. Thus, the accuracy to identify, remember, or categorize a word, a picture, or some other stimulus, together with the speed of responding, are the ‘grande dames’ of psychological measurement. Weak supervenience is the assumption that mental operations, e.g. the operations of information processing or computation, supervene upon operations of the nervous system (Kim, 1993). For example, information processing would be rate-limited by the speed with which action potentials are propagated through the nervous system, assuming that information processing supervenes upon action potentials’ capacity to travel through space across neuropil. Whatever the actual link between mental and physical operations, that which we observe behaviourally as a change in response category or response time implies a change in mental operations and a linked change in brain operations. Conventionally, scientists who endorse computation as a working framework also endorse most or all of the previous ideas, which define the theory constitutive metaphor within which responsetime data are collected and interpreted.1 And yet ultrafast cognition culminates in response times that are as fast as, or faster than, the estimated physiological limits on information processing (Thorpe, 2002). If cognition requires any additional time, then it should add this time to the minimum physiological time — at least — even taking into account the range of variability of response times. Consequently, ultrafast cognition brings into question the traditional straightforward sensibility of information processing. Ultrafast cognition poses a challenge to the very idea of processing because no extra time exists in which ‘processing’ could occur. Ultrafast cognition does not contradict weak supervenience of mental events upon brain events. It only contradicts the metaphor of computation and information processing as exclusively or primarily a reaction to a stimulus. The emphasis instead is the anticipation of possible actions in a given situation. This shift of focus away from reaction [1]

Only theories and methods equating some aspect of mind or brain with some amount of time that must pass to complete a cognitive function are jeopardized by the observation of ultrafast cognition. Of course, other metaphors of computation may exist. Perhaps a small world network that communicates mostly through gap junctions (e.g. Draguhn et al., 1998), which oscillate in very fast activity (>80 Hz), is the right way to think about information processing, as one reviewer pointed out. We do not pretend to have exhausted such unconventional metaphors.

ULTRAFAST COGNITION

3

and toward anticipation of the actions suggests that the massive recursion between cortex and the cerebellum renders the entailed cortical activity anticipatory, as has been argued by Jordan (2008; 2009). We will review several attempts to reconcile ultrafast cognition with information processing before exploring an alternative to information processing and computation, because the time that passes between a stimulus onset and the recorded response is simply too short to ‘process’ relatively complicated stimuli (Thorpe, 2002). In the alternative, the ultrafast enaction of response behaviour is a cascade of symmetry breaking across a mind and body poised symmetrically among propensities to act.

Ultrafast Response Times The go/no-go task is often used to observe ultrafast performance. To minimize response times, the participant holds down a response key until a stimulus appears, and then responds on go-trials by releasing the key. Task instructions set up the criteria for when a ‘go’ response (release the key) is appropriate, and when a ‘no-go’ response (don’t release the key) is appropriate. In one of the best examples, the go response was made when pictures of natural scenes contained a live animal but pictures that lacked an animal required a no-go response (Macé et al., 2009b). Participants were seated in front of a computer monitor and they initiated a trial by placing a finger over the response pad, triggering the appearance of a fixation cross. Shortly after the fixation cross appeared, it was replaced by a photograph of a natural scene, presented for only 26 ms, and participants were instructed to release the response key (‘go’) as quickly as possible, but only when the scene contained an animal. The no-go response was registered if 1 sec passed from stimulus onset without a response. The overall accuracy of the category judgments was 95.7% correct and the average response time was 390 ms, impressive but not ultrafast. However, these authors were focused on describing the minimum response time at which reliable judgments were being made. To estimate the minimum response time a histogram was constructed using the correct go-times with bins spanning 10 ms. The ratio of correct hits to incorrect false alarms was calculated for each bin, and the bin of the fastest times, at which hits still reliably outnumbered false alarms, was called the minimum response time. Although the stimulus photographs were relatively complex, the minimum response time was only 265 ms, which is also a good estimate of the limit of fast

4


performance in go/no-go tasks (see also Joubert et al., 2007; Thorpe, 2002). Fabre-Thorpe et al. (2001) reported a similar study except participants received extensive practice in the go/no-go task with the photograph stimuli, as already described. Each participant performed the task on fourteen separate days during a three-week period on 200 randomly selected images, presented repeatedly. Performance improved reliably during this period, both in terms of accuracy (94% on day one, 97% on day fourteen) and response time, which decreased from an initial mean value of 480 ms to 436 ms on day fourteen. On the two days following, participants performed the task again with the 200 familiar pictures randomly intermixed among 1200 novel pictures. Accuracy was higher to the familiar pictures compared to the novel pictures (96.9% vs. 94.7%). However, the lower accuracy to novel pictures came exclusively from a few particularly difficult targets. Average response time was also shorter to familiar than to novel targets (424 ms for familiar vs. 444 ms for novel), but the slower times were once again times to the few difficult items. The fast responsetime tails of the distributions of familiar and novel pictures’ response times were virtually identical, with the fastest reliable responses spanning 260–360 ms (ibid., 2001). Other scientists have observed that participants extract the gist of pictures in fewer than 200 ms, when a novel scene is presented for a single monitor refresh without masking, performing at the limit of response speed (Greene and Oliva, 2009; Grill-Spector and Kanwisher, 2005). And only a 9 ms glimpse can provide distance information, even with masking (Gajewski et al., 2010). The removal of chromatic information does not qualify ultrafast performance (Delorme, Richard and Fabre-Thorpe, 2000), and ultrafast performance is relatively unaffected when a picture is severely degraded — accuracy finally dropped to chance when the contrast was reduced to 3% of the original presentation conditions (Macé et al., 2009a). The previous tasks used visual presentation, but Porter and Castellanos (1980) presented vowel-consonant-vowel (VCV, e.g. /apa/, /ama/) constructions in which the first vowel was always an /a/, and the participant was instructed to repeat it out loud as quickly as possible. The trailing CV of the VCV was presented next, following a randomly selected gap of two to five seconds. The participant had also been instructed to repeat out loud the trailing CV as fast as possible. The CV consonant was always one of the set: /p/, /b/, /m/, /g/, or /k/ and always ended in /a/. The crucial measurement was the shadowing time to the correct CV articulations. On average, shadowing time was

ULTRAFAST COGNITION

5

223 ms with the fastest participant responding within 172 ms, shadowing times virtually equivalent to the same participants’ simple reaction time of 171 ms on average. Kozhevnikov and Chistovich (1965) report shadowing times of less than 150 ms in a similar task set-up.

Information Processing To hold on to both information processing and weak supervenience, the speed of information processing must respect that there are inherent limits on the speed of transmission of action potentials across the nervous system. As regards these limits, Thorpe (2002) has pointed out that the ultrafast response times, already discussed, impose strict limits on the nature of information processing. The fastest response times observed would limit the nervous system to a single pass eye-tofinger, or ear-to-mouth. No time remains for neural activity that we might meaningfully see as ‘information processing’. These facts basically rule out information processing as neural rate coding or summation of activation, because for every putative stage of information processing there are only 10 to 20 ms available, but ‘the firing rates seen in cortical neurons rarely go over about 100 spikes per second, which means that in 10 ms, one is very unlikely to see more than one spike’ (ibid., p. 5). Thorpe describes alternative models of information processing, however, that do respect the available facts: information encoded in higher moments of a neural signal or in a rank-order coding. The higher moment of signal amplitude could encode information, for instance, if the output of a neuron is proportional to its input — in that case, the strength of the incoming signal could carry information, as could the numbers of activating and inhibiting inputs. Still at issue, though, is whether higher moments could achieve the high accuracy of ultrafast decision performance, observed of humans and monkeys, because too few neurons may be involved to support such reliable outcomes (ibid., 2002). Also, these schemes rely on locally connected strands of single, individual neurons, creating an overly fragile neural representation, too easily disrupted. Yet local damage to specific neural chains does not eliminate stored information, either piecemeal or catastrophically (Lashley, 1929). The equally vulnerable alternative idea, rank-order coding, encodes information in the order in which cells fire, not their firing frequency. Seeing a stimulus photo of a dog activates a precise order, along a chain of neurons that distinguishes the dog’s picture from other pictures that activated the same neurons in a different order.

6


Thorpe (2002) used both rank-order and higher moment coding in a SpikeNet model, which did achieve a high accuracy of picture recognition, in model conditions that could translate into ultrafast performance. But as Thorpe points out, the stimuli for the simulations were much less complex than the pictures used in human experiments. The concern might be answered by future progress in computer technology, but the conceptual incompatibilities of this approach (single neurons connected in feedforward strands) with nervous systems (small world networks of recurrent feedback) also matter in our calling for a different approach altogether. A third contender to explain ultrafast cognition is that the context that is provided within a complex photograph makes possible the ultrafast performance. Whether context can lighten the load of information processing, however, very much depends on what is actually meant by context. Context must somehow enrich a target stimulus for faster processing: an animal can be expected in natural environments, for example, such as woods instead of city streets, or in a living room containing a cage versus a living room without a cage. And perhaps the rich contrasting features in the photograph of a whole animal (compared to an abstract stylized picture of an animal) are contexts as well, each facilitating perception of the other. For context to be effective under this premise, both sources of information must be extracted initially, nevertheless, if the context is to facilitate later stages of perception and cognition (e.g. Arminoff, Gronau and Bar, 2007; Bar, 2004). This context-as-enrichment conception is not without problems. If perception derives initially from elementary features, then the simultaneous processing of context and stimulus features could increase the load of early processing stages. All else equal, context increases the number of features that need to be extracted initially, so that context can play a facilitative role at a later stage. This initial analysis would seem to be a necessary basis for downstream effects of the ‘context stimulus’ (e.g. Crowder, 1982; Lindsay and Norman, 1972; Neisser, 1967). These increased demands at earlier stages could actually motivate an expectation of slower response times, with the addition of context, because the feature extraction demands are higher, and processing cannot be streamlined any further. That is, ultrafast cognition is so fast that it leaves no wiggle room for ad hoc explanations. With respect to the physiological underpinnings of information processing, processing time is already close to physiologically realistic minima, so even facilitative effects would be grossly attenuated. We are not claiming, here, that a within-picture context has no

ULTRAFAST COGNITION

7

influence on response times; it does (e.g. Joubert et al., 2007) — although arbitrary geometrical shapes, lacking contextual cues, produce response times just as fast as those observed in natural scene categorization (Rauschenberger et al., 2004). We simply claim that a context appearing simultaneously with a target has equivocal capacities to disadvantage or advantage information processing, which must qualify its potential to motivate a successful information processing explanation.

Context is Constitutive of Cognition Perhaps the answer lies in a re-conceptualization of what counts as context, and how fundamental the role of the re-conceptualized context is. From the information processing perspective a context is like another stimulus that may precede, follow, or appear simultaneous with the stimulus at issue. Underlying cognitive processes that process the stimulus at issue can be primed or biased by a context, but are not drastically changed by the context-stimulus. There is, however, a broader conceptualization of context in which context has a truly constitutive role in cognition and cognitive activities, and does not merely modulate quasi-static stimulus processing. Timo Järvilehto (1998) has outlined a theory of the organism-environment system that places context in a more fundamental role. The crucial question in which Järvilehto’s theory of the organism-environment system originates is whether the organism and the environment are two causally distinct systems that somehow make contact with one another, or interdependent parts of a single integrated system? The answer has both conceptual and empirical implications because Järvilehto contends that stimulus, context, and cognizer make no separate or causally dissociable contributions in cognitive performance. Järvilehto (1998) claims that it is not possible to reliably demarcate where an organism ends and its environment begins. At first hearing this may sound nonsensical, because the boundary between a quiet, still organism and an unchanging environment is perceptually salient — the organism ends at its skin, right, or at the ends of hairs extending past the skin, or surely in the fuzzy boundary between the so-called matter composing the organism and the matter composing its surroundings? The idea becomes less silly and more appropriately challenging once we rephrase the question to become a question about measuring the system (whether organism alone or organism-environment system) and what we can discern from measurements alone (see also Holden et al., in press; Van Orden, Kello and Holden, 2010).

8


Measurements reveal the nature of systems by the way the measurements change when the system changes. To induce a system change, we manipulate the system by changing it or the environment that the system responds to. In the case of an organism, however, we cannot induce a change in the organism’s response without, strictly speaking, also changing its environment (our actions are part of its environment) and we cannot change the environment it responds to without changing the organism that responds (a change in the environment entails change(s) in the organism, in so far as the change will be perceived or will have other imperceptible consequences for interaction). Laboratory manipulations and measurements of behaviour thus, strictly speaking, always induce changes in an organism-environment system. What is at issue, fundamentally, is the constitutive role of the environment in an organism’s history, and in the organism’s present performance. Quasi-static systems exist outside of history, in the sense of reversible time. An idealized clock is a quasi-static system, for example, that changes in the same way at all times and would change in the same way at all times were it running backwards. An organism, however, changes irreversibly over time and its history of discontinuous choice points is the source of the irreversibility. At any given point in time an organism confronts choices. It can jump to the left or the right, eat ice cream or cake, engage in fight or flight, but once the choice occurs the alternative is lost to the past. Time cannot be reversed to retrieve a missed opportunity. Järvilehto (1998) supplies a more poetic picture of this entanglement. A person reaches out to take in hand and drink a cup of coffee, and Järvilehto asks at what point does cognition make contact with the coffee? When the hand touched the cup? When the coffee entered the mouth? The stomach? When does the coffee become a sensation? When the caffeine enters the bloodstream? The brain? Or did cognition come into play when the coffee was purchased, or when the grower harvested the bean, or when long-dead men and women discovered the pleasant and energizing effects of boiled beans? These are all historical conditions on which coffee drinking is contingent. Järvilehto (ibid.) concludes that no point exists at which such a line can be drawn. With respect to cognition and coffee, it is unreasonable to ask for a line of demarcation between the environment and the organism. He explains further that to define behaviour, one needs to specify the elements of each system unambiguously because, on a fundamental level, behaviour in the two-systems perspective entails the idea of elements changing in one system without regard to the

ULTRAFAST COGNITION

9

other system, which is only possible if there is a definable boundary between the systems in the first place. Response outcomes are choices, whether we focus on the response category or on the details of movement kinematics enacted in the response trajectory (Hollis, 2010). Each response behaviour is a singularity, a unique behavioural trajectory, shaped by the contingencies of circumstances (Bernstein, 1967). A response outcome does not distinguish an organism from its environment, however, because the organism’s surroundings are entwined with the production of the response, its functionality, and consequences (Ulanowicz, 2006). A measurement outcome that could distinguish an organism from its surroundings would be additive effects of crossed environmental factors, as in multifactorial ANOVA, indicating that effects of environmental factors are independent of the organism’s history (cf. Lewontin, 1974), but that outcome is rarely or never observed (Van Orden, Pennington and Stone, 2001). Thus, despite the fact that the organism clearly embodies the response, the embodiment itself is causally embedded within the organism’s circumstantial history of present and previous surroundings. No matter how narrowly and mechanically one conceives of choice, whether the rigidly determined product of an organism-environment history, or the probabilistic combination of the current states of the organism and its surroundings, both organism and environment are always complicit in the unique response outcome that results. But Järvilehto (1998) suggests that the organization of behaviour must be seen in a broader context, taking into account that the organism and the environment are a single system in behaviour’s organization: ‘To continue its life process every organism must archive positive results. Thus, the general architecture of any organism-environment system corresponds to these results’ (ibid., p. 330), and only taking a historical perspective and looking at the development of the necessary condition for the achievement of certain results, or outcomes, may understand its system dynamics. System outcomes are the factors to which all the system organization is dedicated. Järvilehto’s hypothesis supplies a broader notion of what context is, which we can use to re-evaluate the animacy judgments of Macé et al. (2009) for example. In that example, the ‘go’ response indicating animacy follows from products of learning, development, and intentionality, all tightly coordinated to anticipate, and react to, the appearance of the stimulus. As Järvilehto points out, ‘The subject must have undergone a certain phylo- and ontogenetic development. S/he must have acquired ears, fingers, and finger muscles. S/he must

10


have come to the experiment. S/he must sit during the experiment in a certain way, listen and remember the instructions of the experimenter, and so on’ (Järvilehto, 1998, p. 331). The stimulus can no longer be seen simply as the cause of a response, the stimulus is no more causally distinct for the response than the fact that participants have fingers. The consequences for the body of the immediate pre-organization that anticipates a stimulus can be observed throughout the body as an anticipatory poise supporting fast performance. For example, the intention to respond results in muscle tensions across the body, including ‘heightened tonicity of the reactive mechanisms… widespread contraction of skeletal muscles… marked changes in breathing, heart rate, and vascular processes… and an increased readiness of arousal for associations within a given sphere’ (Bills, 1934, p. 408). The anticipatory poise, prior to the stimulus, also makes clear that the stimulus is just one of the very many pieces that must converge to produce an ultrafast performance. Previous treatments of the stimulus as though it is a single cause of performance have ignored the elaborate coincident organization of the organism-environment system. Among other things, this organization limits responding to the choices dictated by the experimenter, prior to the stimulus appearance — hence, ‘processing’ of the stimulus is already almost complete before the stimulus appears. That is why ultrafast response times are obtained in these tasks, so close to the minimum ‘fast’ boundary of simple reaction times. And that is why the stimulus appears to be ‘processed’ in a single feedforward pass, near the ultimate speed of neural signal propagation (Thorpe, 2002). Summing up, ultrafast performance has forced us to seriously consider that context is a crucial constituent of ultrafast cognition. Context is the immediate embedding of organization and coordination between the organism and the environment, as well as significant previous history. Once context is seen in this broader constitutive role, it can also be seen to contribute to a solution to ultrafast cognition. The intertwining of participant and circumstances to favour the appropriate response options on a stimulus trial can make ‘responding’ appear more like ‘reacting’ and ‘response time’, more like ‘simple reaction time’. Behavioural choices are sufficiently narrowed that choice itself may contribute minimally to the overall response time.

ULTRAFAST COGNITION

11

Perturbation of Perception and Action Järvilehto’s (1998) organism-environment hypothesis presents the most promising basis, considered so far, on which to explain ultrafast cognition. This section reviews additional instances of ultrafast performance, before elaborating further on a possible explanation. These reports of ultrafast performance come from perturbation paradigms, which are a means to study a system’s dynamics. In a perturbation experiment, the participant is already in the flow of action when a perturbation stimulus occurs, requiring a new action trajectory, which necessitates information processing and new computations. Perturbation studies also test the hypothesis that the flow of action always entails anticipation of future action possibilities, as proposed by Jordan (2008; 2009; see also Stepp and Turvey, 2009). Altogether, we describe six more examples — two eye-movement studies, a force production experiment, gait adjustment to an unexpected obstacle, a speeded pointing task, and articulatory adjustments after a perturbation of ongoing speech. Kirchner and Thorpe (2006) report a variation of the go/no-go task using eye-movements instead of key presses. Although the eye-movement set-up is similar to the go/no-go animacy judgments reviewed already, the eye-movement set-up is a perturbation task because the eyes are ceaselessly in motion. Microsaccades occur as an ongoing behavioural stream that cannot be suppressed or put on hold (Rolfs, 2009). In this perturbation set-up, participants were presented with two pictures, simultaneously, on a video monitor. The simultaneous pictures were natural scenes flashed briefly, for only 20 ms, one in the left hemifield and one in the right hemifield. The correct response was a saccade, made as fast as possible, to the natural scene containing an animal, and in this regard was a straightforward eye-movement version of the animacy judgment. Animate targets were equiprobable in both hemifields and performance was very accurate, averaging over 90% correct. Eye-movement responses occurred reliably within 150 ms after stimulus onset, with the fastest participants’ saccades coming within 120 ms. Even at the fastest response times, accuracy remained high at over 85% correct, on average. Altmann (2010) reported a similarly ultrafast eye-movement performance. Participants were shown pictures, presented simultaneously on a computer monitor, that included a man, a woman, and various other items. One second after the pictures appeared, participants heard a sentence referring to either the man or the woman, and to one of the other items in the picture, e.g. ‘the man will eat the cake’ or

12


‘the woman will read the newspaper’. The response measurement was the time that passed from the onset of the spoken word ‘man’ or ‘woman’ until a participant initiated a saccade toward the pictured man, or the pictured woman. The distribution of saccade times was binned using intervals of 40 ms, and the notable result was that the ultrafast 81–120 ms bin of saccade times contained reliably correct saccade responses. Slifkin, Vaillancourt and Newell (2000) used the force-production task with feedback to estimate the minimum timespan between action and useful feedback. Participants produced a designated level of force against a wire cable with their index finger and visual feedback was represented as the distance from a horizontal line, representing the designated force, on a computer monitor. The perturbation in this case was the feedback itself, which was displayed at refresh durations ranging from 4 ms to 500 ms. The refresh duration rate-limited the availability of the feedback information to changes in force production, and participants better capitalized on shorter refresh durations, reaching durations as short as 150 ms, but did not benefit further from durations shorter than 150 ms. At shorter durations, feedback is as much a perturbation as it is useful information. Thus 150 ms estimates the minimum time between action and perception for use of feedback in force production. Weerdesteyn et al. (2004) observed slightly faster intervals between perception and action, in gait adjustments contingent upon an unexpected obstacle. Markers were attached to a participant’s heel and big toe and the positions of the markers were recorded with a six-camera 3D motion analysis system, sampling the locations of the heel and toe every ten milliseconds. The unexpected obstacle was released while participants walked on a treadmill, and the time to an adjustment in gait could be measured using changes in heel and toe positions. The shortest intervals recorded were within 120 ms, to change gait reliably and avoid the unexpected obstacle. Brenner and Smeets (2003) used a perturbation to redirect in-progress pointing movements away from one target and toward another. On each trial, participants moved their index finger from a start position to a lighted target position. The average movement time, from start position to target, was estimated at 395 ms; and the average time after a target lighted, to lift the finger from the start position, was estimated at 200 ms. On some of the trials, however, after the finger left the start position, toward a lit target, the light went out and a new second ‘perturbation’ target lit up either to the left or to the right of the first target. The position of the subject’s index finger was recorded

ULTRAFAST COGNITION

13

every two milliseconds, to estimate how quickly participants could change their finger trajectory toward the new target position. Changes to the new trajectory occurred in just over 100 ms.2 Kelso et al. (1984) have observed perhaps the shortest duration between perception and action, relative to a perturbation. Participants said aloud either /baeb/ or /baez/, and a perturbing force was applied unexpectedly to the jaw during the closing gesture for the final /b/ or /z/. The perturbation was a short precise pull to the jaw. The compensation for the jaw’s displacement began within five to ten milliseconds, faster than a loop of neurons could have calculated a new configuration. The new configuration was necessary nonetheless because it is the lower lip that compensates for the perturbation, not the jaw that was tugged. And yet within five to ten milliseconds, the lower lip already begins to stretch upward to supply the necessary relation with the upper lip, and with the other articulatory components, to say /b/, preserving the legibility of articulation in this idiosyncratic arrangement, equivalent to an unperturbed utterance (see also Abbs and Gracco, 1984; Folkins and Zimmermann, 1982). What do we conclude from these perturbation studies? First, all of these studies of ongoing activity demonstrate ultrafast compensation for perturbations, compensation that comes too soon to have required information processing. Second, the compensation must have taken into account the immediate history of activity in the solution, consistent with Järvilehto’s hypothesis, lest participants stumble, make errors, or go cross-eyed. Third, the ongoing activity appears to have anticipated solutions to the perturbations (within boundary conditions), consistent with Jordan’s anticipation hypothesis. We cash in these conclusions in the next section, describing a solution to ultrafast performance that is anticipatory and contextually embedded.

Change in Context Triggers Symmetry Breaking As we explain, mind–body synergies accomplish the dynamical organization of behaviour. Synergies are temporary (softly assembled) feedback loops reflecting the immediate constraints of a task situation or context, and anticipating dynamical solutions as ‘choice options’ and compensations for perturbations (Kloos and Van Orden, 2009). Consequently, ultrafast performance can occur as fast as a one-way chain of activation across the body, or faster. For instance, zero[2]

As one of the reviewers pointed out, it seems likely that the response observed by Brenner and Smeets (2003) began even sooner than 100 ms, as the arm’s momentum had to be overcome before an acceleration in a different direction could be detectable.

14


lag-time mechanical feedback exists in the body, capitalizing on the elastic properties of the body, creating so-called preflexes, which anticipate and dampen perturbations without computation (Brown and Loeb, 2000; Ostry and Feldman, 2003; Nishikawa et al., 2007). Anticipatory mechanical feedback shares its anticipatory capability with abstract counterparts — namely, the synergies spanning the mind and body (Turvey, 2007). Synergies self-organize the mind and body together, in a tighter coupling than was expected based upon the metaphors of computation and information processing. Synergies of neuromusculoskeletal dynamics extend to the sensorimotor periphery of the individual, to extend the coupling of mind and body into the immediate context of the individual, and provide the means by which context constrains and is constitutive of cognition. In a laboratory study, the participant’s intentions, inherited from the instructions, constrain the degrees of freedom for behaviour to the trajectories of allowed responses (Van Orden, Kloos and Wallot , 2011). Thus intentions poise the body as an at-the-ready set of propensities to respond, and any contingency of context, favouring one trajectory over the others, even by the slightest fraction, will realize that response trajectory (Kloos and Van Orden, 2010; Van Orden, 2010). This symmetry-breaking hypothesis minimizes the role of the stimulus in response decisions — a stimulus is just one changing aspect of context — explaining ultrafast cognition as a consequence. To test whether a synergy exists, you perturb one of its components and observe how the others change (Kelso, 2009). For instance, the perturbation experiment that included the unexpected pull to the jaw, while speaking, was a successful test, corroborating a synergy (Kelso et al., 1984). Perturbation studies have also demonstrated that synergetic coupling exists between cognitive and linguistic factors and the periphery of motor coordination. For example, an abrupt tug to one lip, during on going speech, has an immediate consequence for both bilabial and laryngeal gestures. The ultrafast compensation of lip gestures takes into account abstract phonology while simultaneous ultrafast compensation coordinates the concrete peripheral kinematics of the larynx (Saltzman et al., 1998; see also Bauer, Jancke and Kalveram, 1995). Complex dynamical systems characteristically include coupling across different levels of organization, consistent with the observation of bi-level coupling of kinematic and linguistic dynamics (van Lieshout, 2004). For example, any time that an arm is raised, remote muscles on the opposite side of the body must anticipate the arm movement and

ULTRAFAST COGNITION

15

change to prepare for the movement prior to any change in arm position, to keep from falling over. Suppose that the arm movement is to signal a cognitive choice, such as whether a visually presented letter string correctly spells a word in American English — raise one arm for a ‘word’ response and the other for a ‘non-word’ response. Response times of multi-level anticipatory changes can be measured by the onset of electromyographic activity in thigh, trunk, and shoulder muscles, in addition to measuring arm movements. Moreno, Stepp and Turvey (2011) conducted this experiment, observing average ‘word’ decision times of 649 ms in arm movement data. Words were also reliably distinguished by the anticipatory changes of the body, however, which preceded arm movement response times, on average, by 225 ms at the thigh, 189 ms at the trunk, and 120 ms at the shoulder of the ‘word’ arm. The multi-level ‘whole body’ anticipatory ‘word’ response is possible because abstract synergies span the mind and body to coordinate ‘whole organism’ activity (ibid.). Synergies enable ultrafast performance in cognitive tasks. For instance, ordinary speech articulation is enacted in about 70 muscles, working in concert. Uttering the equivalent of a single phoneme will realize coordinated changes among all these muscles, as the articulators get into the right arrangement by the right time (Turvey, 2007). In the protracted ballet of a conversation, each muscle will take on multiple different roles to actualize the desired speech. The essential problem of speech then is not one of rigid, hierarchical, top-down control but a heterarchical interdependent coordination among its many components, within the narrow alleyways required for legible speech with respect to one component’s position relative to the others’ (e.g. van Lieshout et al., 2007). The implied degrees of freedom problem contrasts the vastly larger number of possible arrangements of speech components, and other neuromusculoskeletal components, compared to the smaller number of arrangements that produce legible speech, or other contextually appropriate behaviours. Synergies solve the degrees of freedom problem in temporary, feedback loop linkages among neuromusculoskeletal components (Bernstein, 1967; Turvey, 1990; 2007; van Lieshout, 2004). Temporary links emerge in neurophysiological, chemical, and mechanical feedback interactions, which constrain the possibilities for behaviour to those behaviours that satisfy linked constraints. Dynamical feedback links the components together to constrain each other’s activities, which entails prediction about where one component must be when a linked component is somewhere else, yielding robust anticipation, while simultaneously ruling out inappropriate actions.

16


Synergetic coupling of mind and body is also consistent with evidence of synergetic coupling between brain and behaviour, described in Kelso (1995). Participants were presented with incrementally changing, synthetic, auditory presentations of the word ‘say’ and pressed one of two keys to indicate whether ‘say’ or ‘stay’ was perceived (compare Tuller et al., 1994). Incremental changes were made in a gap of silence, immediately after the initial fricative /s/, which was lengthened in increments of ten milliseconds until participants reported hearing the word ‘stay’ (instead of ‘say’). SQUID images of the participant’s brain, EEG, and behavioural measures were all collected coincident with the increments of change along the ‘say-stay’ continuum. On each presentation of ‘say’, just past the onset of the vowel, brain dynamics were highly similar but then suddenly collapsed into one of two distinct patterns, predicting reliably whether the participant would report ‘say’ or ‘stay’. With a sufficiently long gap of silence prior to the vowel onset, however, perception changed from one trial to the next, from ‘say’ to ‘stay’. In the process, brain and behavioural dynamics exhibited diagnostic features of a bifurcation or phase transition, the wholesale reorganization of the brain-behavioural system in a sudden jump, with the brain ‘jumping’ less than 100 ms before the behaviour (Kelso, 1995). Thus, changes in the environment (incremental changes in a gap of silence), mind (the perception of ‘say’), brain and body (SQUID images and the dynamics of the EEG signal), and behaviour (the key-press response) reorganize ultrafast altogether, and it becomes impossible to draw a sharp line between the interactions of one with another, to separate the activities of one from the other. Computations are too costly and combinatorially explosive to do the job of anticipation, producing expectations that are too narrow, fragile, and intolerant of perturbation or variety (Dreyfus, 1979; 1992). Simulation studies have introduced models with a capacity for ‘strong anticipation’ that are consistent with the present conclusions (e.g. see Stepp and Turvey, 2009). And muscles and tendons do not require a centralized computational controller; the components are, in a sense, already coordinated in a whole-body tensegrity structure (Turvey, 2007). Tautly constrained musculoskeletal relations enact behaviour, another concrete analogue of the abstract synergies that span mind and body. Possible actions are in this way inherent in the constraints that govern synergetic tensegrity; possible actions are anticipated in the immediate organization of the mind and body (compare Stepp and Turvey, 2009; Van Orden et al., 2011).

ULTRAFAST COGNITION

17

In sum, neuromusculoskeletal synergies span the mind and body and make possible ultrafast response times in cognitive data. Intentions to perform self-organize as constraints, poising the body to sustain propensities to respond. Past this point, no more than a minimal change in context, to favour one propensity, enacts performance. The careful attention to details of experimental control in laboratory contexts creates of the participant the sufficiently constrained propensities for action to support ultrafast cognition. Subsequently, the ‘stimulus’ can supply the slight change in a given context to favour one response, enacting a cascade of symmetry breaking resulting in response behaviour, in no more time than it would take for a single cascade of activation from eye to hand.

Acknowledgments We thank our colleagues at the CAP Center, especially Mike Riley and Kevin Shockley. Preparation of this article was supported by NSF grants to Guy Van Orden (BCS #0642716; BCS #0843133; DHB #0728743). Sebastian Wallot acknowledges funding from the Marie Curie TESIS network. References Abbs, J.H. & Gracco, V.L. (1984) Control of complex motor gestures: Orofacial muscle responses to load perturbations of lip during speech, Journal of Neurophysiology, 51 (4), pp. 705–723. Altmann, G.T.M. (2010) Language can mediate eye movement control within 100 milliseconds, regardless of whether there is anything to move the eyes to, Acta Psychologica, doi:10.1016/j.actpsy.2010.09.009. Arminoff, E., Gronau, N. & Bar, M. (2007) The parahippocampal cortex mediates spatial and nonspatial associations, Cerebral Cortex, 17 (7), pp. 1493–1503. Bar, M. (2004) Visual objects in context, Nature Reviews Neuroscience, 5, pp. 617–629. Bauer, A., Jancke, L. & Kalveram, K.T. (1995) Mechanical perturbation of jaw movements during speech: Effects on articulation and phonation, Perceptual & Motor Skills, 80 (3), pp. 1108–1112. Bernstein, N.A. (1967) Coordination and Regulation of Movements, New York: Pergamon Press. Bills, A.G. (1934) General Experimental Psychology, New York: Longmans, Green and Co. Brenner, E. & Smeets, J.B.J. (2003) Perceptual requirements for fast manual responses, Experimental Brain Research, 153 (2), pp. 246–252. Brown, I.E. & Loeb, G.E. (2000) A reductionist approach to creating and using neuromusculoskeletal models, in Winter, J.M. & Cargo, P.E. (eds.) Biomechanics and Neural Control of Posture and Movement, New York: Springer. Crowder, R.G. (1982) The Psychology Of Reading: An Introduction, New York: Oxford University Press. Delorme, A., Richard, G. & Fabre-Thorpe, M. (2000) Ultra-rapid categorisation of natural scenes does not rely on colour cues: A study in monkeys and humans, Vision Research, 40 (16), pp. 2187–2200.

18


Donders, F.C. (1868/1969) On the speed of mental processes, Acta Psychologica, 30 (C), pp. 412–431. Draguhn, A., Traub, R.D., Schmitz, D. & Jefferys, J.G.R. (1998) Electrical coupling underlies high-frequency oscillations in the hippocampus in vitro, Nature, 394 (6689), pp. 189–192. Dreyfus, H.L. (1979) What Computers Can’t Do, Cambridge, MA: MIT Press. Dreyfus, H.L. (1992) What Computers Still Can’t Do, Cambridge, MA: MIT Press. Fabre-Thorpe, M., Delorme, A., Marlot, C. & Thorpe, S. (2001) A limit to the speed of processing in ultra-rapid visual categorization of novel natural scenes, Journal of Cognitive Neuroscience, 13 (2), pp. 1–10. Folkins, J.W. & Zimmermann, G.N. (1982) Lip and jaw interaction during speech: Responses to perturbation of lower-lip movement prior to bilabial closure, Journal of the Acoustical Society of America, 71 (5), pp. 1225–1233. Gajewski, D.A., Philbeck, J.W., Pothier, S. & Chichka, D. (2010) From the most fleeting of glimpses: On the time course for the extraction of distance information, Psychological Science, 21 (10), pp. 1446–1453. Greene, M.R. & Oliva, A. (2009) The briefest of glances: The time course of natural scene understanding, Psychological Science, 20 (4), pp. 464–472. Grill-Spector, K. & Kanwisher, N. (2005) Visual recognition: As soon as you know it is there, you know what it is, Psychological Science, 16 (2), pp. 152–160. Holden, J.G., Choi, I., Amazeen, P.G. & Van Orden, G. (2010) Fractal 1/f dynamics suggest entanglement of measurement and human performance, Journal of Experimental Psychology: Human Perception and Performance, 37 (3), pp. 935–948. Hollis, G. (2010) The Role of Task Constraints in Ambiguity Resolution, unpublished dissertation, University of Cincinnati, Ohio. Järvilehto, T. (1998) The theory of the organism-environment system: I. Description of the theory, Integrative Physiological and Behavioral Science, 33 (4), pp. 321–334. Jordan, J.S. (2008) Wild-agency: Nested intentionalities in neuroscience and archeology, Philosophical Transactions of the Royal Society B, 363 (1499), pp. 1981–1991. Jordan, J.S. (2009) Forward-looking aspects of perception-action coupling as a basis for embodied communication, Discourse Processes, 46 (2–3), pp. 127–144. Joubert, O.R., Rousselet, G.A., Fize, D. & Fabre-Thorope, M. (2007) Processing scene context: Fast categorization and object interference, Vision Research, 47 (26), pp. 3286–3297. Kelso, J.A.S. (1995) Dynamic Patterns, Cambridge, MA: MIT Press. Kelso, J.A.S. (2009) Synergies: Atoms of brain and behavior, in Sternad, D. (ed.) Progress in Motor Control: A Multidisciplinary Perspective, New York: Springer. Kelso, J.A.S., Tuller, B., Vatikiotis-Bateson, E. & Fowler, C.A. (1984) Functionally specific articulatory cooperation following jaw perturbations during speech: Evidence for coordinative structures, Journal of Experimental Psychology: Human Perception and Performance, 10 (6), pp. 812–832. Kirchner H. & Thorpe S.J. (2006) Ultra-rapid object detection with saccadic eye movements: Visual processing speed revisited, Vision Research, 46 (11), pp. 1762–1776. Kim, J. (1993) Supervenience and Mind: Selected Philosophical Essays, Cambridge: Cambridge University Press.

ULTRAFAST COGNITION

19

Kloos, H. & Van Orden, G.C. (2009) Soft-assembled mechanisms for the unified theory, in Spencer J.P., Thomas, M. & McClelland, J. (eds.) Toward a New Grand Theory of Development? Connectionism and Dynamics Systems Theory Reconsidered, Oxford: Oxford University Press. Kloos, H. & Van Orden, G. (2010) Voluntary behavior in cognitive and motor tasks, Mind & Matter, 8 (1), pp. 19–43. Kozhevnikov, V.A. & Chistovich, L.A. (1965) Speech: Articulation and Perception, Washington, DC: Clearinghouse for Federal Scientific and Technological Information. Lashley, K.S. (1929) Brain Mechanisms and Intelligence: A Quantitative Study of Injuries to the Brain, Chicago, IL: University of Chicago Press. Lewontin, R.C. (1974) The analysis of variance and the analysis of causes, American Journal of Human Genetics, 26 (3), pp. 400–411. Lindsay, P.H. & Norman, D.A. (1972) Human Information Processing, New York: Academic Press. Macé, M.J.-M., Delorme, A., Richard, G. & Fabre-Thorpe, M. (2009a) Spotting animal in natural scenes: Efficiency of humans and monkeys at very low contrasts, Animal Cognition, DOI 10.10071-009-0290-4. Macé, M.J.-M., Joubert, O.R., Nespoulous, J.-L. & Fabre-Thorpe, M. (2009b) The time-course of visual categorizations: You spot the animal faster than the bird, PLoS one, 4 (1), p. e5927. Moreno, M.A., Stepp, N. & Turvey, M.T. (2011) Whole body lexical decision, Neuroscience Letters, 490 (2), pp. 126–129. Neisser, U. (1967) Cognitive Psychology, New York: Appleton-Century-Crofts. Nishikawa, K., Biewener, A.A., Aerts, P., Ahn, A.N., Chiel, H.J., Daley, M.A., Daniel, T.L., Full, R.J., Hale, M.E., Hedrick, T.L., Lappin, A.K., Nichols, T.R., Quinn, R.D., Satterlie, R.A. & Szymik, B. (2007) Neuromechanics: An integrative approach for understanding motor control, Integrative and Comparative Biology, 47 (1), pp. 16–54. Ostry, D.J. & Feldman, A.G. (2003) A critical evaluation of the force control hypothesis in motor control, Experimental Brain Research, 221 (3), pp. 275–288. Porter, R.J. & Castellanos, F.X. (1980) Speech-production measures of speech perception: Rapid shadowing of VCV syllables, Journal of the Acoustical Society of America, 67 (4), pp. 1349–1356. Rauschenberger, R., Peterson, M.A., Mosca, F. & Bruno, N. (2004) Amodal completion in visual search: Preemption or context effects?, Psychological Science, 15 (5), pp. 351–355. Riley, M.A., Shockley, K. & Van Orden, G. (2011) Learning from the body about the mind, Topics, DOI: 10.1111/j.1756-8765.2011.01163.x. Rolfs, M. (2009) Microsaccades: Small steps on a long way, Vision Research, 49 (20), pp. 2415–2441. Saltzman, E., Löfqvist, A., Kay, B., Kinsella-Shaw, J. & Rubin, P. (1998) Dynamics of intergestural timing: A perturbation study of lip-larynx coordination, Experimental Brain Research, 123 (4), pp. 412–424. Slifkin, A.B., Vaillancourt, D.E. & Newell, K.M. (2000) Intermittency in control of continuous force production, Journal of Neurophysiology, 84 (4), pp. 1708–1718. Stepp, N. & Turvey, M.T. (2009) On strong anticipation, Cognitive Systems Research, 11 (2), pp. 148–164. Sternberg, S. (1969) The discovery of processing stages: Extensions of Donders’ method, Acta Psychologica, 30 (C), pp. 276–315.

20


Thorpe, S.J. (2002) Ultra-rapid scene categorization with a wave of spikes, Lecture Notes in Computer Science, 2525 (1), pp. 1–15. Thorpe, S., Fize, D. & Marlot, C. (1996) Speed of processing in the human visual system, Nature, 381 (6582), pp. 520–522. Tuller, B., Case, P., Ding, M. & Kelso, J.A.S. (1994) The nonlinear dynamics of speech categorization, Journal of Experimental Psychology: Human Perception and Performance, 20 (1), pp. 1–16. Turvey, M.T. (1990) Coordination, American Psychologist, 45 (8), pp. 938–953. Turvey, M.T. (2007) Action and perception at the level of synergies, Human Movement Science, 26 (4), pp. 657–697. Ulanowicz, R.E. (2006), Process ecology: A transactional worldview, International Journal of Ecodynamics, I (2), pp. 103–114. van Lieshout, P.H.H.M. (2004) Dynamical systems theory and its application in speech, in Maassen, B., Kent, R., Peters, H., van Lieshout, P. & Hulstijn, W. (eds.) Speech Motor Control in Normal and Disordered Speech, Oxford: Oxford University Press. van Lieshout, P.H.H.M., Bose, A., Square, P.A. & Steele, C.M. (2007) Speech motor control in fluent and dysfluent speech production of an individual with apraxia of speech and Broca’s aphasia, Clinical Linguistics & Phonetics, 21 (3), pp. 159–188. Van Orden, G. (2010) Voluntary performance, Medicina, 46 (9), pp. 581–594. Van Orden, G., Kello, C.T. & Holden, J.G. (2010) Situated behavior and the place of measurement in psychological theory, Ecological Psychology, 22 (1), pp. 24–43. Van Orden, G., Kloos, H. & Wallot, S. (2011) Living in the pink: Intentionality, wellness, and complexity, in Hooker, C. (ed.) Handbook of the Philosophy of Science. Volume 10: Philosophy of Complex Systems, Amsterdam: Elsevier. Van Orden, G., Pennington, B.F. & Stone, G.O. (2001) What do double dissociations prove?, Cognitive Science, 25 (1), pp. 111–172. Wallot, S. & Van Orden, G. (2011) Grounding language performance in the anticipatory dynamics of the body, Ecological Psychology, 23 (3), pp. 157–184. Weerdesteyn, V., Nienhuis, B., Hampsink. B. & Duysens, J. (2004) Gait adjustment in response to an obstacle are faster than voluntary reactions, Human Movement Science, 23 (3–4), pp. 351–363.