Delayed Feedback Disrupts the Procedural-Learning ... - CiteSeerX

Journal of Experimental Psychology: Learning, Memory, and Cognition 2005, Vol. 31, No. 1, 100 –107

Copyright 2005 by the American Psychological Association 0278-7393/05/$12.00 DOI: 10.1037/0278-7393.31.1.100

Delayed Feedback Disrupts the Procedural-Learning System but Not the Hypothesis-Testing System in Perceptual Category Learning W. Todd Maddox and A. David Ing University of Texas, Austin W. T. Maddox, F. G. Ashby, and C. J. Bohil (2003) found that delayed feedback adversely affects information-integration but not rule-based category learning in support of a multiple-systems approach to category learning. However, differences in the number of stimulus dimensions relevant to solving the task and perceptual similarity failed to rule out 2 single-system interpretations. The authors conducted an experiment that remedied these problems and replicated W. T. Maddox et al.’s findings. The experiment revealed a strong performance decrement for information-integration but not rule-based category learning under delayed feedback that was due to an increase in the number of observers using hypothesis-testing strategies to solve the information-integration task, and lower accuracy rates for the few observers using information-integration strategies.

Maddox, Ashby, and Bohil (2003) examined the effects of delayed feedback on rule-based and information-integration category learning. Rule-based category-learning tasks are those in which the category structures can be learned via some explicit reasoning process. Often, the rule that maximizes accuracy (i.e., the optimal rule) is easy to describe verbally. Informationintegration category-learning tasks are those in which accuracy is maximized only if information from two or more stimulus components (or dimensions) is integrated at some predecisional stage (Ashby & Gott, 1988). For example, the perceptual integration might take the form of a weighted linear combination of the dimensional values.1 Maddox et al. (2003) offered a critical test of a prediction derived from Ashby, Alfonso-Reese, Turken, and Waldron’s (1998) competition between verbal and implicit systems (COVIS) model of category learning. In COVIS, like in other recently proposed multiple-systems models (e.g., Erickson & Kruschke, 1998; Pickering, 1997; Reber & Squire, 1994; Smith, Patalano, & Jonides, 1998; however, see Nosofsky & Johansen, 2000), it is assumed that in humans, the learning of different types of category structures is mediated by different categorization and memory systems. Assumptions about the underlying neurobiology that mediates learning in these systems are also made in COVIS. Here we review these assumptions in enough detail to motivate this study. More thorough reviews can be found in other sources (e.g., Ashby et al., 1998; Ashby & Ell, 2001; Maddox & Ashby, 2004).

In COVIS, it is assumed that learning in rule-based tasks is dominated by an explicit hypothesis-testing system that uses working memory and executive attention and is mediated by a circuit that includes the anterior cingulate, the prefrontal cortex, and the head of the caudate nucleus. This system learns through a conscious process of hypothesis generation and testing. In contrast, learning in information-integration tasks is assumed to be dominated by an implicit, procedural-learning-based system that depends on a reward signal to strengthen the appropriate (stimulus– category) associations in a relatively automatic fashion (Ashby et al., 1998; Ashby & Ell, 2001). The procedural-learning system is mediated largely within the tail of the caudate nucleus (with visual stimuli). In primates, all of extrastriate visual cortex projects directly to the tail of the caudate nucleus, with about 10,000 visual cortical synapses converging onto each medium spiny cell in the caudate (Wilson, 1995). These medium spiny cells then project to prefrontal and premotor cortex (via the globus pallidus and thalamus; see, e.g., Alexander, DeLong, & Strick, 1986). A dopaminemediated reward signal is critical for learning in this system. The idea is that an unexpected reward causes substantia nigra neurons to release dopamine from their terminals in the caudate nucleus (Hollerman & Schultz, 1997; Schultz, 1992), and that the presence of this dopamine strengthens recently active synapses (Arbuthnott, Ingham, & Wickens, 2000; Kerr & Wickens, 2001). In rewardmediated learning, it is essential to strengthen those (and only those) synapses that actively participated in the response that elicited the reward. Because there is necessarily some delay be-

W. Todd Maddox and A. David Ing, Department of Psychology, University of Texas, Austin. This research was supported in part by National Institutes of Health Grant R01 MH59196 and by the Center for Perceptual Systems at the University of Texas, Austin. We thank Kelli Hejl and Mina WilcoxGhanoonparvar for help with data collection. Correspondence concerning this article should be addressed to W. Todd Maddox, Department of Psychology, 1 University Station A8000, University of Texas, Austin, TX 78712. E-mail: [email protected]

1

However, a conjunctive task (e.g., respond A if the stimulus is small on Dimension X and small on Dimension y) is a rule-based rather than an information-integration task, because separate decisions are first made about each dimension (e.g., small or large), and then the outcome of these decisions is combined (integration is postdecisional). Notice that the optimal rule for the conjunctive task is amenable to verbal description. In contrast, the optimal rule in information-integration tasks is often difficult or impossible to describe verbally.

100

DELAYED-FEEDBACK EFFECTS

tween response and reward delivery, a chemical trace must be maintained that signals which spines were recently active. Toni, Buchs, Nikonenko, Bron, and Muller (1999) demonstrated that calcium catalyzes morphological changes in cells that have spiny dendritic complexes (see also Calabresi, Pisani, Centonze, & Bernardi, 1996). The morphology of the dendritic spines allows this calcium-dependent trace to exist for some time after the response is initiated (Gamble & Koch, 1987; MacDermott, Mayer, Westbrook, Smith, & Barker, 1986). Under ideal conditions, the dopamine-mediated reward signal will arrive during this time and act synergistically with the calcium-mediated perceptual trace, eventually resulting in structural changes that result in learning. If the reward is delayed, then the perceptual signal will degrade, and the ensuing dopamine release will modify inappropriate synapses, which causes learning to be adversely affected. Thus, COVIS predicts that the information learned and the processing of the feedback signal should be very different in the two systems. In the explicit system, verbalizable rules that can be stored in working memory until feedback is provided are learned, even if an extended period elapses between the response and feedback. Once the feedback is presented, time and attention are required to process it. In the implicit, procedural-learning system, stimulus-to-category associations are learned that require a close temporal correspondence between the perceptual signal and the feedback; but once the feedback signal is presented, feedback processing is essentially automatic. Maddox et al. (2003) tested the prediction that a close temporal correspondence between the perceptual signal and the feedback is required for efficient learning in the procedural-learning system but not in the hypothesis-testing system by comparing rule-based and information-integration category learning across immediateand delayed-feedback conditions. In both category-learning tasks (and in the current study), the stimulus on every trial was a single sine-wave grating (i.e., Gabor stimulus) that varied across trials in spatial frequency and spatial orientation. In the rule-based condition, spatial frequency was relevant, and spatial orientation was irrelevant, and the optimal decision rule was to respond A if the spatial frequency was low and B if the spatial frequency was high. In the information-integration condition, the optimal decision rule required a linear integration of the spatial frequency and orientation values and was not verbalizable. In line with predictions from COVIS, learning under delayed-feedback conditions was as effective as under immediate-feedback conditions with rule-based categories, but with information-integration categories, delayed feedback led to a significant decrease in category-learning performance. This result is in line with the COVIS prediction that a temporal delay in feedback presentation should lead to synaptic activity in the tail of the caudate that is nonspecific to the stimulus, resulting in a modification of inappropriate synapses and, finally, leading to poor information-integration category learning. It is important to note that under ideal conditions, rule-based tasks will be solved by the explicit, hypothesis-testing system, and informationintegration tasks will be solved by the implicit, procedurallearning-based system; but under nonoptimal training conditions such as delayed feedback, COVIS predicts that the participant will likely rely on the explicit, hypothesis-testing system to solve the information-integration task, even though hypothesis-testing strat-

101

egies are not optimal. To test this prediction, Maddox et al. (2003) applied a set of models that tested for the presence of hypothesistesting or information-integration strategies. Model sets were fit individually to each participant’s data. As predicted, they found that participants were much more likely to use hypothesis-testing strategies to solve the information-integration task in delayedfeedback conditions as compared with immediate-feedback conditions. Although these data provide strong support for a theory of multiple systems, there are at least two alternative single system explanations for these results. First, one might argue that a single category-learning system solves both tasks, but that the information-integration task is more complex or difficult, because it requires the observer to represent both stimulus dimensions, whereas the rule-based task requires the observer to represent only one dimension. Delaying the feedback to this single-category learning system does not affect a task that requires a unidimensional representation, but it does affect a task that requires a two-dimensional representation. One might argue against this hypothesis by pointing out that rule-based and informationintegration category-learning performance was equated in the immediate-feedback condition to ensure that task-difficulty differences could not be used to explain the observed performance dissociation. However, to equate immediate-feedback performance, Maddox et al. (2003) lowered the category discriminability in the rule-based condition relative to the information-integration condition. This leads to the second single system explanation of the results, which states that the rule-based and informationintegration categories were difficult for different reasons. Rulebased category learning was difficult because the stimuli were perceptually very similar, whereas information-integration category learning was difficult because observers must learn to associate different regions of stimulus space with each category. Because the two tasks were difficult for different reasons, the delayed-feedback effect might not be attributable to separate category-learning systems.

Experiment Both of these alternatives are reasonable and must be ruled out before a multiple-system interpretation of the delayed-feedback prediction is accepted. To achieve this goal, we took the following approach. First, whereas the optimal rule in Maddox et al.’s (2003) rule-based task required only a unidimensional representation, the rule-based task used in the current study required a twodimensional representation (see Figure 1). In particular, the optimal rule required the observer to set a criterion on the spatial frequency dimension to determine whether the stimulus had a “low” or “high” spatial frequency and to set a separate criterion on the spatial orientation dimension to determine whether the stimulus had a “shallow” or “steep” orientation. These decisions were then integrated to generate a categorization response (postdecisional integration). The information-integration task also required a two-dimensional representation, but in this case, the integration was predecisional. Notice from Figure 1 that both the rule-based and information-integration tasks require the use of two decision rules (or bounds) and use four categories instead of two. Because few category-learning studies have examined learning with more

102

MADDOX AND ING

than two categories (however, see Maddox, Filoteo, Hejl, & Ing, 2004), this is a unique extension of the current work. Second, whereas the rule-based discriminability was very low in Maddox et al. (2003), and many stimuli from each category were perceptually similar, possibly leading to problems with perceptual discrimination, in this study, we used rule-based category structures that were more perceptually dissimilar (see Figure 1). In fact, to equate rule-based and information-integration category learning in the immediate-feedback conditions, we had to increase the rule-based category discriminability relative to information-integration category discriminability. The rule-based and information-integration category structures used in the experiment are depicted in Figure 1 along with the decision bounds that maximize accuracy. The distribution parameters are outlined in Table 1. In the rule-based (RB) task, the optimal bounds require the participant to use the conjunctive rules: Respond A if the frequency is low and the orientation is shallow, respond B if the frequency is high and the orientation is steep,

Table 1 Category Distribution Parameters for the Experiment Condition and category RB A B C D II A B C D Note.

␮f

␮o

␴f2

␴o2

covf,o

268 268 332 332

93 157 93 157

75 75 75 75

75 75 75 75

0 0 0 0

268 300 300 332

125 157 93 125

75 75 75 75

75 75 75 75

0 0 0 0

RB ⫽ rule-based; II ⫽ information integration.

respond C if the frequency is low and the orientation is shallow, or respond D if the frequency is high and the orientation is steep. In the information-integration (II) task, we constructed the category distributions by rotating the RB distributions clockwise 45° around the center of the frequency-orientation space and then reducing discriminability in order to equate performance in the immediatefeedback condition. (We conducted a series of small sample pilot studies to determine the II category distributions that met this criterion.) The optimal rule in the II condition had no simple verbal description.

Method Participants and Design We solicited 112 participants from the University of Texas, Austin community; they received course credit for participation. Twenty-seven and 29 participants completed the RB and II immediate-feedback conditions, respectively; and 29 and 27 completed the RB and II delayedfeedback conditions, respectively. No participant completed more than one experimental condition. We tested all participants for 20/20 vision using an eye chart. In nearly all of our current work with two categories, we define a “learner” as a participant who achieves 65% accuracy during the final block of trials. Because the experiment included four categories, we lowered the criterion proportionally to 32.5% accuracy during the final block of trials. The data from participants who did not meet this criterion were excluded from all subsequent analyses. This criterion excluded 5 and 3 participants from the RB and II immediate-feedback conditions, respectively, and 4 and 7 participants from the RB and II delayed-feedback conditions, respectively.2

2

Figure 1. Rule-based (A) and information-integration category structures (B) from the experiment. Each open circle denotes the spatial frequency and spatial orientation of a Gabor pattern from Category A. Each filled circle denotes a Gabor pattern from Category B. Each open square denotes a Gabor pattern from Category C. Each filled square denotes a Gabor pattern from Category D.

Two comments are in order regarding this learning criterion. First, after data collection, we examined other learning criterion values, including one that excluded no data. The qualitative pattern of results was relatively unchanged across conditions. Second, to determine whether the nonlearners displayed essentially random responding or whether they displayed systematic responding, we fit decision-bound models to the data and also models that assumed a fixed response probability for all stimuli (indicative of random or biased random responding). In all but a few cases, the models indicative of random or biased random responding provided the best account of the nonlearners’ data.


103

Stimuli and Stimulus Generation In this experiment, we used the randomization technique introduced by Ashby and Gott (1988). We generated 80 stimuli (20 from each category) from the RB categories by sampling randomly from four bivariate normal distributions. We generated the stimuli for the II categories by rotating the 80 RB stimuli clockwise by 45° around the center of the frequencyorientation space and then shifting the stimuli away from the center of the space. Each set of 80 stimuli was displayed in a random order in each of four blocks of trials. The stimuli were computer generated and displayed on a 21-in. monitor with 1,360 ⫻ 1,024 resolution in a dimly lit room. Each Gabor patch was generated using Matlab routines from Brainard’s (1997) psychophysics toolbox. We converted each random sample (x1, x2) to a stimulus by deriving the frequency, f ⫽ .25 ⫹ (x1/50), and orientation, o ⫽ x2(␲/500). The orientation scaling factors were chosen to approximately equate the salience of frequency and orientation.

Procedure The participants were informed that there were four equally likely categories, that perfect performance was possible, and that they should be as accurate as possible and not worry about speed of responding. The procedure for a typical trial was as follows: Immediate-feedback condition: Response terminated stimulus display—500 ms Mask—750 ms feedback—5-s blank screen intertrial interval Delayed-feedback condition: Response terminated stimulus display— 5-s Mask—750 ms feedback—500 ms blank screen intertrial interval The mask was a Gabor pattern that subtended approximately 11° of visual angle and was of a random frequency and orientation from within the range of stimulus values.

Results and Theoretical Analysis Analyses were performed separately on each block of data. In the ANOVA Results section, we analyze the accuracy rates using analyses of variance (ANOVAs). In the Modeling Results section, we introduce the model-based analyses.

ANOVA Results We begin by comparing RB and II immediate-feedback performance to determine whether performance was equivalent. We conducted a 2 (category structure: RB vs. II) ⫻ 4 (block) mixeddesign ANOVA on the accuracy rates. The accuracy rates averaged across participants are displayed in Figure 2A. The main effect of block was significant, F(3, 123) ⫽ 99.50, p ⬍ .001, MSE ⫽ .008, suggesting improved performance with experience but, more important, the main effect of category structure (F ⬍ 1) and the interaction (F ⬍ 1) were both nonsignificant. The learning curves for the RB and II conditions suggest that immediatefeedback performance was equated across the two types of category structures. To determine whether delayed feedback differentially affected rule-based and information-integration category learning, we conducted a 2 (category structure: RB vs. II) ⫻ 2 (feedback condition: immediate vs. delayed) ⫻ 4 (block) mixed-design ANOVA on the accuracy rates. The accuracy rates averaged across participants for

Figure 2. Proportion correct for the rule-based (RB) and informationintegration (II) immediate-feedback conditions (A), II delayed and immediate-feedback conditions (B), and RB delayed- and immediatefeedback conditions (C). Error bars represent one unit of standard error.

the immediate and delayed feedback II conditions are displayed in Figure 2B, and for the RB condition, they are displayed in Figure 2C. The main effects of block, F(3, 252) ⫽ 161.62, p ⬍ .001,

104

MADDOX AND ING

MSE ⫽ .010; and category structure, F(1, 84) ⫽ 4.40, p ⬍ .05, MSE ⫽ .101, were significant, whereas the main effect of feedback condition was not, F(1, 84) ⫽ 2.36, ns. Block did not interact with any other factor (all Fs ⬍ 1), but, most important, there was a Category Structure ⫻ Feedback Condition interaction, F(1, 84) ⫽ 3.89, p ⫽.05, MSE ⫽ .101. Post hoc analyses revealed a large decline in II performance under delayed-feedback (56%) relative to immediate-feedback (68%) conditions that was significant, t(43) ⫽ 3.29, p ⬍ .01, but no effect on RB performance on delayed feedback (69%) relative to immediate feedback (68%; t ⬍ 1). These results provide strong evidence against the number of relevant dimensions and perceptual discriminability explanations of the Maddox et al. (2003) results and, instead, provide support for the predicted dissociation between rule-based and informationintegration category learning under delayed- and immediatefeedback conditions.

Modeling Results The accuracy-based analyses provide important information regarding overall performance but tell us nothing about the types of strategies that participants might use to solve these tasks. An understanding of strategy use and how these strategies might be affected by the delay manipulation is of critical importance in a complete understanding of category learning. One hypothesis suggested by COVIS and the Maddox et al. (2003) study is that participants in the information-integration condition will be forced to resort to hypothesis-testing strategies when feedback is delayed, because learning in the tail of the caudate nucleus will be impaired. As a test of this hypothesis, we fit a number of different decisionbound models (Ashby, 1992a; Maddox & Ashby, 1993) to the data from the information-integration conditions separately by participant. Because our focus was on asymptotic performance, the models were fit only to the final block of data. Two different classes of decision-bound models were fit to the data (see Ashby, 1992a; Maddox & Ashby, 1993, for a more formal treatment of these models). One type is compatible with the assumption that participants used an explicit hypothesis-testing strategy, and in one type, an information-integration strategy is assumed. Even so, it is important to note that in these models, no detailed process assumptions are made in the sense that a number of different process accounts are compatible with each of the models (e.g., Ashby, 1992a; Ashby & Waldron, 1999). For example, if an information-integration model fits significantly better than a hypothesis-testing model, then we can be reasonably confident that participants did not use a hypothesis-testing strategy, but we learn little about which information-integration strategy might have been used (e.g., decision-bound, exemplar, or prototype interpretations would all be compatible with such results). In contrast, if a hypothesis-testing model fits significantly better than the information-integration models, then we gain confidence that participants used a hypothesis-testing strategy, but we cannot rule out all information-integration strategies, because some of these can mimic hypothesis-testing strategies. In summary, the modeling described in this section provides a powerful vehicle by which to test hypotheses about the decision strategies used by participants, but it has little to say about psychological process. The following

models were fit to each information-integration participant’s responses. Hypothesis-testing models. Three conjunctive models were applied. In the conjunction(A) model, it is assumed that the participant sets one criterion on the spatial frequency dimension and another criterion on the spatial orientation, makes an explicit decision about the level of the stimulus on each dimension, and integrates that information to generate a categorization response (Ashby & Gott, 1988; Shaw, 1982). For example, the participant might use the rule: Respond A if the spatial frequency is low and the orientation is shallow; Respond B if the spatial frequency is low and the orientation is steep; Respond C if the spatial frequency is high and the orientation is shallow; or Respond D if the spatial frequency is high and the orientation is steep. The conjunction(A) model has three free parameters: a decision criterion on spatial frequency, a decision criterion on orientation, and the variance of internal (perceptual and criterial) noise (i.e., ␴2). In the conjunction(B) model, it is assumed that the participant sets two criteria on the spatial frequency dimension that divide the dimension into low, medium, and high spatial frequencies and one criterion on the spatial orientation dimension that divides that dimension into low and high orientations. The participant then uses the following rule: Respond A if the spatial frequency is low regardless of the orientation, Respond B if the spatial frequency is medium and the orientation is high, Respond C if the spatial frequency is medium and the orientation is low, or Respond D if the spatial frequency is high, regardless of the orientation. In the conjunction(C) model, it is assumed that the participant sets one criterion on the spatial frequency dimension that divides that dimension into low and high frequency and two criteria on the spatial orientation dimension that divide the dimension into low, medium, and high orientations. The participant then uses the following rule: Respond A if the spatial frequency is low and the spatial orientation is medium, Respond B if the spatial orientation is high, Respond C if the spatial orientation is low, or Respond D if the spatial frequency is high and the orientation is medium. Both of these models have four free parameters: three decision criteria and one noise variance. Notice also that both of these models instantiate “extreme values”-type strategies, because an extreme value on one dimension, regardless of the value on the other dimension, determines category membership. Information-integration models. In the general bilinear classifier, it is assumed that two linear decision bounds partition the space into four response regions. This model has five parameters (slope and intercept of the linear bounds and ␴2). The optimal model is a special case of the bilinear classifier in which the optimal slopes and intercepts are applied. This model has only one free parameter (noise variance). In the minimum distance classifier, it is assumed that there are four units, one associated with each category, in the frequency-orientation space. On each trial, the participant determines which unit is closest to the perceptual effect and gives the associated response. Because the location of one of the units can be fixed, and because a uniform expansion or contraction of the space will not affect the location of the resulting (minimum distance) decision bounds, the model contains six free parameters (i.e., five that determine the location of the units and one noise variance).


Model fits. We fit each of these models separately to the final block of data from each participant who participated in an information-integration condition. We estimated the model parameters using maximum likelihood (Ashby, 1992b; Wickens, 1982), and the goodness-of-fit statistic was AIC ⫽ 2r ⫺ 2lnL,

(1)

where r is the number of free parameters, and L is the likelihood of the model given the data (Akaike, 1974; Takane & Shibayama, 1992). The AIC statistic penalizes a model for extra free parameters in such a way that the smaller the AIC, the closer a model is to the true model, regardless of the number of free parameters. Thus, to find the best model among a given set of competitors, one simply computes an AIC value for each model and chooses the model associated with the smallest AIC value. Using AIC, we determined which model type, hypothesistesting or information-integration, provided the best account of the data. The proportion of data sets best fit by either a hypothesistesting or information-integration model is displayed in the stacked bar chart in Figure 3 separately for the delayed- and immediatefeedback conditions. In addition, the percent correct for the participants classified as using a hypothesis-testing or informationintegration strategy is displayed. Several comments are in order. First, as predicted by COVIS, participants were much more likely to use a hypothesis-testing strategy to solve the informationintegration task under delayed-feedback conditions (.55) than under immediate-feedback conditions (.22). In addition, whereas accuracy rates for hypothesis-testing participants were comparable across the delayed- (69%) and immediate-feedback (77%) conditions, accuracy rates for information-integration participants were much worse under delayed-feedback conditions (57%) than they were under immediate-feedback (79%) conditions. Thus, the accuracy deficit observed in the delayed-feedback condition resulted from an increase in the use of hypothesis-testing strategies and a decrease in the accuracy rate achieved by those participants who used an information-integration strategy.

Figure 3. Proportion of information-integration participants’ final block data that was best fit by either a hypothesis-testing (solid part of the bar) or an information-integration (open part of the bar) model. The percentages embedded in the plot indicate the average accuracy rates achieved by participants whose data were best fit by each model class.

105

COVIS predicts no effect of delayed feedback on rule-based category learning; a result that was observed in the accuracy results. To determine whether this also held in the strategy analyses, we applied the same models to the final block of trials from the participants who learned the rule-based category structures. As predicted, the use of hypothesis-testing strategies was high for both immediate- (.75) and delayed-feedback (.70) conditions, and the accuracy rate achieved by these participants was also high (82% and 85% for the immediate- and delayed-feedback conditions, respectively).

General Discussion In this article, we report the results from an experiment that addresses two reasonable single-system interpretations— one based on difference in perceptual similarity and one based on difference in the number of relevant dimensions across conditions— of Maddox et al.’s (2003) test of the COVIS prediction that delayed feedback should adversely affect information-integration but not rule-based category learning. In support of COVIS and in line with the results from Maddox et al., delayed feedback had no effect on rule-based category learning but led to a performance decrement in information-integration category learning that was characterized by an increase in the use of hypothesis-testing strategies under delayed-feedback conditions.

Perceptual Similarity and the Number of Relevant Dimensions One concern with Maddox et al.’s (2003) results is that rulebased category learning was difficult because the stimuli were perceptually similar, whereas information-integration category learning was difficult because participants must learn to associate different regions of stimulus space with particular categories. Another concern is that Maddox et al.’s rule-based task was unidimensional, whereas the information-integration task was twodimensional. It is possible that delayed feedback does not affect unidimensional tasks but does affect two-dimensional tasks. If either of these alternatives is correct, the delayed-feedback effect might not be attributable to separate category-learning systems. In the experiment reported above, we alleviated these problems by using highly discriminable two-dimensional rule-based and information-integration category structures. In fact, to equate immediate-feedback performance across rule-based and information-integration category structures, we had to use more highly discriminable rule-based than information-integration category structures, which is the opposite of what Maddox et al. were required to do. Therefore, we can reject those alternative explanations of the data, because information-integration category learning continued to be affected by the delay manipulation, whereas rule-based category learning was not. One might argue that there was no effect on rule-based category learning because category discriminability was larger in the rulebased condition. Although this explanation can be rejected on the basis of the original Maddox et al. (2003) findings, as a test of this hypothesis, we placed 53 additional participants (23 in the immediate-feedback and 30 in the delayed-feedback condition) in a rule-based category condition for which the category discriminability was identical to that of the II condition. The results were

106

MADDOX AND ING

clear. An ANOVA revealed a main effect of block, F(3, 129) ⫽ 64.24, p ⬍ .001, MSE ⫽ .011, but a nonsignificant main effect of category structure and interaction (Fs ⬍ 1). Thus, even when rule-based and information-integration discriminabilities were perfectly equated, and immediate-feedback rule-based performance was worse than immediate-feedback information-integration performance, no effect of delay emerged.

Importance of Identifying Strategies Used to Solve a Task COVIS postulates that the hypothesis-testing and procedurallearning systems compete throughout learning but that initially, there is a bias toward the hypothesis-testing system. This bias reverses only if the performance predicted by this system is poor relative to the implicit system. In a recent study, we examined information-integration category learning under immediate- and delayed-feedback conditions using category structures like those in Figure 1, but with higher discriminability. To our surprise, delayed feedback did not adversely affect information-integration category learning. Subsequent model-based analyses suggested that participants in the delayed- and immediate-feedback conditions were likely to use hypothesis-testing strategies. To be specific, 80% and 71% of the participants in the delayed- and immediate-feedback conditions, respectively, used hypothesis-testing strategies.3 This finding is significant because it highlights the importance of identifying the types of strategies that yield high performance levels in a task, but it has farther reaching implications for theories of category learning (Ashby & Maddox, in press). Most important, one must not assume that the participant will use a strategy of the same form as the optimal. Rather, one must be careful to determine the range of performance levels possible for different strategies in a given categorization condition and should choose conditions that are maximally diagnostic. Had we not applied the strategy analysis to these data, we might have reached a false conclusion. This issue is not unique to our task or to this particular experiment. One of the most-used paradigms for studying the neurobiological underpinnings of category learning is the weather prediction task (Knowlton, Mangels, & Squire, 1996; Knowlton, Squire, & Gluck, 1994). When it was originally introduced, it was described as a probabilistic classification task and thus was assumed to involve an implicit habit learning system. Although the optimal rule is probabilistic, Gluck, Shohamy, and Myers (2002) identified a number of strategies that people use to solve the task, suggesting that many participants solve this so-called implicit-learning task using simple verbalizable rules. In fact, the difference in predicted accuracy rate for the optimal rule and the best unidimensional rule is only 1%. In related work, Seger and Cincotta (2002) had participants learn rule-based and information-integration tasks using the randomization technique (as in our experiment). They found surprisingly few differences in brain activation across the two tasks, whereas multiple-systems theories, such as COVIS, would predict differences. It is unfortunate that one could solve the informationintegration category structures used in their study with a high level of accuracy by using a simple hypothesis-testing strategy, and model-based strategy analyses were not performed to identify the strategies used by participants. We are not claiming that participants did use hypothesis-testing strategies but only that they might have, and that this might partially explain the lack of differences in

the neural correlates observed in the two tasks. Our recommendation is that model-based strategy analyses should become the rule and not the exception in this type of research.

Working Memory Effects on Rule-Based but Not Information-Integration Category Learning The results of Maddox et al. (2003) and the current study suggest that delayed feedback adversely affects informationintegration category learning but not rule-based category learning. A number of other related effects have been observed in the literature and are reviewed by Maddox and Ashby (2004). For example, when learning is unsupervised (Ashby, Queller, & Berretty, 1999), or training is observational (Ashby, Maddox, & Bohil, 2002), information-integration category learning but not rulebased category learning is adversely affected. Likewise, when the response location associated with each categorization response is reversed or is not constant on each trial, information-integration, but not rule-based category learning, suffers (Ashby, Ell, & Waldron, 2003; Maddox, Bohil, & Ing, in press). Dissociations in the opposite direction have also been identified. For example, if participants have to perform a category-learning and a numerical Stroop task simultaneously, then rule-based category learning is hindered, whereas information-integration category learning is not (Waldron & Ashby, 2001). The numerical Stroop task taps working memory resources and thus affects rulebased category learning. In a related study, Maddox, Ashby, Ing, and Pickering (in press) asked participants on each trial to perform a category-learning task followed by a memory scanning task. In the long-delay condition, a long temporal delay was inserted between the category-learning and memory-scanning tasks. In the short-delay condition, a short temporal delay was included. If feedback processing in the rule-based task requires time, effort, and working memory, then rule-based category learning should have been worse in the short-delay condition than it was in the long-delay condition. On the other hand, if feedback processing in the information-integration task is essentially automatic (assuming a close temporal correspondence between response and feedback presentation), then the delay should have had no effect on information-integration category learning. In line with these predictions, rule-based, but not information-integration category learning, was affected by the short delay. Taken together, these two lines of work (those that introduce manipulations that affect information-integration and those that affect rule-based category learning) provide strong support for a multiple-systems approach and for the neurobiological underpinnings proposed in COVIS. 3 Ninety participants completed this study. We conducted a 2 (category structure: RB vs. II) ⫻ 2 (feedback condition: immediate vs. delayed) ⫻ 4 (block) mixed-design ANOVA on the accuracy rates. The main effect of block, F(3, 225) ⫽ 145.97, p ⬍ .001, MSE ⫽ .009, was significant. The main effect of category structure, F(1, 75) ⫽ 2.29, ns, and feedback condition (F ⬍ 1), and the Category Structure ⫻ Feedback Condition interaction (F ⬍ 1) were all nonsignificant. Both two-way interactions with block, Category Structure ⫻ Block: F ⬍ 1; Feedback Condition ⫻ Block: F(3, 225) ⫽ 1.14, ns, and the three-way interaction, F(3, 225) ⫽ 2.37, ns, were nonsignificant.


Summary In this article, we report the results from an experiment that addresses two shortcomings of Maddox et al.’s (2003) original investigation of the effects of delayed feedback on rule-based and information-integration category learning. In line with the results from Maddox et al., delayed feedback had no effect on rule-based category learning but adversely affected information-integration category learning by biasing participants toward the use of hypothesis-testing strategies that are unaffected by delayed feedback.

References Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 716 –723. Alexander, G. E., DeLong, M. R., & Strick, P. L. (1986). Parallel organization of functionally segregated circuits linking basal ganglia and cortex. Annual Review of Neuroscience, 9, 357–381. Arbuthnott, G. W., Ingham, C. A., & Wickens, J. R. (2000). Dopamine and synaptic plasticity in the neostriatum. Journal of Anatomy, 196, 587–596. Ashby, F. G. (1992a). Multidimensional models of categorization. In F. G. Ashby (Ed.), Multidimensional models of perception and cognition (pp. 449 – 483). Hillsdale, NJ: Erlbaum. Ashby, F. G. (1992b). Multivariate probability distributions. In F. G. Ashby (Ed.), Multidimensional models of perception and cognition (pp. 1–34). Hillsdale, NJ: Erlbaum. Ashby, F. G., Alfonso-Reese, L. A., Turken, A. U., & Waldron, E. M. (1998). A neuropsychological theory of multiple systems in category learning. Psychological Review, 105, 442– 481. Ashby, F. G., & Ell, S. W. (2001). The neurobiological basis of category learning. Trends in Cognitive Sciences, 5, 204 –210. Ashby, F. G., Ell, S. W., & Waldron, E. M. (2003). Procedural learning in perceptual categorization. Memory & Cognition, 31, 1114 –1125. Ashby, F. G., & Gott, R. E. (1988). Decision rules in the perception and categorization of multidimensional stimuli. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 33–53. Ashby, F. G., & Maddox, W. T. (in press). Human category learning. Annual Review of Psychology. Ashby, F. G., Maddox, W. T., & Bohil, C. J. (2002). Observational versus feedback training in rule-based and information-integration category learning. Memory & Cognition, 30, 666 – 667. Ashby, F. G., Queller, S., & Berretty, P. T. (1999). On the dominance of unidimensional rules in unsupervised categorization. Perception & Psychophysics, 61, 1178 –1199. Ashby, F. G., & Waldron, E. M. (1999). The nature of implicit categorization. Psychonomic Bulletin & Review, 6, 363–378. Brainard, D. H. (1997). Psychophysics software for use with MATLAB. Spatial Vision, 10, 433– 436. Calabresi, P., Pisani, A., Centonze, D., & Bernardi, G. (1996). Role of Ca2⫹ in striatal LTD and LTP. Seminars in the Neurosciences, 8, 321–328. Erickson, M. A., & Kruschke, J. K. (1998). Rules and exemplars in category learning. Journal of Experimental Psychology: General, 127, 107–140. Gamble, E., & Koch, C. (1987). The dynamics of free calcium in dendritic spines in response to repetitive synaptic input. Science, 236, 1311–1315. Gluck, M. A., Shohamy, D., & Myers, C. (2002). How do people solve the “weather prediction” task? Individual variability in strategies for probabilistic category learning. Learning and Memory, 9, 408 – 418. Hollerman, J. R., & Schultz, W. (1997). Dopamine neurons report an error in the temporal prediction of reward during learning. Nature Neuroscience, 1, 304 –308. Kerr, J. N. D., & Wickens, J. R. (2001). Dopamine D-1/D-5 receptor activation is required for long-term potentiation in the rat neostriatum in vitro. Journal of Neurophysiology, 85, 117–124.

107

Knowlton, B. J., Mangels, J. A., & Squire, L. R. (1996). A neostriatal habit learning system in humans. Science, 273, 1399 –1402. Knowlton, B. J., Squire, L. R. , & Gluck, M. A. (1994). Probabilistic category learning in amnesia. Learning and Memory, 1, 106 –120. MacDermott, A. B., Mayer, M. L., Westbrook, G. L., Smith, S. J., & Barker, J. L. (1986). NMDA-receptor activation increases cytoplasmic calcium concentration in cultured spinal cord neurones. Nature, 321, 519 –522. Maddox, W. T., & Ashby, F. G. (1993). Comparing decision bound and exemplar models of categorization. Perception and Psychophysics, 53, 49 –70. Maddox, W. T., & Ashby, F. G. (2004). Dissociating explicit and procedural-learning based systems of perceptual category learning. Behavioral Processes, 66, 309 –332. Maddox, W. T., Ashby, F. G., & Bohil, C. J. (2003). Delayed feedback effects on rule-based and information-integration category learning. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 650 – 662. Maddox, W. T., Ashby, F. G., Ing, A. D., & Pickering, A. D. (in press) Disrupting feedback processing interferes with rule-based but not information integration category learning. Memory & Cognition. Maddox, W. T., Bohil, C. J., & Ing, A. D. (in press). Evidence for a procedural learning-based system in category learning. Psychonomic Bulletin & Review. Maddox, W. T., Filoteo, J. V., Hejl, K. D., & Ing, A. D. (2004). Category number impact rule-based but not information-integration category learning: Further evidence for dissociable category-learning systems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30, 227–235. Nosofsky, R. M., & Johansen, M. K. (2000). Exemplar-based accounts of “multiple-system” phenomena in perceptual categorization. Psychonomic Bulletin & Review. Pickering, A. D. (1997). New approaches to study of amnesic patients: What can a neurofunctional philosophy and neural network methods offer? Memory, 5, 255–300. Reber, P. J., & Squire, L. R. (1994). Parallel brain systems for learning with and without awareness. Learning & Memory, 1, 217–229. Schultz, W. (1992). Activity of dopamine neurons in the behaving primate. Seminars in Neuroscience, 4, 129 –138. Seger, C. A., & Cincotta, C. M. (2002). Striatal activity in concept learning. Cognitive, Affective, & Behavioral Neuroscience, 2, 149 –161. Shaw, M. L. (1982). Attending to multiple sources of information. I: The integration of information in decision making. Cognitive Psychology, 14, 353–409. Smith, E. E., Patalano, A., & Jonides, J. (1998). Alternative strategies of categorization. Cognition, 65, 167–196. Takane, Y., & Shibayama, T. (1992). Structures in stimulus identification data. In F. G. Ashby (Ed.), Multidimensional models of perception and cognition (pp. 335–362). Hillsdale, NJ: Erlbaum. Toni, N., Buchs, P. A., Nikonenko, I., Bron, C. R., & Muller, D. (1999, November 25) LTP promotes formation of multiple spine synapses between a single axon terminal and a dendrite. Nature, 402, 421– 425. Waldron, E. M., & Ashby, F. G. (2001). The effects of concurrent task interference on category learning. Psychonomic Bulletin & Review, 8, 168 –176. Wickens, T. D. (1982). Models for behavior: Stochastic processes in psychology. San Francisco: Freeman. Wilson, C. J. (1995). The contribution of cortical neurons to the firing pattern of striatal spiny neurons. In J. C. Houk, J. L. Davis, & D. G. Beiser (Eds.), Models of information processing in the basal ganglia (pp. 29 –50). Cambridge, MA: Bradford.

Received December 22, 2003 Revision received July 27, 2004 Accepted July 30, 2004 䡲