Supplementary Information for Tandem Internal Models Execute Motor Learning in the Cerebellum Authors: Takeru Hondaa,b,c,d,1, Soichi Nagaob,c,e, Yuji Hashimotob, Kinya Ishikawab, Takanori Yokotab, Hidehiro Mizusawab,d, Masao Itof,1 Affiliations: a
Motor Disorders Project, Tokyo Metropolitan Institute of Medical Science, 2-1-6 Kamikitazawa, Setagaya-ku, Tokyo, 156-8506, Japan. b
Department of Neurology and Neurological Science, Graduate School, Tokyo Medical and Dental University, 1-5-45, Yushima, Bunkyo-ku, Tokyo 113-8510, Japan. c
Laboratory for Motor Learning Control, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan. d
National Center Hospital, National Center of Neurology and Psychiatry, 4-1-1 Ogawahigashicho, Kodaira, Tokyo 187-8551, Japan. e
Laboratory for Integrative Brain Function, Nozomi Hospital, Kitaadachi-gun, Saitama 362-0806, Japan. f Senior Advisor’s Office, RIKEN Brain Science Institute, 2-1 Hirosawa, Wako, Saitama 351-0198, Japan. 1
Correspondence: Takeru Honda Email:
[email protected] Masao Ito Email:
[email protected]
www.pnas.org/cgi/doi/10.1073/pnas.1716489115
This PDF file includes: Supplementary text SI Text 1 to 5 Figs. S1 to S3 Tables S1 to S3 Supplementary Information Text SI Text 1 Relationship between adaptation and gain of forward and inverse models Control system evaluation reveals that the gain of the inverse model [GI(t, X)] is linearly related to Islow (Fig. S3D and Eq. S24). When the inverse model is impaired so that its gain is no longer updated during prism adaptation, GI(t, X) remains at (X+A)/[(X+P)G] (Fig. S3B), whereas Islow decreases to zero (Fig. S3D). On the other hand, when the forward model is impaired resulting in the gain being no longer updated during prism adaptation, GF(t, X) remains at [(X+P)G]/(X+A), and both Islow and Ifast decrease to zero (Fig. S3E). Note that whereas GI(t, x) is a function of Islow (Eq. S24), GF(t, x) is a function of both Islow and Ifast (Eq. S25). Therefore, we can quantitatively assess the gain of the inverse model alone and that of the inverse and forward models involved in the prism adaptation of hand reaching. The ranges of Eqs. S10 and S15 indicate that abnormal updating of the forward model and normal updating of the inverse model are outside of the blue and gray triangle areas (orange dot in Fig. S3C), predicting that these cases do not exist. This prediction fits with the result showing no cases of low Ifast and high Islow in Fig. 3C, indicating that our model could be proper. SI Text 2 Two types of control system models There are two distinct models that represent the cerebellar contribution to motor learning. The first model is for the adaptive control of reflexes and in this model the controller is placed in the spinal cord and brain stem [for examples of such adaptive reflex control, see Ito (9)]. The other model was originally proposed by Kawato et al. (2) and it applies to a situation where the controller is placed in the motor cortex and controls voluntary movements. The motor cortex acts upon a controlled-object that nests segmental circuits and motor apparatus [see Ito (9)]. There is also biological evidence for the activity of motor cortex neurons related to hand reaching [Inoue et al. (10)]. The tandem model developed in this article is of the second type. SI Text 3 Theoretical analysis of a tandem system of the forward and inverse models We obtained the time constants τfast and τslow by fitting experimental plots to Eq. 1 (Fig. 2C). It is assumed that the positions of touch points in the fast adaptation [yfast(t, x)] and slow adaptation [yslow(t, x)] are represented by Eqs. S1 and S2, respectively, yfast(t, x) = A・exp(− t / τfast) + x [S1] yslow(t, x) = A・exp(− t / τslow) + x, [S2] where A is the average horizontal displacement caused by the prism plate during 10 non-feedback trials (dashed line in Fig. 2C), t is the number of trials, and x is the position
of the target. We define dfast(t) = yfast(t, x) – x and dslow(t) = yslow(t, x) – x as dfast(t) and dslow(t) indicate the horizontal displacements in the fast (black line in Fig. 2C) and slow adaptation (red line in Fig. 2C), respectively. In the hybrid control system scheme in Fig. 4A, the total motor command [Z(t, x)] consists of the motor command generated from the inverse model [ZI(t, x)] and that from the motor cortex [ZM(t, x)]. Representing the transfer function of the controlled object by G, that of the controller by H, the deviation caused by the prism plate by P, and the gain of the forward model by GF(t, x), the final touch point [yfast(t, x)] is given by yfast(t, x) = G Z(t, x) = G[ZI(t, x) + ZM(t, x)] = G ZI(t, x) + G H[(x + P) – GF(t, x) Z(t, x)]. [S3] Likewise, the touch point in the slow adaptation [yslow(t, x)] is given by G ZI(t, x) = G (x + P) GI(t, x), [S4] where GI(t, x) is the gain of the inverse model. Eq. S3 and S4 are combined to give 𝐺! 𝑡, 𝑥 + 𝐻 𝑦!"#$ 𝑡, 𝑥 = 𝐺 𝑥+𝑃 [S5] 𝐻 𝐺! 𝑡, 𝑥 + 1 and 𝑦!"#$ 𝑡, 𝑥 + 𝐻𝐺 𝑥 + 𝑃 𝑦!"#$ 𝑡, 𝑥 = . [S6] 𝐻 𝐺! 𝑡, 𝑥 + 1 Next, using Eqs. S3−5, we calculated GF(t, x) and GI(t, x) as 𝐺! 𝑡, 𝑥 + 𝐻 𝐺 𝑥 + 𝑃 − 𝑦!"#$ 𝑡, 𝑥 𝐺! (𝑡, 𝑥) = [S7] 𝐻 𝑦!"#$ 𝑡, 𝑥 and 𝑦!"#$ 𝑡, 𝑥 𝐺! (𝑡, 𝑥) = . [S8] 𝐺 𝑥+𝑃 From Eqs. S2 and S8, it follows that !
!
A𝑒 !!"#$ + 𝑥 𝐺! (𝑡, 𝑥) = . 𝐺 𝑥+𝑃 !
[S9]
!
Since 0 < 𝑒 !!"#$ < 1, the range of GI(t, x) (Eq. S9) is derived as 𝑥 𝑥+𝐴 < 𝐺! (𝑡, 𝑥) < . 𝐺 (𝑥 + 𝑃) 𝐺 𝑥+𝑃 Under the condition of Eq. S10, 𝑡 𝜏!"#$ = − 𝐺 (𝑡, 𝑥) 𝐺 (𝑥 + 𝑃) − 𝑥 log ! 𝐴 is derived from Eq. S9. From Eqs. S1 and S5, it follows that 𝑡 𝜏!"#$ = − . 𝐺! (𝑡, 𝑥) + 𝐻 𝑥 log{ 𝐺 (𝑥 + 𝑃) − 𝐴} 𝐴 [𝐻 𝐺! (𝑡, 𝑥) + 1] Since 0 < 𝜏!"#$ , 𝐺! (𝑡, 𝑥) + 𝐻 𝑥 0< 𝐺 (𝑥 + 𝑃) − < 1. 𝐴 [𝐻 𝐺! (𝑡, 𝑥) + 1] 𝐴
[S10]
[S11]
[S12]
[S13]
Additionally, since 𝜏!"#$ ≤ 𝜏!"#$ , Eqs. S11 and S12 in combination lead to 1 ≤ 𝐺! (𝑡, 𝑥). [S14] 𝐺! (𝑡, 𝑥) Eventually, we obtain the range of GF(t, x) from Eqs. S13 and S14: 1 [𝐺! (𝑡, 𝑥) + 𝐻] 𝐺 (𝑥 + 𝑃) − 𝑥 ≤ 𝐺! 𝑡, 𝑥 < . [S15] 𝐺! 𝑡, 𝑥 𝐻𝑥 The ranges of Eqs. S10 and S15 can be visually observed in Fig. S3C. When x=X, the functions of GI(t, X) and GF(t, X) depend on only t. It is interesting to see in Fig. S3A that when t approaches a sufficient large T, GI(t, X) and GF(t, X) converge to X/[(X+P)G] and [(X+P)G]/X, respectively. Therefore, GI(T, X) GF(T, X) = 1. This means that at T, the gain of the forward model [GF(T, X)] and the gain of the inverse model [GI(T, X)] are reciprocally related. When GI(t, X) remains at (X+A)/[(X+P)G], GF(t, X) converges to [(X+P)G]/X + A/(XH), as shown in Fig. S3B. In conclusion, we provide these formulations on the basis of the tandem model as shown above. By using A, τfast, and τslow from our experimental data, the formulations estimate the gains of the forward GF(t, x) and inverse GI(t, x) models, suggesting that updating these gains may show plasticity of the synapses between parallel fibers and Purkinje cells in the cerebellum [Ito (12)]. Thus, the formulations are not merely theoretical but have empirical support on the basis of the proposed tandem model. SI Text 4 Analyses for the strength of the fast and slow adaptations We also calculated the strength of the fast adaptation [Sfast(t)] and slow adaptation [Sslow(t)] defined in the main text (Fig. 5A) as (𝑥 + 𝐴) − 𝑦!"#$ (𝑡, 𝑥) 𝑆!"#$ 𝑡 = ×100, [S16] 𝐴 (𝑥 + 𝐴) − 𝑦!"#$ (𝑡, 𝑥) 𝑆!"#$ 𝑡 = ×100 [S17] 𝐴 and 𝑦!"#$ (𝑡, 𝑥) − 𝑦!"#$ (𝑡, 𝑥) 𝑆!"## 𝑡 = 𝑆!"#$ 𝑡 − 𝑆!"!" 𝑡 = ×100. [S18] 𝐴 Eqs. S6 and S8 lead to the expressions (𝑥 + 𝐴) − 𝐺(𝑥 + 𝑃)𝐺! (𝑡, 𝑥) 𝑆!"#$ 𝑡 = ×100 [S19] 𝐴 and 𝐻 𝑦!"#$ (𝑡, 𝑥)𝐺! (𝑡, 𝑥) − 𝐻 𝐺(𝑥 + 𝑃) 𝑆!"## 𝑡 = ×100. [S20] 𝐴 Whereas Sslow(t) (the strength of slow adaptation) is related to the inverse model (Eq. S19), Sdiff(t) is related to the forward model (Eq. S20). Therefore, the ratio of Sdiff(t) to Sfast(t) given by [1 − Sslow(t) / Sfast(t)]×100 represents the relative gain of the forward model to the inverse model (Fig. 5B). A difficulty arises from the fact that the inverse plant is a one-to-many mapping [Schenck (27)] meaning that the inverse model has an infinite number of solutions and some constraints must be imposed [Poggio and Girosi (26)]. The forward model copying the controlled objects is a many-to-one mapping, indicating that the forward model is a constraint. Therefore, the forward model has a finite number of solutions. In our model, after updating the forward model, the inverse model is updated (Fig. 5). This implies that
the one-to-many problem is solved by updating the forward model before the inverse one. SI Text 5 Clinical indices for fast and slow adaptation In the main text, Ifast and Islow are defined as [1 – (Db/Da)] and [1 – (Dc/Da)], respectively, where Da, Db, and Dc represent the mean values of horizontal displacements during periods a, b, and c, measured at trials, ta, tb, and tc, respectively. In this study, we assume tb and tc to be 110 (Fig. 3A and B). It is important to understand that Db equals Da・exp(−tb /τfast) for the fast adaptation at the tbth trial and Dc equals Da・exp(−tc /τslow) for the slow adaptation measured at the tcth trial. Therefore, Islow is equated to [1 − Da・exp(−tc /τslow)/Da]. It follows that 𝑡! 𝜏!"#$ = − . [S21] log(1 − 𝐼!"#$ ) Likewise, Ifast is equated to [1 − Da・exp(−tb /τfast)/Da]. It follows that 𝑡! 𝜏!"#$ = − . [S22] log 1 − 𝐼!"#$ Since 𝜏!"#$ ≤ 𝜏!"#$ , 0≤ Islow ≤ Ifast ≤1. [S23] Eq. S23 covers the triangular area in Fig. 3C. Next, we calculated the relationship between internal models and these indices, as shown in Fig. S3D and E. Combining Eqs. S9 and S21, we obtain −𝐴 𝐼!"#$ + 𝐴 + 𝑋 𝐺! 110, 𝑋 = , [S24] 𝐺 𝑋+𝑃 as shown in Fig. S3D. Likewise, from Eqs. S1, S2, S6, S8, S21, and S22, we obtain 𝐴 (𝐼!"#$ − 𝐼!"#$ ) + 𝐻 𝐺 (𝑋 + 𝑃) 𝐺! (110, 𝑋) = . [S25] −𝐻 𝐴 𝐼!"#$ + 𝐻 (𝑋 + 𝐴) as mapped in Fig. S3E. In the present simulation, we assumed X=500, P=25, A=10, G=1, and H=1.005. We obtained τfast =15.73 and τfast=64.47 from the fitting curves in Fig. 2C.
Fig. S1. Model of hand-reaching control system. Block diagram of theoretical hand-reaching control system. (A) Forward model mediating internal feedback [Ito (1)]. (B) Inverse model mediating feedback-error-dependent learning [Kawato et al. (2)]. (C) Tandem system with forward and inverse models.
A Input Instruction
Controller
Controlled Objects
Output Response
Controlled Objects
Output Response
Controlled Objects
Output Response
Forward Model
B Inverse Model Input Instruction
Controller
C Inverse Model Input Instruction
Controller
Forward Model
Fig. S2. Adaptation curves for healthy and cerebellar patient groups. (A) Average of five healthy subjects (Ctr 1−Ctr 5 in Table S1). (B) Single healthy subject (Ctr 4). (C) Single case (Dis 9) in patient group (Table S2). (D) Similar to c but for Dis 1 (Table S2). (E) Single case (Ctr 9) in alternation of AOF (black dots, 10 trials, fast prism adaptation) and NF (red dots, 5 trials, slow prism adaptation) tasks. Black and red dots indicate AOF and NF conditions, respectively.
C
15 10 5 0 -5 1
20
40
60
80
100
120
Trials Dis Dis66
PRISM
15 10 5 0 -5
0
20
40
60
80
100
120
Trials
E
Horizontal displacement (degree)
B Horizontal displacement (degree)
PRISM PRISM
Ctr Ctr 99
PRISM
15 10 5 0 -5 0
50
100 Trials
150
200
D
Horizontal displacement (degree)
Horizontal displacement (degree)
Healthy Subjects (n=5)
Horizontal displacement (degree)
A
Ctr4 Ctr 4
PRISM PRISM
15 10 5 0 -5
1
20
40
60
80
100
120
Trials Dis 1 PRISM PRISM
15 10 5 0 -5
1
20
40
60 Trials
80
100
120
Fig. S3. Adaptive gains for forward and inverse models. (A) The green and red lines show the gains of forward [GF(t, X)] and inverse [GI(t, X)] models, respectively (see Eqs. S6 and S7) when x is X (x = X). (B) Compensation in the forward model. When GI(t, X) remains (X+A)/[G(X+P)], GF(t, X) is over [G(X+P)]/X. The green and red lines indicate GF(t, X) and GI(t, X), respectively. (D) Relationship between Islow and GI when t=110 (see Eq. S24). (C) The relationship between GF(T, X) and GI(T, X). A blue area shows that the forward model is updated over [(X+P)G]/X to compensate for the inverse model. Black, green, red, and orange dots indicate normal or abnormal updating of both the forward and inverse models, compensation of the forward model, and abnormal updating of the forward and normal updating of the inverse model, respectively. (E) Relationship among Islow, Ifast, and GF when t=110 (see Eq. S25).
A
B PRISM
PRISM (X+P)G A + X XH
GF GI
1
X+A (X+P)G X (X+P)G 0
C
(X+P)G X+A
Gain
Gain
(X+P)G X (X+P)G X+A
1 X+A (X+P)G
20
40
60 80 Trials
D
100
120
140
0
E
20
40
60 80 Trials
100
GF GI
1 1
Islow
0.8
GF = 0.6
Islow
0.8 0.6 0.4 I fast 0.2
(X+P)G X+A
0.4
0.2
140
GF = (X+P)G + A X XH
(X+P)G GF = X
X+A (X+P)G
X (X+P)G
120
00
Table S1. Characteristics of healthy control subjects. M, male. F, female. R, right. Identifier
Gender
Age (years)
Handedness
Ctr 1
F
51
R
Ctr 2
F
51
R
Ctr 3
F
62
R
Ctr 4
M
69
R
Ctr 5
M
86
R
Ctr 6
F
27
R
Ctr 7
M
33
R
Ctr 8
M
34
R
Ctr 9
M
35
R
Ctr 10
F
42
R
Group I
Group II
Table S2. Characteristics of patients with cerebellar degeneration. SCA, spinocerebellar ataxia. SARA, Scale for the Assessment and Rating of Ataxia. Identifier
Gender
Age (years)
Handedness
Diagnosis
SARA
Dis1
M
55
R
SCA6
14.5
Dis2
M
66
R
SCA6
11
Dis3
F
72
R
SCA31
21.5
Dis4
F
72
R
SCA31
10.5
Dis5
F
76
R
SCA31
26.5
Dis6
M
55
R
SCA6
6.5
Dis7
M
62
R
SCA6
16.5
Dis9
M
71
R
SCA31
14.5
Dis9
M
73
R
SCA31
10
Dis10
F
81
R
SCA31
13
Table S3. Comparison between participants in age and SARA. S.D., standard deviation. Identifier
The number
Age (years)
SARA
±S.D.
±S.D.
Healthy Subjects in Group I
5
63.8±14.5
0
Dis1−5
5
68.2±8.2
16.8±7.0
Dis6−10
5
68.4±10.1
12.1±3.9
