A Framework for Constructing Animations via Declarative Mapping Rules Shin Takahashi Ken Miyashita Satoshi Matsuoka Akinori Yonezawa3 Department of Information Science, University of Tokyo Abstract Animation is useful in user interfaces, especially for representing dynamic features. However, it is often difficult and laborious to create animation in an existing user interface environment. In this research, we extend our bi-directional translation model to deal with animations. The new model incorporates the notion of time, and includes operations defined for each representation in the model. Using this model, animation is basically achieved by interpolating successive visualized pictures of application data. Other than specifying the way of visualizing application data, programmers have only to provide mapping rules between abstract operations and interpolating methods, and as a result, complex animation effects can be easily specified. We have developed a prototype system based on this model, and applied it to several algorithm animations, such as sorting algorithms, the tower of Hanoi, and Kruskal’s algorithm.
1 Introduction Animation is an effective technique for graphical user interface (GUI). It is useful for showing various changes, processes, and movements of data or execution of an application program, and used in numerous areas. Needless to say, they are widely used in entertainment; video games and cartoons use computer animations effectively. The results of computations in various scientific fields, such as simulations of fluid mechanics, are often visualized and animated. Algorithm animation, which illustrates how a target algorithm works, is an attractive tool for teaching algorithms. Moreover, it is useful for creating and debugging algorithms. Nevertheless, these advantages are not fully utilized in current GUIs, because it takes so much time and cost to create animations, despite many tools available. The primary reason is that the process of creating animations is still procedural, and the programmer has to be concerned with various details of object movements, etc. As the result, although the performance of computing and graphics have improved dramatically, we cannot fully utilize it for visualization and animation in GUIs. 3 E-Mail:ftakahasi,miyasita,matsu,
[email protected]
The purpose of this paper is to decrease the cost of creating animations, and to facilitate using them, by providing a tool for creating animations in a fully declarative framework. However, we do not attempt to cover every conceivable kind of animations in this paper. Our target is abstract animations, that is, we do not intend to deal with animations that illustrate ‘real’ shape of some real objects, nor the movements which are based on certain physical phenomena. For example, animations that represent the result of simulations of aeromechanics, animations of a human body, or animations of dinosours are outside of our scope. To be more precise, we are to treat animations that describe the process of changing abstract data or relations, such as algorithm animations or program visualizations. There are two features of our target animations: 1. The abstract data and the relations to be animated do not have their own intrinsic ‘shape’s. How to visualize and animate them should be specified in some way so that they can easily be understood. This is a stark contrast to animations of real objects, which have their own shapes, and usually images of them should resemble their real shapes. 2. These abstract data are in the application program, and change during its execution. In order to create an animation of application program, we have to obtain the up-to-date status from application program during its execution. We have already proposed the bi-directional translation model[18, 27], which enables programmers to easily create direct manipulation style interfaces for manipulating abstract data. In this model, visualization of application data and recognition of pictures are regarded as translations from/to application data to/from pictorial data via abstract structure representation (ASR) and visual structure representation (VSR). This translation is specified by a declarative mapping rule between ASR and VSR (Visual Mapping Rule). We extend this bi-directional translation model to handle animations. The extended model includes the notion of time, i.e. it incorporates a series of changing data. Animations are basically achieved by interpolating a series of pictures translated from a series of application data. In addition, we define operations on four data representations
in our model. These operations change their data into the next state. The change of pictures — animations — are also regarded as operations on pictures, and how an animation works can be altered by providing a transition mapping rule that maps abstract operation on ASR to transitional operation on VSR. Using this model, only two sets of mapping rules — visual mapping rules and transitional mapping rules — are necessary to create an animation. Complex animation techniques, such as slow-in-slow-out and squash-andstretch[16], are easily specified in our model. We also describe a prototype system based on this extended model. This system uses a similar visual mapping module to TRIP[14, 15], TRIP2[18, 27], and TRIP3[19, 20]. Thus, the same visual mapping rules can be used in our animation system. In particular, using the TRIP3 system, users can easily create visual mapping rules by example for this animation system. The rest of this paper is organized as follows. Section 2 describes the bi-directional translation model, and the extended model for animations. Section 3 explains how the programmer make an animation, and Section 4 describes the implementation details of our system. Section 5 shows two examples of using our prototype system. We conclude in Section 6.
2 The Extended Bi-Directional Translation Model
Abstract Structure Representation(ASR) This represents the underlying abstract structure of application data, which is a set of abstract relations among abstract objects that are extracted from application’s data. Application’s data are analyzed and translated into this universal representation. In the inverse translation process, the original data representation of the application is generated from this representation. In our system, ASR data are represented as a set of Prolog predicates defined by the programmers. Visual Structure Representation(VSR) This represents the underlying structure of a picture, which is a set of graphical relations among graphical objects. In VSR, no absolute coordinates are specified. Similar to ASR, this representation is translated into and from the pictorial representation. In our system, VSR data are also represented as a set of Prolog predicates. In contrast to ASR predicates, which are user-defined, VSR data are special predicates predefined in the system corresponding to system-supplied graphical objects and relations. Pictorial Representation(PR) This is the representation of pictures rendered directly on the display device. Users view and manipulate pictures generated from this data via direct manipulation.
"X consists of p,q,and r"
AR
(1)
2.1 The Bi-Directional Translation Model The original bi-directional translation model is a model of translation between abstract application data and pictorial data. It aims at modeling the two necessary functions for realizing direct manipulation interfaces: visualization of application data, and recognition of the visualized picture. To generalize these translations, we employ two intermediate, universal representations: One is Abstract Structure Representation (ASR), and the other is Visual Structure Representation (VSR). The intermediate representations serve as pivots, so that when either the abstract application data or pictorial representations change, the mapping rules between ASR and VSR (visual mapping rules) are essentially unaffected. Altogether, the translation process handles the following four data representations:
(6) consists_of(x,[p,q,r]). object(x). object(q). object(p). object(r). Visual Mapping Rules
(2)
ASR
(5)
above([x],[p,q,r],Ygap,[]). hor−listing([p,q,r],Xgap,[]). mconnect([x],[p,q,r],bottom, VSR top,[solid,orthogonal]). boxwithlabel(x,Width,Height,x, [visible]). (4) (3)
CCCCCCCC CCCCCCCC x CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC q p r CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCC CCCCCCCCCCCCCCCC
PR
Figure 1: An Example of the Model
Application’s Data Representation(AR) This is the application-specific data representation. It may thus take any form of representation. They are, for example, natural languages, program lists, and form data for a database.
Figure 1 illustrates an example of the four types of representations and their relationships: The application’s data is represented as a natural language sentence “x consists of p, q, and r”. ASR and VSR data are repre-
sented as a set of Prolog predicates1 . The ASR data corresponding to the above English sentence are consists of(x, [p, q, r]), and object(x), object(p), etc., where consists of() is an abstract relation, and object()s are abstract objects, which are defined by the application programmer. The corresponding VSR data are above([x], [p, q, r], Ygap, [rigid]), hor-listing([p, q, r], Xgap, [rigid]), mconnect([x],[p,q,r],...), boxwithlabel(x, Width, Height, x, [visible]), etc2 . Above(), hor-listing(), and mconnect() are graphical relations, and the boxwithlabel()s are graphical objects. These predicates are predefined in the system, and its parameters represent their attributes. The first parameter of a graphical object is its identifier (Constants, lists, and predicates can be used as identifiers.); graphical relations use these identifiers to designate their constituent graphical objects. As a pictorial representation, a picture of a tree is visualized in Figure 1. In our model, visualization of application data corresponds to the translation from AR into PR, that is, Figure 1 (1), Figure 1 (2), and Figure 1 (3). Inversely, recognition of a picture corresponds to the translation from PR into AR, that is, Figure 1 (4), Figure 1 (5), and Figure 1 (6). The essential parts of these translations are Figure 1 (2) and (5), and the programmer has only to provide visual mapping rules to achieve these translations. More details are described in [18, 27].
2.2 The Extension of the Model for Animation As mentioned before, we are to animate the behavior of application programs or algorithms. They embody some abstract data within their computations, and change them during its execution. Thus, an execution of an application program can be regarded as a sequence of changing abstract data and operations. When an operation in an application is executed, application’s data is changed, and goes to the next state in that sequence. At the same time, an animation can also be viewed as a sequence of changing pictures corresponding to a series of abstract data and operations. When an application executes its operation, a picture corresponding to application’s data is animated to the next picture corresponding to the next application data in a way corresponding to the executed operation. According to this idea, we extend the bi-directional translation model to be composed of a series of data on each representation in the model, and incorporate four types of 1 Predicates have the form pred name(p1,p2,: : : ,pn). Pns could be ground literals such as integers and strings, structures, or variables (capitalized). Square brackets (‘[’ and ‘]’) are used to delimit lists. 2 The VSR data here denotes the following pictorial structure: Object x is placed above objects p, q, and r. Objects p, q, and r are horizontally arranged. Object x is connected to objects p, q, r by solid orthogonal lines. Object x is a box with label ‘x’.
‘operations’ to each representation in the model (Figure 2). The horizontal sequences in Figure 2 are the histories of changing data and operations on each representation. The vertical sequences correspond to the processes of translating application data to a picture via ASR and VSR. AR(t)
application’s operation (c)
ASR(t)
VSR(t)
AR(t+2)
ASR(t+1)
ASR(t+2)
execution
abstract operation (b)
AR(t+1)
transition mapping
transitional operation
(a)
visual mapping
VSR(t+1)
VSR(t+2)
PR(t+1)
PR(t+2)
generating animation PR(t)
animation (operation on PR)
AR : application data ASR : abstract structure representation
VSR : visual structure representation PR : picture data
: translation : operation
Figure 2: The Extended Model
2.3
Specifying an Animation Using the Extended Model
Using the extended model, we can create an animation by (1) obtaining application’s internal data, (2) visualizing the application’s data, and (3) generating intermediate pictures between pictures visualized from application’s data. Since the visualization from application data into a picture (i.e. (2)) is achieved only by specifying visual mapping rules as described in Section 2.1, it is possible to specify an animation only by visual mapping rules. However, since there are infinite number of ways to generate intermediate pictures between two pictures, we must provide a method of specifying how to change a picture gradually to the next picture. The method of generating intermediate pictures can be viewed as the way of moving, scaling, and rotating the objects in one picture to the places/sizes/positions of the objects in another picture, which corresponds to operations that change a set of VSR data to another set of VSR data in our model. We call these operations transitional operations. Specifying the way of generating intermediate pictures corresponds to determining transitional operations in our model. The specifications of transitional operations are also based on mapping in our model. Programmers specify transitional operations by providing mapping rules between Abstract Operations, which are the operations on ASR, and Transitional Operations, which are the operations on VSR. We call these rules transition mapping rules. When an abstract operation that appears in a transition mapping rule
is executed, transitional operations mapped from the executed abstract operation are selected and used for creating an animation. The process of translating application’s operations into animation is illustrated in Figure 2. Firstly, an application executes its operation on AR, and also invokes an abstract operation. Since it is almost impossible to automatically recognize meaningful abstract operations in an application program, programmers themselves must define and insert the codes to invoke abstract operations in the program. Abstract operation corresponds to interesting events in BALSA-II[4], or Zeus[5]. The executed abstract operation generates the next ASR data corresponding to changed application data. They are translated into the next VSR and PR by visual mapping. Then a transition mapping is invoked, with some graphical objects in VSR being associated with transitional operations specified by the mapping rule. Other objects are associated with default transitional operations. Finally, intermediate pictures that smoothly connect the previous and the new picture is generated according to the transitional operations. These pictures are displayed consecutively, which serve as a fragment of an animation that represents the application operation.
3 How to construct an Animation — Insertion Sort Example To create an animation of an algorithm or an application, programmers have to write the followings: Visual mapping rules Visual mapping rules are used for translating ASR into VSR. They specify how to visualize application data, that is, specify how to layout graphical objects in the target animation. (Figure 2(a)) Transition mapping rules Transition mapping rules are used for mapping abstract operations to transitional operations. They specify how to move graphical objects in an animation. (Figure 2(b)) Annotations Annotations output abstract operations when an application executes its corresponding operations. They are used for extracting information from the executed application program. They can be regarded as specifications of translations from application’s operations into abstract operations. (Figure 2(c)) For example, to create an animation of insertion sort algorithm, a programmer must write an insertion sort program, and annotate it so that the information on the execution of that program is obtained. Figure 3 shows an insertion sort program written in Prolog. Basically, annotations are
% For demo go :- tell(’sort.anim’), % Write Initial State write_asr([5,2,6,7,1,9,4,8,3],[]), write(’%’),nl, % Start Sorting sort([3,8,4,9,1,7,6,2,5], L), told. % Sorting Program – Insertion Sort sort(I, O) :- sort1([], I, O). sort1(_, [], []). sort1(L, [IH|IT], O) :% Sort IT sort1([IH|L], IT, TMP), % and Insert a number to the sorted numbers insert(IH, TMP, O), % Output ASR data and Abstract Operations write_asr(L, O), write_ao(IH). % Insert N to the sorted list insert(N, [], [N]). insert(N, [F|S], [N, F|S]) :- N =< F. insert(N, [F|S], [F|L]) :- insert(N, S, L). % Program for output ASR and AO write_asr(L1, L2) :reverse(L1, L3), write(numlist(L3,L2)), write(’.’), nl. write_ao(O) :write(insert(O)), write(’.’), nl, write(’%’), nl.
Figure 3: Annotated Insertion Sort Program inserted at points that the application executes an interesting algorithm operation. In this example, after performing insert/3, write asr/2 outputs ASR data such as: numlist([3,8,4,9],[1,2,5,6,7]). The first argument of numlist represents numbers to be inserted, and the second argument represents already sorted (= inserted) numbers. At the same time, the predicate write ao/1 outputs an abstract operation insert(N) which means that the program inserted the number N. The character % output just after insert/1 separates these data from the data generated at the next execution of insert/1. These output are logged onto a file (Figure 4), and used for
numlist([3,8,4,9,1,7,6,2,5], []). % insert(5). numlist([3,8,4,9,1,7,6,2], [5]). % insert(2). numlist([3,8,4,9,1,7,6], [2,5]). % insert(6). numlist([3,8,4,9,1,7], [2,5,6]). % ..... omitted ..... % insert(3). numlist([],[1,2,3,4,5,6,7,8,9]). %
Figure 4: A Log File of the Insertion Sort Program
% Mapping Numbers to Graphical Objects objectmap([o]). o :- numlist(L1, L2), % Get numlist/2 num2barmap(L1), % Map numbers to boxes num2barmap(L2). % Map numbers to boxes num2barmap([]). num2barmap([B|L]) :Y is B * 20, % a box with width=30 and height=Y box(B, 30, Y, [shaded]), num2barmap(L). relationmap([r]). r :- numlist([H|L1], L2), % Get numlist/2 do_layout([H|L1], L2),% Layout numbers place(H,80,100,[]). % Layout numbers horizontally do_layout(L1, []) :horizontallisting(L1, 12, [bottom_align]). do_layout([], L2) :horizontallisting(L2, 12, [bottom_align]). do_layout(L1, L2) :horizontallisting(L1, 12, [bottom_align]), horizontallisting(L2, 12, [bottom_align]), tail(L1, T), L2 = [H|_], horizontallisting([T, H], 50, [bottom_align]).
Figure 5: A Visual Mapping rule for Insertion Sort Animation
creating an animation of this execution. Then, the programmer writes the visual mapping rule in Figure 5 to visualize the ASR data numlist(). There are two parts in this rule. In the first part, the predicate ‘o’ translates all the numbers in numlist/2 into box/4 using num2barmap/1, and in the second part, the predicate ‘r’ translates numlist/2 into three horizontallisting/3 which arranges boxes horizontally with appropriate intervals. Lastly, the following transition mapping rule like the following is provided: transitionmap([t]). % insert(X) -> move(X,[clockwise]) t :- insert(X), % get insert() move(X, [clockwise]). % output move()
This rule specifies that the inserted object (X) should move clockwise. Figure 6 shows the resulting animation created with the above mapping rules3 . Note that not only the inserting bar is moving, but also the two other bars are being moved as well to make room for the inserting bar. The programmer did not describe such movements, but the system automatically created the transitions for the movements of the two bars. (See Section 4.1) 3 Our system uses color, but all snapshots of the system in this paper are converted to the gray-scale pictures. Also, some figures in this paper show trails of objects, which are specially displayed for illustrating an animation on papers, and actually are not presented in the real system.
Figure 6: An Animation of an Insertion Sort Algorithm
4
Implementation of TRIP2a
We have implemented a prototype system based on the extended bi-directional translation model. It is integrated with TRIP2[18, 27] and shares the TRIP module and the interaction module. This system is written in Objective-C, and created on NextStep using Application Kit, and Interface Builders. This section describes how we generate intermediate pictures between pictures generated from application’s data, what kind of transitional operations (including various animation effects) we have built, and how we obtain data from a running application program.
4.1
Implementation of Animations
As mentioned in Section 2.1, a picture is represented as a set of picture objects in Objective-C. Therefore, an animation is a set of transitions from the objects in one picture to the objects in another picture. An animation from picture A to picture B is created in the following manner: 1. Find correspondences between the objects in picture A and the objects in picture B, and examine how the object in picture A is changed to the corresponding object in picture B. (Figure 7(1)) 2. Make instance objects of Transition class with above pairs of corresponding two objects, which will change object in A into object in B. (Figure 7(2)) If the corresponding object is not found, the system make a transition from/to the object to/from a null object (an object whose size is zero). The correspondences of objects in two pictures are searched by using the name of objects. For example, if objX in picture A and objY in picture B have the same name, it is assumed that they are the same object, and that objX is moved and/or resized to objY. This means that in order to animate an object, the programmer must maintain its name in a sequence of pictures, i.e., he/she has to name objects in a consistent way. The names of objects are specified in visual mapping rules. Other animation systems often need explicit specifications of transformations of graphical objects in an animation.
A (1)
B a
b
b c
(2)
c
Transition
a b
a
c
Figure 7: Making an Animation For example, to generate animations using Pavane[21, 9], the programmer has to write a rule that maps data in old and current state of a program into graphical objects which change as time goes by. In our system, the transformations of graphical objects are automatically detected, and programmers do not need to specify it. Programmers have only to write a rule to layout abstract data, and alter the way of transformations if the default way is not sufficient. One interesting feature of our method of creating animations is that the structure of moving sub-picture is often preserved automatically. For example, in Figure 8 which presents an animation of a list structure, the layout of the sub-structure ((b) c d) is maintained during its movement4 . This is because the transitions of these cells and arrows are created using the same (straight) transitional operation, and the starting and the ending layout of the sub-tree is the same. This kind of the movement would be cumbersome to specify if the programmer had to specify movements of each and every graphical objects in an animation.
Figure 8: A Cons-cell Animation
operations on ASR into transitional operations on VSR. These transitional operations are attached to each graphical object in the picture translated from ASR, and used for making transitions of objects. There are several ways to specify transitional operations: 1. No specification. When the programmer does not specify transition mapping rules, transitional operations for the graphical objects are determined automatically. For example, if a movement of a graphical object is not specified by the programmer, the transitional operations that move the object straight and that scale linearly are used by default. Therefore, at the beginning of constructing an animation, the programmer does not necessarily provide a transition mapping rule. He/she can execute an animation without transition mapping rules, and later, may improve it by specifying transitional operations. 2. Using predefined transitional operations. Following operations are provided: move specifies how to move a graphical object. Following operations are provided: straight A straight path is created. The graphical object that straight is attached to moves straightly from the starting location to the ending location. This operation is used by default when no transitional operation is assigned to a moving object. clockwise,counterclockwise A path that goes clockwise/counterclockwise to the destination is created. lazy,immediate The lazy operation keeps an object from moving until the end of this transition. The immediate operation moves an object immediately at the beginning of this transition, to the final position and the size. up,down,left,right These operations specify tangent vectors at the start and the end of a movement of a graphical object. rotate specifies how to rotate an object during movement. The direction and the number of rotations can be changed. scale specifies how to scale an object. The immediate, lazy, and so forth are provided.
4.2 Specifying Transitional Operations
color specifies how to change the color of an object.
In our system, the motions of graphical objects are specified with transition mapping rules which translate abstract
Furthermore, several decorative animation effects are provided. The techniques of cartoon animation[16] are said to be important also in graphical user interfaces[8]. We provide the followings:
4 Here,
the list (e a (b) c d) is changed to (((b) c d) a)
Sending abstract operations and data by using ToTripSpeaker class which is a subclass of Speaker class in ApplicationKit of NextStep. In this case, the execution of the application and its animation runs concurrently. However, the application program must be written in Objective-C.
blink Blinking. An object blinks during the movement, or blinks before and after its movement. shake Shaking. An object trembles before moving so that the start of its movement is easily recognized. slow-in-and-slow-out This operation slows down a graphical object near the start and the end of its movement, which makes the movement more natural. squash-and-stretch This operation squashes or stretches a graphical object as it moves. This effect can depict rigidity of an object, and makes an object more vivid. Figure 9 shows the animation of the tower of Hanoi. This is an example of using slow-in-and-slow-out and squash-and-stretch. The moving disk in the figure is specified the following simple transitional operations: move(X, [up,squash-and-stretch]). move(X, [slow-in-and-slow-out]). The disk moves upward at the start. The movement of the disk is slowed down near the start and the end, and the disk is stretched as it moves5 .
Figure 9: Slow-In-Slow-Out and Squash-and-Stretch 3. Defining transitional operations. The programmer can define transitional operations by writing functions that describe the motion of a graphical object. We are using a fast constraint solver for evaluating these functions.
4.3 Application Interface As mentioned above, the layout and the movement of the graphical object is determined only by specifying two mapping rules — a visual mapping rule and a transitional mapping rule. However, to obtain information from the running application, some programs must be inserted to the target application program. For the present, there are three ways to do this:
Writing the operations and the data in the application to a log file as ASRs and AOs. This way can be used for almost all programming language, since it only uses file operations. 5 Here,
the object is stretched in proportion to its acceleration.
Inserting the predicate intrEvent in the application program on ASR6 . All of the above methods require modifications to the application program, which is obviously undesirable. We are planning to use the technique employed in debuggers to obtain operations and data in the application without modifications of the programs.
5 5.1
Examples Heap Sort
Heap sort sorts numbers by finding the maximum number by making a heap structure. We visualize these heap structures as binary trees, and animated this algorithm. Figure 10(a) describes the exchange of the number at the root and the number at the end of the heap structure, which removes the number at the root (= maximum number). The color of the removed number is changed (Figure 10(b)). Figure 10(b)(c) shows the process of restructuring the heap. In order to help identify the movement of removing the max number and the movement of restructuring the heap structure, we use the clockwise movement for removing the max number. Since the heap structure is visualized as a binary tree, a mapping rule for visualizing a tree structure is used for this animation. In the same way, a number of animations can be constructed by modifying mapping rules for ordinary data structures, such as a graph structure and a tree structure. A number of mapping rules for various data structures are already available.
5.2
Finding a Minimum Spanning Tree (MST)
We have animated Kruskal’s algorithm, which finds a minimum spanning tree from a given undirected graph. In this example, the user inputs the graph by drawing it with MacDraw-like graphical editor (Figure 11(a)). The drawn graph is parsed, and inversely translated into the abstract representation of that graph, and then used as the input for the MST program. (See Section 2.1) After that, TRIP2a beautifies the drawn graph by visualizing the abstract representation of the input graph, and then 6 ASR is implemented as clauses in Prolog. In order to use this method, the program has to be written in Prolog.
(a) (a)
(b)
(b)
(c) Figure 10: Heap Sort
starts the program. Firstly, all edges are removed from the graph, and sorted by their weight. Then edges are inserted to the graph from the lighter one if it does not form any cycles. Figure 11(c) shows the process of sorting, and Figure 11(d) represents an insertion of an edge. Since TRIP2a has a special constraint solver for graph layout, only the data of nodes and edges are necessary for visualizing general undirected-graphs like one the used in this animation.
6 Related Work In the field of algorithm animation, the most famous and practical system is BALSA and BALSA-II[4]. It is applied to many algorithms, such as sorting algorithms, bin-packing algorithms, and all sorts of graph algorithms, and has been used by numerous students and researchers. Interesting events, which informs the animation system that an ‘interesting’ operation is executed, was introduced by the BALSA system. Zeus[5, 6] is a successor of BALSA II, which is implemented on Modula-3. However, neither BALSA-II nor Zeus do not directly support the creation of graphics part of animations in a declarative way. TANGO[23, 24, 22] is based on a Path-Transition paradigm. In this model, there are four data types: location, image, path, and transition. Basically, an animation consists of transitions which is a pair of an image and a path. An image is an object drawn on a screen, and a path describes a series of locations. TANGO also uses interesting events for calling these animations. The path can be generated by user’s demonstration using the Dance system[25]. However, we believe that it would be more laborious to specify com-
(c)
(d)
Figure 11: Minimum Spanning Tree plex animations involving compound data structures with TANGO compared to our system. Polka[26] is a successor of TANGO which enables to create 3D animations of concurrent programs. Pavane[9, 21] is a declarative animation system. It visualizes programs written in the Swarm language, which is based on a state-transition model. Animation is specified by declarative mapping from State space, which is a set of tuples that represents the data of a program to be visualized, to Animation space, which contains graphical objects with time-dependent attributes. The difference with our model is that our way of specifying an animation is more abstract, and thus more concise. For example, by using our system, the programmer can specify the motions of objects including animation effects by simple transitional operations. Also the layout of picture can be easily specified by using graphical constraints in our system, which also makes it easy to specify an animation. Animus[10] uses temporal constraints for creating animations. They are specified in the same manner with ThingLab[3] and ThingLabII[17]. In Animus, the appearance and structure of a picture as well as how those pictures evolve in time are described by constraints. Dynamic mechanical and electrical circuit simulations, problems in operating systems, and geometric curve drawing algorithms, are animated using Animus in [10]. Pictorial Janus[12, 13] is a visual representation of Janus — a general-purpose concurrent constraint programming
language. It is complete, which means that pictures are programs and also represents snapshots of computation, and it completely corresponds to the text version of Janus. In addition, the execution of program (pictures) can be animated. Bentley and Kernighan developed ANIM[1, 2]. It aims easy creation of animations. An animation is made from script file, which is output from the target application program, that contains commands for drawing an image, such as box and line. There are some systems which incorporate the techniques of cartoon animations[16]. The user inteface of Self[7] uses such techniques for the movements of windows, and Artkit[11] provides animation abstraction, which also supports cartooning techniques.
7 Conclusions We have proposed the extended bi-directional translation model for animations, and have implemented a prototype system based on this model. Animations can be created by providing visual and transitional mapping rules, and by annotating an application program so that the data and the operation of the application are passed to the animation system. We have applied it to several algorithms, such as sorting algorithms, the tower of Hanoi, and Kruskal’s algorithm for finding a minimum spanning tree. Experience shows that animations can be created quite effortlessly. In future work, we plan to support animating concurrent programs, and to provide a way to specify animations demonstrationally using the techniques of TRIP3[19, 20].
References [1] Jon L. Bentley and Brian W. Kernighan. A system for algorithm animation: Tutorial and user manual. Computing Science Technical Report 132, AT&T Bell Laboratories, January 1987. [2] Jon L. Bentley and Brian W. Kernighan. A system for algorithm animation. In Computing Systems, volume 4, pages 5–30. USENIX, winter 1991. [3] Alan Borning and Robert Duisberg. Constraint-based tools for building user interfaces. ACM Transactions on Graphics, 5(4):345–374, October 1986. [4] Marc H. Brown. Algorithm Animation. MIT Press, Massachusetts, 1988. [5] Marc H. Brown. Zeus: A system for algorithm animation and multiview editing. In Proceedings 1991 IEEE Workshop on Visual Languages, pages 4–9, October 1991. [6] Marc H. Brown and John Hershberger. Color and sound in algorithm animation. IEEE Computer, pages 52–63, December 1992. [7] Bay-Wei Chang and David Ungar. Animation: From cartoons to the user interface. In Proceedings of the ACM Symposium on User Interface Software and Technology, pages 45–56, November 1993. [8] Chuck Clanton, Jock Mackinlay, Dave Ungar, and Emilie Young. Animation of user interfaces, November 1992. Panel at UIST’92.
[9] Kenneth C. Cox and Gruia-Catalin Roman. Visualizing concurrent computations. In Proceedings 1991 IEEE Workshop on Visual Languages, pages 18–24, 1991. [10] Rovert Adamy Duisberg. Animation using temporal constraints: An overview of the animus system. Human-Computer Interaction, 3(3):275–307, 1987/88. [11] Scott E. Hudson and John T. Stasko. Animation support in a user interface toolkit: Flexible, robust, and reusable abstractions. In Proceedings of the ACM Symposium on User Interface Software and Technology, pages 57–68, November 1993. [12] Kenneth M. Kahn. Concurrent constraint programs to parse and animate pictures of concurrent constraint programs. In Proceedings of the International Conference on Fifth Generation Computer Systems, pages 943–950. ICOT, 1992. [13] Kenneth M. Kahn and Vijay A. Saraswat. Complete visualizations of concurrent programs and their executions. In Proceedings of the 1990 IEEE Workshop on Visual Languages, pages 7–15, October 1990. [14] T. Kamada. Visualizing Abstract Objects and Relations, A ConstraintBased Approach. World Scientific, Singapore, 1989. [15] Tomihisa Kamada and Satoru Kawai. A General Framework for Visualizing Abstract Objects and Relations. ACM Transactions on Graphics, 10(1):1–39, Jan. 1991. [16] John Lasseter. Principles of traditional animation applied to 3D computer animation. ACM Computer Graphics, 21(4):35–44, July 1987. [17] John Maloney, Alan Borning, and Bjorn N. Freeman-Benson. Constraint technology for user-interface in ThingLabII. In OOPSLA ’89 Conference Proceedings, pages 381–388, 1989. [18] Satoshi Matsuoka, Shin Takahashi, Tomihisa Kamada, and Akinori Yonezawa. A General Framework for Bi-directional Translation between Abstract and Pictorial Data. ACM Transactions on Information Systems, 10(4):408–437, October 1992. [19] Ken Miyashita, Satoshi Matsuoka, Shin Takahashi, and Akinori Yonezawa. Declarative Programming of Graphical Interfaces by Visual Examples. In Proceedings of the ACM Symposium on User Interface Software and Technology (UIST’92), volume 5, pages 107– 116, November 1992. [20] Ken Miyashita, Satoshi Matsuoka, Shin Takahashi, and Akinori Yonezawa. Interactive generation of graphical user interfaces by multiple visual examples. In Proceedings of the ACM Symposium on User Interface Software and Technology, 1994 (To appear). [21] G. C. Roman and K. C. Cox. A declarative approach to visualizing concurrent commputations. IEEE Computer, 22(10):25–36, October 1989. [22] John T. Stasko. The path-transition paradigm: a practical methodology for adding animation to program interfaces. Journal of Visual Languages and Computing, 1(3):213–236, 1990. [23] John T. Stasko. Simplifying algorithm animation with tango. In Proceedings of the 1990 IEEE Workshop on Visual Languages, pages 1–6, October 1990. [24] John T. Stasko. Tango: A framework and system for algorithm animation. IEEE Computer, 23(9):27–39, September 1990. [25] John T. Stasko. Using direct manipulation to build algorithm animations by demonstration. In ACM Human Factors in Computing Systems, CHI’91 Conference Proceedings, pages 307–314, 1991. [26] John T. Stasko and Joseph F. Wehrli. Three-dimensional computation visualization. In Proceedings of the 1993 IEEE Symposium on Visual Languages, pages 100–107. IEEE, 1993. [27] Shin Takahashi, Satoshi Matsuoka, Akinori Yonezawa, and Tomihisa Kamada. A General Framework for Bidirectional Translation between Abstract and Pictorial Data. In Proceedings of the ACM Symposium on User Interface Software and Technology, volume 4, pages 165–174, November 1991.
