fish: a virtual art installation

I NVENCAO AUG 1999

FISH: A VIRTUAL ART INSTALLATION

FISH: A VIRTUAL ART INSTALLATION I SAAC R UDOMIN , P H .D.

( ITESM-CEM)

D ANIEL R IVERA -S TERLING ( DGSCA-UNAM)

ABSTRACT:

An art Installation, “Fish”, and the research project behind it are explored. The issues raised by the construction of partitioned, shared VR applications populated with autonomous creatures endowed with synthetic vision are analyzed.. INTRODUCTION:

A-Life is a new research paradigm that studies life by recreating or imitating biological phenomena within the context of computer simulation. In contrast with biology, which takes apart living organisms to understand how they work, this synthetic discipline builds artificial systems that have some of the characteristics of living systems, such as autonomy, evolution, self-reproduction and auto-replication. Art has a similar agenda. It has, for a long time, intended to represent and/or recreate nature. A-Life can thus be a tool for artists, giving them new avenues to explore in a creative manner. In this paper we present a virtual art installation called “Fish”, as well as the research project on which this installation is based, and discuss technical and other issues raised by the project. The research project behind “Fish” involves the construction of virtual worlds with the following characteristics: 1. The virtual world is populated by "intelligent" artificial creatures, in the manner of research in artificial life, and consists of autonomous agents with sensors, effectors, and some kind of “intelligent” decision engine. Initially, we chose fish for their simplicity. 2. The creatures are endowed with synthetic vision that can be used to sense the virtual environment. 3. Computer vision is also used as a form of user interface. 4. The virtual world is multiuser and can be accessed via the Internet. All the participants can see the same scene simultaneously but from their own point of view. 5. The virtual world is too large to fit in any one computer, so it is partitioned and distributed to different computers.

1

I NVENCAO AUG 1999


This has important technical implications in vision, for example, since constructing a view of the world requires transferring partial views from neighboring computers and then assembling these views to form the image the fish can "see", as in figure 1.

Figure 1: Assembling partial views In the following sections we will first describe the prototype installation then discuss the technical issues raised by the research project, finally we focus on other important issues that arise from considering an installation as artwork.

THE INSTALLATION :

“Fish” is to be an interactive installation based on a research project consisting of virtual partitioned environments populated by autonomous agents with a vision-based user interface. “Fish” redefines the notion of an aquarium as an ornamental structure placing artificial life in a visual context based on the esthetics developed by astronautics in the 60's. The goal is to emphasize the esthetics of the artificial regardless of its being organic life or not. The piece will occupy a room with as yet undefined measurements and will use a SGI Octane or Onyx Workstation, two video projectors and a museographic structure with the necessary illumination. A prototype installation was shown in the conference Computación Visual 1998 as an art installation using a SGI O2 computer. In this prototype 1. 2. 3. 4.

The fish were projected on two large screens. A video was projected on top of the computer simulation using a second projector. There was a digital camera looking at the visitor. The visitor could also interact using a mouse (that was placed on top of a transparent platform covered with grass on the sides, through which you could see the monitor as if it was part of an aquarium or lake) 5. The setting was forest-like (real plants were used) in order to imply the belonging of the fish agents in the real world. 6. This prototype application was nothing more than a proof-of-concept, and containsed simple synthetic vision and a simple decision engine. The application was not shared or partitioned as in the research described below (this prototype and installation is based on research by Isaac Rudomin, Saskia de Winter, Montserrat Morales published in 1997 [1,2,3] which will be described in the next section).

2

I NVENCAO AUG 1999


7. More research has been done since (by the group at ITESM-CEM led by Dr. Rudomin) in the related subjects of networked VR, agent modeling and animation[4,5] and multimodal interfaces, but will not be discussed here.

PROTOTYPE APPLICATION:

As was described above, the prototype installation contains a projection of a computer simulation, which is a proof of concept application within the research project. As it can be seen in figure 2, the application is a computer simulation which consists of a world populated with fish, one of which can react to images presented on a simulated television.

Figure 2: Fish watching T.V. We describe this in more detail: 1. The prototype application was programmed using VRML and Java, using EAI. To view we use Netscape and Cosmo Player under Irix and/or Windows 95/98. In the VRML world there is a big fish, a TV set, several small fish and submarine plants.This can be seen in figure 3.

Figure 3: Fish: VRML using Netscape 2. The big fish reacts to what he sees on TV after the button "Inicio" has been pushed; the TV shows pictures of different fish or images taken with the camera in the installation. The images are selected at random.

3

I NVENCAO AUG 1999


3. The big fish analyzes the image in order to decide where to move next. If the fish is too far away or no more new images are being shown on the TV, the button "Reinicio" may be pushed. If you click on the big fish, a different shape is selected. 4. There is a button to add more small fish. There are several pre-programmed viewpoints, or navigation can happen using Cosmo Player's controls, as can be seen in figure 4.

Figure 4: Fish within Cosmo Player

TECHNICAL ISSUES:

Terzopoulos et. al. [6], describe “Go Fish”, an application simulating hydrodynamic movement of fish that have been endowed with vision and the capability of learning different behaviors. Patty Maes et. al. [7], present an application called “Alive” in which an artificial creature, a dog, can interact with the user through a vision interface. Each of these projects embodies part of what we want to achieve. We desire to have creatures with synthetic vision like in “Go Fish”, that interacts with the user using vision, like in “Alive”, that are intelligent and autonomous (at least like in either of them), but all within a multiuser environment available through the Internet. The architecture for partitioned shared virtual environments and its implications for synthetic vision are the main points of the research described above. For efficiency and size reasons, the virtual environments we wish to construct are to be partitioned, that is, subdivided into regions to be handled by different server processes. Consistency between regions must be preserved. Another interesting consequence of partitioning is that generating images, both for viewers and agents (because they have synthetic vision they require these images) involves assembling the image from pieces in different servers. This will be our focus. ARCHITECTURE OF THE VIRTUAL WORLD:

The term Distributed VR refers to a simulated world that exists on several connected computers. A virtual world can be subdivided into regions so that better use is made of the CPUs of the connected

4

I NVENCAO AUG 1999


computers. Each region must therefore be controlled independently, but coordinated with the others to ensure consistency. Several architectures can be used: 1. The simplest one is a centralized model where a single server controls the simulation and distributes the information to the users. This server can become a bottleneck. 2. Another model is a completely decentralized model where all machines have a complete model of the simulation. Here the communications can saturate the network. 3. Another solution is a partial decentralization by regions, and is the model we will describe in detail below, and we have chosen to implement. The partially decentralized model involves having a server for each region, aware of the positions of agents (fish) and avatars (users) within the region, and communicating these to the neighboring regions when necessary. A user has a copy of the region his avatar is in, and registers with this region. Both the server and the user’s workstation are calculating the new positions for the agents. When the difference between the simulations is beyond a certain threshold, the user’s description of the agents' positions is replaced by the more accurate version from the server. This process allows for consistency while minimizing communication overhead. It is essentially an extension of what in the literature is described as dead reckoning. VISION IN A PARTITIONED WORLD:

Computer vision is typically part of a robotics project. In a certain way, this project is similar. Autonomous agents obtain information by several sensing methods, among them synthetic vision. In this project, the agents (fish) use vision to collect information about the simulated world they live in. There is, however a problem that must be solved in partitioned worlds that is not present in monolithic worlds. In the case of partitioned worlds we have access to images and objects from the region we are in, and not from the others. We must ask the manager of the region we are in to request this information from the neighboring regions and use this information to generate composite images of what an agent or avatar can see, before we invoke the usual vision algorithms. This was mentioned above, (check figure 1 above). RECENT DEVELOPMENTS:

After the piece “Fish” was developed, our group has continued work on several fronts on issues raised by the previous research: 1. Use of VRML and Java was problematic in that generating offline a rendered image of what the fish could “see” was not possible on the available platforms (Cosmo Player). A switch to Java3D is being analyzed. This is needed in order to use synthetic vision. 2. The distributed architecture described above can be improved on using an object-oriented approach. 3. The behavior model can be made much more complex[4] and generalized to other types of agents beyond fish[5].

5

I NVENCAO AUG 1999


Our group continues researching these topics, and articles will be forthcoming. We will incorporate these results into art installations as available.

CONCLUSIONS:

There are important technical issues to be solved in this line of research, but when applying the results to art, other important issues arise. When using ARTificial Life as a tool for art, we are involved in the always present issue of art as representation of nature, but we go beyond that, in recreating nature in some fundamental sense. Another issue is raised by using autonomous agents, since the esthetic decisions are removed from the artist’s direct control, and put under the responsibility of the users and, more importantly, the artificial agents. When we endow our creations with vision, in a way we go beyond the usual way in which we perceive art, since not only do we see art, now art sees us. This is something new in practice, but it is something that has been talked about even as long ago as ancient Greece, and part of traditions and stories such as that of The Golem and Frankenstein.

Figure 5: Art sees you If we have a multiuser world on the Internet as an art piece, it is really no longer a piece that is in one place. It becomes a shared social experience as well. When we partition the world, and distribute it among different computers, it is not really anywhere, its everywhere and nowhere. What we are trying to do is to construct a natural, inmersive, interactive, ephemeral, social and esthetic experience. We think that in the hands of an artist this becomes a very useful medium, with unprecedented creative potential, which we endeavour to explore.

6

I NVENCAO AUG 1999


REFERENCES

1. Isaac Rudomin, Saskia De Winter, Montserrat Morales, "Ambientes Virtuales Particionados Habitados por Agentes Autonomos y con Interaccion con el Usuario Basada en Vision", CLEI 97, Santiago de Chile 2. Saskia De Winter, Isaac Rudomin "Synthetic Computer Vision for Autonomous Agents in Distributed Partitioned Environments", Computacion Visual 97, Mexico Df 3. Montserrat Morales, Isaac Rudomin, "Arquitectura para Realidad Virtual Distribuida con Fisica Simulada, Poblada de Agentes Autonomos", Computacion Visual 97, Mexico DF 4. Laura Marina Bernal, Isaac Rudomin, “E Especificación de un modelo y desarrollo de una herramienta para animación basada en comportamientos”, Encuentro Nacional de Computacion, Pachuca Mexico, Septiembre 1999. 5. Maria Elena Melon, Isaac Rudomin, “Sistema para modelar y animar personajes articulados flexibles vestidos”, Encuentro Nacional de Computacion, Pachuca Mexico, Septiembre 1999. 6. D. Terzopoulos et.al. “Artificial Fishes: Autonomous Locomotion, Perception, Behavior, and Learning in a Simulated Physical World” , Artificial Life 1,4, December 1994, 327-351 7. P. Maes et. al. “The Alive System: Full Body Interaction with Autonomous Agents”, Proceedings of The Computer Animation ’95 Conference, Geneva, Switzerland 11-18, IEEE Press 1995

7