Using Palmtop Computers and Immersive Virtual Reality for Cooperative archaeological analysis: the Appian Way case study Elisabetta Farella*, Davide Brunelli*, Maria Elena Bonfigli°, Luca Benini*, Marco Gaiani^, Bruno Riccò* *DEIS - Dept. of Electronics, Computer Science and Systems - Bologna University, ITALY °Vis.I.T. Lab - CINECA Supercomputing Centre - Casalecchio di Reno (BO), ITALY ^ INDACO - Dept. Industrial Design, Arts & Communication – Politecnico di Milano, ITALY {lbenini|dbrunelli|efarella|bricco}@deis.unibo.it
[email protected] [email protected] Abstract This paper aims to describe a prototypal immersive virtual reality application, running in a Virtual Theater, that uses portable terminals to visualize multimedia data and to manage the interaction with 3D models regarding the survey and restoration of eight tombs along the ancient Rome’s Appian Way. The starting point of our work is a 3D Web application where the user, interacting with VRML models of the tombs, can access to archaeological data: the baseline conditions before restoration, the various phases of restoration work and the final outcome [4]. A Virtual Theater Technology enables the users to experience a more realistic fruition of the 3D models of the tombs and the use of portable terminals greatly increases the possibilities of interaction in virtual environments, maintaining also the capability to access via Internet all the multimedia archaeological data developed for the previous web application. Moreover the application is designed in order to enable peer-to-peer collaboration among users who can share data and interactions during their virtual experience.
1.
Introduction
This work describes an integrated system that exploits palmtop computers to enable archaeological analysis of eight tombs in the Ancient Appian Way. The main features of the resulting application are quite the same of a previous work made on the same eight tombs exploiting Internet technologies [4]. The introduction of the new mix of technologies [5,6] provides archaeologists with Immersivity, Multimedia data support and Cooperation. Immersivity. VR with its increasing dynamic, interactive and experiential characteristics can simulate real environments with various degrees of realism. In particular Immersive VR enables a fuller contact between users and the virtual environments. The integration of immersivity in virtual archaeology applications enables archaeologists to understand and examine better 3D models [8]. In our application the illusion of immersion is obtained projecting 3D models in the large semi-circular screen system of the CINECA’s Virtual Theater that fills the field of view of the users. Archaeologists, wearing stereographic glasses can appreciate the maximum level of precision of single models – regarding current state, restored state as thought by Canina and excavation of the eight tombs - converted in the previous application to VRML format to make them accessible through the Internet. Moreover immersivity, and the consequential preservation of the sense of presence in the Virtual Environment, is strengthened by the design choice to delegate to untethered systems the management of interaction tools and of the user interface. The choice is based on the consideration that the palmtop computer is experienced as immersed with the user in the virtual environment.
Multimedia support. The integration with palmtop devices gives to virtual reality technologies great advantages. The most common PDAs are portable, lightweight and enhanced with fast processors, RAM and ROM capabilities, supporting complex tasks such as the visualization of multimedia data (images, texts, voice, sounds, video/audio streaming), etc [1]. Thanks to these features, palmtop computers are used in the new Virtual Appian Way Immersive Application not only for granting a friendly active participation to the virtual experience, but also to visualize and insert multimedia data and comments both during the virtual experience and not. Using the same device to insert and catalogue notes and physical evidences in situ directly during the excavation process and to manage the virtual experience, it gives the possibility to establish relationships between data and models, to collect/retrieve data, and to make better archaeological analysis. Moreover the use of palmtop computers enables archaeologists to access to the network database available in Internet [4]. Cooperation. Almost all palmtop computers of the last generation on the market today feature wireless networking capabilities [1]. Usage of PDAs in Computer-Supported Cooperative Work (CSCW) is an interesting field of research and has particular relevance in Single Display Groupware (SDG), where people are physically close to each other and share the same output display [3]. Our application supports concurrent multi-user interaction for networked environment, providing exchange and collection of multimedia data. This can be particularly useful in archaeology field for make hypothesis of archaeological reconstructions and to share multimedia data. The paper is organized as follows: Section 2 shows the Hardware and Software technologies used for the application; Section 3 describes the application itself, Section 4 depicts System Architecture and implementation aspects; finally section 5 concludes the paper. 2.
Hardware and software technologies
We can describe the new Virtual Appian Way Immersive Application as the combination of four functional components: 1. Stereographic visualization: the 3D virtual models of the Tombs in different statuses and the 3D models of most significant tomb details. This functional parts corresponds to the server side of the application and is enabled by the Virtual Theatre 2. Interaction/Navigation: the combination of all the tasks enabled by the widget on the PDA GUI at client side and by the graphical libraries at server side. In particular it is the ability to i) move an avatar along the Ancient Appian Way 2D map [7] ii) select the tomb to visualize in the Theatre iii) selecting a tomb status (e.g. navigation through restored, ideal, original state) iv) manipulate the tomb v) changing the system status vi) connecting to the object visualized hyper textual information (images, text and any sources of the reconstruction) 3. Peer-to-peer communication: all the features that enables communication between clients supporting collaborative work: i) Theatre control exchange; ii) communication through whiteboard; iii) other clients awareness through colored dots on the 2D map iv) in future, shared files, exchange of text, images, video and sound between clients 4. Storage and organization of heterogeneous data. That is the exploitation of the chances offered by the PDA platform, which has powerful hardware characteristics. In fact is useful to a more effective work support the ability to organize, store, process different data types and the chance to run concurrently different applications. In the following lines all enabling hardware and software that realize all these four functionalities will be described in details.
2.1
Server hardware and software
As already stated Stereographic visualization is delegated at server side, where the enabling hardware and software technologies are: an SGI Reality Center and some graphical libraries, which play a main rule also for realizing Interaction and Navigation. The SGI Reality Centre, i.e. the CINECA’s Virtual Theatre (see Figure 1 below), is the combination of (i) the Onyx2 with 3 graphic pipelines each with 64 MB texture memory and 8 processors MIPS R10000 (ii) 3 BARCO Video Projectors with Edge Blending and related control electronics; (iii) triple screen console for RC monitoring (iv) the semicylindrical screen (3.8m radius, 160deg opening) (v) the audio system. The stereographic vision is obtained through shutter glasses synchronized with the three projectors.
Figure 1:The Virtual Theatre at CINECA
On top of the Operating System, IRIX 6.5.14, different layers of powerful graphical libraries are loaded (see Figure 2). In particular Vega and OpenGL Performer layers are high-performance multi-process rendering graphic libraries that fully exploit the computing power of the graphic supercomputer Silicon Graphics Onyx2 to provide the necessary characteristics of realism and immersion. On top of them there are some C libraries which call Vega and OpenGL Performer functions, but could be referred by Java JNI technology. In this way we have a set of method to interact with the IVE, whose initial status (i.e. all the initial application parameters like visual channels, view points, observers, special effects, time of day, system configuration, Openflight 3D models, interaction modes, etc.) is described in the Application Definition File (.ADF) that can be run with executable files calling Vega functions.
Figure 2: Technologies used at Server side
2.2
Client hardware and software
At client side the enabling hardware for Interaction/Navigation, Peer-to-peer communication and Storage and organization of heterogeneous data are the infrastructure for connecting the portables terminals (PDA) to the network (i.e. the Base
Station, wired to the LAN of CINECA, and the NICs: Network Interface Cards) and the PDAs themselves. Specifically, our terminal was a Compaq iPAQ 3660, with a 320 x 240 resolution TFT color screen, 64Mb RAM and 32Mb ROM memory. It is a slim and powerful platform, and it has the fastest available processor in market (206 MHz Intel® Strong Arm 32-bit RISC proc.). We chose it also because Compaq provides a very complete set of expansions for the iPAQ and the needed hardware for being connected through different communication interfaces (IEEE 802.11b, GPRS, GSM, Bluetooth…). Among them we chose the IEEE 802.11b High Rate standard wireless LAN (WLAN) interface working at 11 Mb/s because our application was for indoor environment and the costs of the infrastructure were reasonable (no adding cost during connection as through a telephone line). In particular, we used Cisco Aironet 340 series of WLAN products, that is the Access Point and the NICs. The client-side software layers are shown in Figure 3, below. The Operating System is Windows CE 3.0 (WinCE). WinCE opens the possibility to develop software, using specific API to write code for PDA devices. We benefit of that when using JNI in our Java application to call WinCE API for concurrently running another application, such as Pocket Explorer, calling it directly from ours (selecting an item on a menu in our GUI). On top of WinCE there is a Java Virtual Machine, optimized for this Operating System and for Pocket PC. The highest layer is our Java Application, which exploits RMI technology.
Figure 3: Technology used for the Client
3.
Visiting the Appian Way through the PDA
The main aim of the new Ancient Appian Way Application is to provide an immersive virtual environment for collaborative work [9,10,11] among archaeologists, but also architects, virtual models designers, art experts and computer scientists, having in mind the already outlined characteristics: shared visualization, peer-to-peer communication, and, particularly important in Virtual Heritage, availability of all needed multimedia data (images, video, sound, text) and usability (for example archaeologists are often non expert users). A typical virtual visit along the Ancient Appian Way (see Figure 4) through the PDA consists in the following four steps: 2D map visualization; Application control management; Tomb selection and manipulation; Peer-to-peer communication. It is worth noting that thanks to the PDA capabilities, while interacting with VR models the user can open Pocket Word and take notes or other Pocket applications concurrently and independently from our GUI and retrieve data previously collected and stored, e.g. during the excavation.
Figure 4: Virtual visit of the ancient Appian Way
3.1
2D map visualization
The application at first presents to users a 2D map [7] of the Ancient Appian Way, only a portion on the PDA (see Figure 5) and whole on the large semi-cylindrical screen of the Virtual Theatre. In both cases the map shows colored dots representing where each client is virtually located along the Appian Way. This feature has two aims: (i) it helps spatial awareness inside the IVE to increase the idea of traveling along the Appian Way; (ii) at the same time it increases awareness of other clients while they are not actively communicating to each others. Other clients awareness will be augmented in future work associating different colors to the avatars according to different clients actions.
Figura 5: The AncientAppian Waymap displayed on the PDA
3.2
Application control management
The control of the Theatre is leased to the first client who selects a tomb, which became temporary the master, or to any other who ask for it if the master releases it (see Figure 6).
Figura 6: Theatre control request by a user to anotheron the PDA
The client who owns the Virtual Theatre control can select a tomb along the way just clicking on his name label. Doing the same a client who does not own the control can ask for it. In this case the event is processed by the server to display on control owner PDA a dialog box with the request to release it. If the client agrees the context changes and the tomb chosen by the new control owner is visualized. Figure 7 shows a change of context.
a
b
Figure 7: example of context change on the Virtual Theater screen (a) sepolcro di Caio Secundo (b) sepolcro di Usia Prima
3.3
Tomb selection and manipulation
When a tomb is selected, two main things happen: on the Virtual Theatre the virtual model of the selected tomb is loaded (see Figure 7). At the same time on the PDA another window is opened (see Figure 6). This new User Interface has buttons to rotate the tomb and to zoom in and out. There are also buttons to control the draw status (wireframe and textures) of the virtual model. Two menus allow the user to switch among the actual state, the restored state, the excavation and the ideal state and to outline details of the tomb of particular relevance (see Figure 8).
Figure 8: The menu to switch among different reconstructions for the same tombon the PDA a n d the corresponding visualization in the Virtual Theater’s screen
At any moment, selecting an item in one of the menus, the user can concurrently display on the PDA a Web page containing hyper textual information about the 3D model displayed in the Virtual Theatre. Through browsing the Internet the user can augment the understanding of the tomb with images and text sources (but also audio and video) related to it (Figure 9).
At any virtual object is in fact associated a Web page where the users can consult multimedia data.
Figure 9: Info recover
3.4
Peer-to-peer communication
Another feature implemented in the application is the establishment of a private session of communication between users. To begin a session a first client has to click with the PDA pen on the avatar (i.e. the red dot on the 2D map) of the user (s)he wants to have cooperation with. The result is a dialog box requesting the communication on the display of the second user. If (s)he agrees a whiteboard is opened on both clients and the communication starts (Figure 10). What is drawn on a whiteboard appears also on the other and viceversa.
Figura 10: Whiteboard use example
4.
System architecture
The software implemented consists on a client-server application implemented following a modular code design in order to allow the adding of future modules. Both client and server are written in Java language, exploiting distributed computing libraries. Java guarantees platform independence, a desirable characteristic in a market in rapid evolution. Moreover, in our implementation we exploit Java Remote Method Invocation (RMI) to manage effectually distributed resources. Server architecture, shown in Figure 11, is organized in the following modules: (i) the Scheduler (SC); (ii) the Avatar Manager (AM), which provides update of status of users
interacting inside the VE; (iii) the Theatre Manager (TM), employed in Virtual Theater scene updating. This module also invokes graphical API needed to modify the VE. The SC is in charge to subscribe new Clients entering the system. The AM manages the Avatars, i.e. the users references on the server, to periodically update the status. When a visitor subscribes, the server instantiates an Avatar and both, Client and Avatar, gain the other reference. From this point onwards they can invoke remote methods and update attributes, so the Avatars are in direct contact with the Clients. Since is previewed the ability to assume the Theatre control showing the subjective user vision, a negotiation protocol was needed. The TM module is in charge to retrieve the position and the orientation of each Avatar and to call through the Java Native Interface (JNI) the Vega and Performer APIs needed to perform the actions, or other C/C++ functions specifically created to modify the scene and the graphical state of the application.
Figure 11: Server architecture
Client architecture, shown in Figure 13, is organized in the following core modules: (i) the Communication Manager (CM); (ii) the Event Manager (EM); (iii) the Screen Manager (SM). CM has the task to communicate with objects residents on the Server; usually it handles the remote reference of his own avatar and, when the user wants to obtain the Virtual Theatre control, it recovers the remote stub of the TM. EM handles the events generated by the user when (s)he navigates inside the IVE, and translates them in actions for the CM. Finally, SM was implemented to update the graphical interface. This is needed often, especially when the Server sends data to refresh the local user copy.
Figure 13: Client architecture
Figure 14 below shows the two type of cooperation implemented in our system. The first corresponds to a multi client-server organization. The clients interact with the shared IVE sending request or events to the server who manages them and eventually distributes to all the users. The second one is peer-to-peer organization, which allows collaboration between clients through a private communication channel, bypassing the server. Peer-to-peer features often requires exchange of multimedia data. At the moment only a whiteboard has been implemented. The user, who decides to open a private channel with another, recovers the remote reference from the Server and sends his request to the interlocutor. Depending on the answer the exchange will be held or not.
Figure 14: Cooperation types
The chance for each client to access the Internet while the user is experiencing the IVE is implemented through JNI. Therefore, the user can visit web pages related to 3D models projected in the Virtual Theater, containing hypertext data related to the objects. Through JNI a portion of C code is recalled. This code takes care of handling the Pocket Explorer application present in the Operating System Windows CE. 5.
Conclusion
Our application, mixing together PDAs and Virtual Reality Technologies, augments Immersivity, Multimedia data support and Cooperation in interactions with eight tombs along the Ancient Appian Way. This represents an advance in supporting archaeological analysis. Moreover we have made tests to evaluate performance and scalability that gave good results [3]. In particular our tests revealed that the application can afford up toi 20 clients concurrently with minimal performance degradation. Future works will focus on adding shared multimedia files among clients in order to augment cooperation features, and supporting different kinds of handheld devices: not only PDAs of different brands, but also laptops or cellular phones. References [1] Benini L., Bonfigli M.E., Calori L., Farella E., Riccò B.; Palmtop Computers for managing Interaction with Immersive Virtual Heritage, in Proceedings of EUROMEDIA2002, pp. 183-189. [2] Bonfigli M.E., Brunelli D., Farella E.; Untethered Interaction for Immersive Virtual Environments through Handheld Devices, accepted for publication. [3] Farella E., Brunelli D., Benini L., Riccò B., Bonfigli M.E., Multi-client cooperation and wireless PDA interaction in immersive virtual Environments; submitted for publication. [4] Gaiani M.; VR as a Tool for Architectural & Archaeological Restoration: The “Ancient Appian Way 3D Web Virtual GIS; in proceedings of 7th International Conference on Virtual Systems and Multimedia, VSMM2001, Berkeley, USA, 2001. [5] Hill L., Cruz-Neira C., "Palmtop interaction methods for immersive projection technology systems," Fourth International Immersive Projection Technology Workshop (IPT 2000), USA, 2000. [6] Watsen, K., Darken, D. P., Capps, W.V., “A Handheld Computer as an Interaction Device to a Virtual Environment” Third International Immersive Projection Technology Workshop (IPT 1999), Stuttgart, Germany, 1999. [7] Kwon T., Choy Y., "A new navigation method in 3D VE (2D Map-based navigation)," in proceedings of 6th International Conference on Virtual Systems and Multimedia, VSMM 2000, Softopia Gifu, Japan, 2000. [8] Acevedo D., Vote E., Laidlaw D. H. and Joukowsky M. S. "Archaeological Data Visualization in VR: Analysis of Lamp Finds at the Great Temple of Petra, a Case Study", In proceedings of IEEE Visualization 2001. San Diego, California. October 2001. [9] Smith R.C., "Shared Vision", Communications of the ACM, December 2001/Vol.44, No.12, pp.45-48 [10] Su S., Loftin R.B., "A shared Virtual Environment for exploring and designing molecules", Communications of the ACM, December 2001/Vol.44, No.12, pp.57-58. [11] Benford S. Grenhalgh C., Rodden T., Pycock J.; "Collaborative Virtual Environments", Communications of the ACM, July 2001/Vol.44.No7 pp.79-85
