Platform for distributed multimedia environments supporting arbitrarily nested team structures Ali. A. Nazari Shirehjini*a, Hannes Guddata, Stefan Nolla, Norbert Schiffnerb a Fraunhofer-IGD, Fraunhoferstr. 5, D-64283 Darmstadt, Germany; b Fraunhofer CRCG, 321 South Main St., Providence, RI USA
ABSTRACT In this paper a novel platform, HOUCOM™, for the development of team based distributed collaborative applications is presented. Its implementation within a scenario for distributed cooperative virtual product development, ProViT, will be shown, describing the features and advantages of the presented platform. The specified platform consists of a decentrally organized, dynamic and user-configurable architecture. The main entities within the given platform are Conferences (working groups), Sessions (sub-groups), Users, Components, and Shared Resources. The system provides support of hierarchical Session Management, allowing for arbitrarily nested groups and multi-conferencing. Within the given platform Users can be individuals as well as technical devices, e.g. a streaming framework. The ProViT scenario builds a collaborative environment for interactive distributed VR Design reviews for the mechanical engineering industry. Here several distributed clusters form a working group, allowing individual partners to immersively collaborate on 3D models and supplementary documents and communicate via A/V-streaming. This paper divides into three chapters, first describing the ProViT scenario and deriving its requirements. The subsequent chapter examines the novel concept in general and the features that helped meeting the given project requirements in particular. In the conclusion the authors give an outlook on future extensions and applications of the developed platform. Keywords: Nested conferencing, collaborative VR design, Collaborative Environments
1. INTRODUCTION We present the necessity of the presented framework design on the basis of concrete developments in a current project, ProViT [2], that demands for nested collaborative conferencing. 1.1. ProViT The use case of the ProViT scenario is to enable virtual conferences for engineering industry experts in a distributed VR environment. The main features of the system are: - immersive A/V conferencing, embedded in the VE environment - integration of visualisation and simulation tools - joint manipulation of virtual objects - support for the special requirements that arise from the hierarchical topology of the expected conference user communities (esp. in competitive design reviews) To achieve this, the system comprises techniques and working environments from the fields of Virtual Collaborative Environments and immersive Telepresence which are enhanced by physics-based real-time simulation components. 1.2. Requirements The system has the following requirements regarding its supporting framework:
*
[email protected]; Phone 0049 6151 155208; Fax 0049 6151 155559; Fraunhofer-IGD
-
Extensibility. Different scenarios (engineering, architecture) as well as different technological platforms (CAVE, Workbench, Workstation) or company policies (security) will demand for additional features not completely foreseeable today, thus future extensions of the system components must be possible.
-
Adaptivity. Within a defined set of services that form a ProViT Conference, different setups are possible. A supporting framework must support dynamic allocation and activation of system components.
-
Reusability. ProViT Conferences will always comprise a standardiseable set of components and conference data. Thus mechanisms for maintaining application modules will alleviate the task of setting up a ProViT Session.
-
Security. All data exchanged in a ProViT Session divides into confidential data and general conference data, necessary to uphold a successful partner interaction. Therefore the conferencing framework must be able to organize and secure different groups of data between different conference participants, while on the other hand still being easy to set up, integrate and control.
1.3. Nested conferencing To guarantee fairness and confidentiality of the data exchange between participants of design reviews and other possibly competitive forms of Conferences, the system must, on the user level, be able to successfully map hierarchical user groups, concerning data protection and interaction control. The system should be able to provide conference and data management rules that match the prerequisites of arbitrary conference constellations. On the other hand nested conferencing is also important for efficient representation of the underlying architecture, as different data services will require separate nested levels of data access. Regarding the given high demands placed against up-to-date conferencing applications the support of group management on the user as well as on the services domain proves to be a powerful feature of a conferencing service framework. Concerning the ProViT system topology it would be of major importance to separate local and global data. By implementation of conferences organised in nested groups we can realise complex scenarios not possible with flat Conference structures: -
by separating global and local data the performance of the framework as well as the system performance concerning network load can be greatly improved. As, for example, local session events and data, exchanged between local data services, will to a great part be irrelevant to other conference participants, they will not have to be exchanged outside the local cluster. This saves bandwidth and minimises negative effects of long-distance network delays that would otherwise unnecessarily affect the local conference nodes. (see Figure 1)
-
as different levels of confidentiality regarding user and application data (model data, conversations or documents) apply between different conference participants, filter and grouping mechanisms must apply to data communication also on the user level. This will enable the hosting of complex conference hierarchies, where confidence and effective cooperation can only be achieved by trustworthy mechanisms of routing and exchanging data.
2. A PROPOSED SOLUTION The HOUCOM™ consists of a decentrally organized, dynamic and user-configurable architecture. The specified platform simplifies the development of distributed cooperative applications. It provides a set of C++ classes a programmer of such application systems can use in order to do hierarchical Conference and Session Management, allowing for arbitrary nested conferencing (section 2.2). HOUCOM™ provides support of -
Conference and Session Management Event Distribution and Handling (section 2.3) Service (Component) Management (section 2.4) Shared Ressource Management (section 2.5) User Management (section 2.6)
The HOUCOM™ system is extensible by integrating additional binary components providing extended functionalities, such as Conference announcement.
2.1. Architecture Overview The Figure 2 shows main classes of the HOUCOM™ in UML [1] notation. Main entities are (Sub-) Conference, Session, User, (Shared) Resource, Info Object, Service Interface, Service Component, Observer, and Event. Following sections give a brief overview of these entities. The Core system provides the very basic and essential functionality all collaborative applications have in common. More specific and specialized functionality is implemented by additional Components that are dynamically linked to the Core System at runtime (ocmComponent’s subclasses). The Core System is a Singleton [5] and provides a low-level API it exposes to client applications (ocmCore). Applications use the Core System and it’s subsystems (e.g. Service Framework, User Management) for creating and maintaining Conferences, Sessions, Events, User, Services, and Resources. Sessions, (Sub)Conferences are Managed through the ocmSessionManagerInterface. Users are managed through the ocmUserManagerInterface. Resources are managed and stored by ocmResourceManagerInterface. Resource specific data (e.g. fields of a table stored in a RDBMS) and Resource information are retrieved through the Proxy of each Resource (ocmResourceProxyInterface). The ocmServiceManagementInterface allows for management (e.g. creation, retrieval, update) of Service Components. Info Database is in charge of Conference state representation (ocmInfoDatabase). Conference (ocmConference) is the logical aggregation of Sessions. The class ocmConferenceInfo contains meta data of the Conference. Sub Conferences are subordinated Conferences (ocmSubConference) controlled by a superior Conference. Session (ocmSessionInfo) represents a cooperative application within a Conference by storing meta data of corresponding application. User is a human or process (ocmUser) that controls a Session. Each User gets a role (the ocmRole and the ocmRoleInfo) by joining a Conference from the master of that Conference. Resource (ocmResource and ocmResourceInfo) allows sharing objects (data containers, controllable web cams) in a Conference among the participants of that Conference. Information Object is an informational abstraction of an entity (i.e. Conference, User) within a Conference. The ocmInfo is the base class for all Info Objects. Service is textual functional specification of a cooperative Service (e.g. Chat) made available to Conference participants. Services that provide Core System functionalities (e.g UDP transport channel) are called Core Components (ocmComponent) or Components in short. Service Interface (ocmServiceInterface) is abstract, formal functional description of a Service functionality (or a part of that functionality). Service Components implement Service interfaces by means of Dynamic Link Library (DLL). A Service derives always from the ocmService class. Observer is an object-oriented design pattern [5] that defines a one-to-many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically. Used in HOUCOM™ in order to notify a cooperative application or a component (user defined Service Components as well as Core Components) on changes in the Conference state, while letting the component or application decide on how to handle the state transition. There exist in HOUCOM™ Conference Observer, Session Manager Observer, or Core Observer. All of them are derived from the ocmObserver. Event (ocmEvent) is a network message exchanged by participants of a (Sub)Conference in order to indicate state changes.
2.1.1. System extensions and reconfiguration One advantage of HOUCOM™ is its high user- modifiable architecture that allows for tailoring it for special requirements of collaborative applications.
Node: Aachen 9( $SSOLFDWLRQ
Video Strea
Network Managem ent
Network Managem ent
HOUCOM SM
Audio
Proxy
Proxy
S
C
Node: Berlin
HOUCOM SM
C
S
Network Managem ent
S
C
C
HOUCOM SM
S Audio
S
HOUCOM SM
Proxy
9($SSOLFDWLRQ
Group A (Application)
C
Video Strea
S
HOUCOM SM
Audio
C
Node: Darmstadt
Video Strea
9( $SSOLFDWLRQ
S
C
Global Group Conference
S
Local Group – Subconference A/V Streaming
Group S (Streaming)
Application
Figure 1 nested Conference structures can help organising the flow of local and global information between distributed nodes
The Core System is restricted to the minimal functionality commonly required by cooperative applications [3] and can be easily extended by Components (ocmComponent, i.e. binary code modules that are linked to the cooperative application at runtime). This as well allows for altering the functionality by linking specialized Services that replace built-in Components of the Core System. Moreover, a programmer of an application is able to partially define the behaviour of the HOUCOM™ through object-oriented call-back classes (observers).
2.2. Conference and Session Management HOUCOM™ provides mechanisms that facilitate the efficient management of small and medium Conferences and the Sessions participating in such a Conference. The Conference and Session Management as well as their organization is done by following objects: Core, Conference, Session, Session Manager, and Conference Observer. Additionally, these objects together with the Info Hierarchy (section 2.2.4) and Info Database (section 2.2.3) are in charge of Conference state representation, Event distribution and handling.
-
-
-
Core is a Singleton [5] which provides methods for starting, joining. Leaving, announcing, retrieving and terminating Conferences. Furthermore, it allows for defining Core System observer objects as well as for sending generic Events to other Conference participants. (Sub)Conference allows for broadcasting generic Events, defining Conference observers, retrieving Conference participants, and transferring master privileges to other Conference participants. Session contains metadata of a cooperative application. It provides methods for retrieving information and for extending the Info Hierarchy by adding Service Infos to itself.
Figure 2 Overall system architecture of the proposed software -
Session Manager (SM) is responsible for the integration of a process into a Conference. It is also responsible for logic constitution of a Conference. The Session Manager Interface (SMI) is designed for asynchronous, Eventbased message processing. This means that all communication with other SMs is based on Events exchanged between SMs, while Event exchange is done asynchronously (i.e. Events are sent without waiting for feedback). Events that are sent as results of previous Events are sent back in form of a independent Event, carrying the Asynchronous Completion Token (ACT) of the original Event, thus allowing the receiver of a respond to associate the response with its origin Event. The SMI allows for defining Session Manager Observers, Conferences Management (creating, joining, announcing, inviting Sessions to, leaving, terminating, retrieving), Service Management (creating, publishing, updating, removing). Furthermore, it indicates Service creation, update, publishing and removal as well as Conferences termination to other Session Managers. It also allow for retrieving special Info Object from remote Sessions.
-
Conference Observer provides virtual methods that must be redefined by application programmers (call-back). There are corresponding methods for several Events including “new Conference”, “heart beat detected”, “new participant”, “ new Service”, “participant is leaving”, “participant left”, “Service is shutting down”, “Service shut down”, “Conference is terminating” “Conference terminated“, “master role transfer requested”, “commit for new Service”, “master role switched“, “Service is reconfiguring”, “Service reconfigured”, “error”, “request for participation”, “generic Event arrived”.
1
ocmCore
0..*
1
ocmObserver
1 1
* 1 1
ocmSessionManager
ocmConference
*
1
ocmSubConference
* ocmSession
* Figure 3 Main Classes of Conference and Session Management 2.2.1. Conference organization. Conferences in HOUCOM™ are logical aggregations of Sessions, each one representing an application instance within that Conference. Sessions belonging to the same Conference are directly cooperating, allowing applications to share processes and data. HOUCOM™ supports nested Conferences, i.e. Conferences that are part of another Conference. This allows for advanced Conference organization by splitting participants into smaller subgroups. The figure below shows an example for such a nested Conference (Figure 4). Conferences and Sub Conferences are established in the same way. At the time of the master application assigns a role to the Session, determining the privileges that Session has within the Conference. A client never can have more privileges than the master of the Conference, i.e. a master can only grant those privileges that he owns as well. Consistency of a Conference with regard to Conference state is accomplished in HOUCOM™ by means of Session Management Events. Changes to the state (e.g. addition of new Sessions, updates to Info Objects) are indicated to participants by sending appropriate Session Management Events to them. Events can be broadcasted on three levels: - Conference-bound: the Event is distributed among the participants of the same Conference - Conference-spanning: the Event is distributed among the participants of another Conference - Session-bound: the Event is distributed to a subset of Sessions (part of the Conference or not)
By default, Events are broadcasted using the Conference-bound scheme. Because Conferences are organized by hierarchical tree structures, Conference-bound broadcasts can be extended to Conference-spanning broadcasts by an optional “depth”, i.e. the Event is broadcasted to Sessions of subordinated Conferences as well, if the level of that Conference is within the depth limitation. 2.2.2. Conference state representation. The global state of a Conference is determined by the states of its constituting parts. The constituting parts of a Conference in HOUCOM™ are given by the (Sub-)Conferences, Sessions, Users, Services, as well as (Shared) Resources. Each constituting part is represented by an associated Info Object. HOUCOM™ takes care of the consistency of this Info Hierarchy, i.e. each participant shares the same view on the common Info Hierarchy. 2.2.3. Info Database. The Info Database is HOUCOM™’s primary mechanism for Conference state representation. It is defined by the ocmInfoDatabase class. The database organizes the Info Objects of a Conference into an Info Hierarchy (section 2.2.4). Consequently, there is a set of different Info Objects: -
Conference Info encapsulates information on a Conference or SubConference; has associations to Session, Service, stream and Resource Infos.
-
Session Info encapsulates information on a Session, e.g. IP addresses, Session configuration, application category, etc.; has an association to a User Info (designating the human user that controls the application).
-
User Info encapsulates information on a user, e.g. address, HOUCOM™ public key (used for creating certificates)
-
Service Info encapsulating information on Service Components, e.g. Service configuration.
-
Resource Info encapsulating information on Resources, e.g. type and category, IP addresses, access restrictions.
The Info Database of a Conference is available to each participant of the same Conference. It is automatically updated by the Core System by Session Management Events. C o n fe re n c e (ro o t c o n fe re n c e )
S u b c o n fe re n c e (S u p e r G ro u p B )
S e s s io n
S u b c o n fe re n c e (G ro u p A )
S u b c o n fe re n c e (G ro u p A )
S e s s io n
S e s s io n
S e s s io n
Figure 4 Conference Organization
S e s s io n
The Info Database provides methods for inserting, updating, deletion, and query of Info Objects. It is possible to query existing Info Objects that have been published to other Sessions. Each Info Object is in charge of its integrity and checks the allowableness of LINK and UNLIK operations. The Info Database also allows for explicitly rebuilding the index. 2.2.4. Info Hierarchy. The Info Hierarchy is encapsulated by the Info Database. Basically, this is a hierarchically structured tree, whose nodes are Info Objects with its edges representing the associations between the Info Objects. The Core System provides a module for representation, modification and streaming of an Info Hierarchy or parts of it. However, streaming of Info Hierarchies is usually done automatically by the Core System, thus an application is not required to stream hierarchies manually, unless it needs to explicitly give copies of an Info Hierarchy to other Sessions. The Info Hierarchy provides methods for building and retrieving the index, setting and getting nodes from the index, setting and getting the root and streaming (traversing) of the hierarchy. 2.3. Event distribution and handling A Conference is represented by a state, e.g. the Sessions within the Conferences and their individual configuration within the Conference. HOUCOM™ provides a built-in mechanism for keeping this state in a consistent manner by broadcasting Session and Conference Management Events among the participants of that Conference. There is also the chance to send Events to a specific Session that allows for some private communication. There are two different mechanisms for Event broadcasting: -
Regular broadcasting: a predefined Conference or Session Management Event is broadcasted to participants of the same Conference.
-
Application-specific broadcasting: an Event that has been specified by a particular application is send to all participants of the same Conference.
INFO HIERARCHY AT CONFERENCE MASTER
INFO HIERARCHY AT CLIENT
Conference Info
Conference Info
Session Info Client
Session Info Master
Service Info Service Instance C (unpublished)
Service Info Service Instance A
Session Info Client
Session Info Master
Service Info Service Instance B
Service Info Service Instance A
Figure 5 Example of two different Info Hierarchies at two different Sessions.
It is up to the application to interpret and handle these (specific) Events in an appropriate manner. In general there are two categories of Events: -
One-way Events are sent from one Session Manager to another (remote) Session Manager, without expecting a feedback for that Event.
-
Duplex Events are sent from one Session Manager to another (remote) Session Manager, expecting a feedback for that Event.
2.4. Service (Component) Management The concept of Service Components is a key benefit of the HOUCOM™ system. A HOUCOM™ Service component (following referred to as „Service“) encapsulates a well-defined functionality and provides this functionality to a collaborative application that is based on the HOUCOM™ system. The Service concept of HOUCOM™ allows also for dynamic extension of the Core System. This as well allows for altering the functionality by linking specialized Services that replace built-in Components of the Core System. HOUCOM™ Services are mobile, i.e. each Service may be physically located within a Service Repository on a remote host, and is then transferred and linked to an application on another host upon request [10]. In contrast to monolithic systems, i.e. systems whose constituting modules are bound in a single executable file, the employment of Services instead allow a very high degree of flexibility and scalability due to their dynamic nature. A collaborative application at the top-most level relies on those Services, and requests and loads them at run-time, respectively. Seen from this point of view, an application based on dynamic Services is highly customizable and extensible. The Service Management provides functionalities for defining, developing, storing, on-demand retrieval and run-time brokerage of Service Components. 2.5. Shared Resource Management Shared Resource Management (SRM) is an implementation of the group memory pattern [3]. It provides necessary tools for sharing common objects between users in Conferences. SRM consists of following entities: -
Resource. Entity within a Conference that is shared among the participants of that Conference, usually refers to shared data containers, controllable web cams.
-
Resource Proxy. Provides and controls access to a Shared Resource.
-
Resource Repository. Component of HOUCOM™ Core System that stores and maintains information on available Resources. Allows for querying available Resources.
-
Resource Proxy Manager. Component of a HOUCOM™ Core System that manages available Resource proxies.
2.6. User Management User Management in HOUCOM™ includes -
Authentication of users during join process
-
Organization of joining users into groups/roles ordered by functional or geographic characteristics
-
Maintenance of persistent user-specific information
Each participant of a HOUCOM™ Conference belongs to one User Group. The membership in a particular User Group determines the privileges that member has within a Conference. Association of users and user groups is facilitated by the Conference master during join process. The management of a group is done by the Core Systems of the respective group members.
3. DISCUSSION This paper has shown the specific requirements of the ProViT scenario regarding its underlying collaboration platform. In order to guarantee fairness and confidentiality between competitive partners within a design review process, the underlying platform must be able to map hierarchical user groups and allow nested conferencing. On the other hand nested conferencing is also important for efficient representation of the ProViT architecture, which is composed of a group of collaborating local clusters. Also collaboration and data exchange between members of each local cluster occurs. By separating local and global data the performance of the system is significantly improved. The paper has focused its attention on the HOUCOM™ platform, which meets the demands of the ProViT scenario. In HOUCOM™ a conference can be part of other conferences. In other words, a conference is composed of sessions (local representation of a process or user) and further conferences. This marks one of the major innovations of the HOUCOM™ system. HOUCOM™ Conferences are organized in structured trees. Furthermore, HOUCOM™ provides mechanisms for broadcasting Events, User and Group management as well as for sharing resources between conference participants. The HOUCOM™ platform is dynamically extensible by means of Service Components. 3.1. Related Work Important technologies for developing collaborative virtual (engineering) applications are component distribution, Event Management, state-synchronization, and rendering. Currently, there exist several frameworks and toolkits providing these functionalities. The Deva system [7], developed at the Advanced Interfaces Group (AIG), is a toolkit for developing multi-user virtual environment applications with the ability to specify multiple active environments, laws and behaviours. BAMBOO [8], developed at the Naval Postgraduate School of Monterey, is a portable system supporting real-time, networked virtual environments. It focuses on dynamic (run-time) extensibility, and supports its application with crossplatform portability, networking, and 3D computer graphics features. The system can be dynamically extended by adding plug-ins. The Terazona [9], developed at the Zona Inc., is a Software Development Kit that support developing of large-scale multiplayer online games. It provides scalable and reliable networking, state tracking and synchronization, user Management, 3D rendering as well as a component framework which allows designing games for extensibility and reusability. There are several projects targeting collaboration support for VR or Virtual Environments in general, but none we are aware of that specifically targets all requirements identified in first sections. 3.2. Future Work and Activities The future work includes a refinement of HOCUOM™’s security and access control concept and Microsoft .NET support.
REFERENCES 1.
Unified Modelling Language. Object Management Group. http://www.omg.org/uml/UML
2.
ProViT. http://www.provit.net
3.
Luis A. Guerrero, David A. Fuller (2001). A pattern system for the development of collaborative applications. Elsevier Journal, Information & Software Technology, Volume 43, Number 7, pages 457-467, June 2001.
4.
Hypermedia and Pattern Based Knowledge Management for Smart Organisation (2000). EU-Programme Information Society Technologies, Nr.: IST-2000-28401. Retrieved May, 12th, 2003, from Web site: http://www.verbundplan.at/HyperKnowledge/
5.
Gamma et al. (1985). Design Pattern. Addison Wesley Professional Computing Series, October 1994, ISBN 0-20163361-2.
6.
S. Pettifer, J. Cook, J. Marsh, and A. West (2000). Deva3: Architecture for a large scale virtual reality system. In Proceedings of ACM Symposium in Virtual Reality Software and Technology 2000, pages 33-39, ACM Press, October 2000, ISBN 1-58103160-2.
7.
Stephen R. Pettifer (1999). An Operating Environment for Large Scale Virtual Reality. PhD thesis. The University of Manchester, Faculty of Science and Engineering, April 1999. Retrieved July, 2003, from Web site University of Manchester: http://aig.cs.man.ac.uk/publications/papers/srp-phd.pdf
8.
K. Watsen and M. Zyda. Bamboo – a portable system for dynamically extensible, real-time, networked, virtual environments. Proceedings of the 1998 IEEE Virtual Reality Annual International Symposium, Atlanta, Georgia, March 1998.
9.
Terazona White Paper(2003). Retrieved July, 2003, from Zone Inc. Web site : http://www.zona.net/whitepaper/
10. M. Einhoff, N. Schiffner, A. Greb, C. Rühl (2000). Konzept eines Kommunikations-Framework für die Unterstützung von Virtuellen Unternehmen. In R.Reichwald, J.Schlichter (Ed.): Verteiltes Arbeiten - Arbeit der Zukunft, proceedings of D-CSCW 2000, German Chapter of the ACM, pages 23-34, Stuttgart, B.G.Teubner, September, 2000, ISBN 3-519-02695-3.
