EVALUATIONS OF AND EXTENSIONS TO A SECOND GENERATION HYPERMEDIA MODEL
by
LLOYD RUTLEDGE M.S. COMPUTER SCIENCE UNIVERSITY OF MASSACHUSETTS LOWELL 1993
SUBMITTED IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF SCIENCE IN COMPUTER SCIENCE UNIVERSITY OF MASSACHUSETTS LOWELL
Signature of Author:
Signature of Thesis Director:
Signatures of Other Thesis Committee Members:
Date:
EVALUATIONS OF AND EXTENSIONS TO A SECOND GENERATION HYPERMEDIA MODEL
by
LLOYD RUTLEDGE
ABSTRACT OF A THESIS SUBMITTED TO THE FACULTY OF THE DEPARTMENT OF COMPUTER SCIENCE IN PARTIAL FULFILLMENT OF THE REQUIREMENTS FOR THE DEGREE OF DOCTOR OF SCIENCE IN COMPUTER SCIENCE UNIVERSITY OF MASSACHUSETTS LOWELL 1996
Thesis Supervisor: John F. Buford, Ph.D. Associate Professor, Department of Computer Science
ii
Abstract This thesis explores the implementation of second generation open document hypermedia processing and presentation in the context of existing formal hypermedia models, including Hypermedia/Time-based Structuring Language (HyTime) and the World Wide Web. HyTime’s use within second generation hypermedia processing is analyzed and evaluated using existing models and established criteria. A sample application of HyTime is presented to provide an empirical evaluation of HyTime and to lay a foundation for other work in this thesis. HyTime document modeling and diagraming techniques are introduced that facilitate the authoring of documents, document sets, and applications. An analysis of the potential for automatically generating applications from HyTime document set code is presented to further facilitate this authoring process. Insights into the incorporation of multimedia scripts and HyTime documents are provided to promote the seamless integration of hypermedia components. Relationships between HyTime and Hypertext Markup Language (HTML) are determined so that the two languages can be cooperatively used in a hypermedia environment.
iii
Acknowledgments First I wish to thank my thesis director Dr. John Buford for his support and sagely advice. Further thanks go to thesis committee members Dr. Patrick Krolak and Dr. John Sieg for their involvement and guidance. I am also indebted to DMSL lab-mate Chetan Gopal and my lab-mate and brother John Rutledge for their insights into hypermedia and for their work on the DMSL HyOctane project, which provides the basis for much of this thesis’ research. Roger Price, another lab-mate and fellow HyTime committee member from early on, also has my gratitude for his experience and insight. Finally, thanks go to Steven Newcomb, Charles Goldfarb, Brian Markey, Len Bullard, and Elliot Kimber whose conversations and explanations in and out of standard development and conference meetings made HyTime comprehensible.
iv
Table of Contents Page
Abstract .............................................................................................................................. iii Acknowledgments.............................................................................................................. iv Table of Contents .................................................................................................................v List of Tables...................................................................................................................... xi List of Figures................................................................................................................... xii I
Introduction..................................................................................................................1 1.1
Context.....................................................................................................................1
1.2
Methodology ............................................................................................................2
1.3
Background ..............................................................................................................3
1.3.1
SGML ................................................................................................................3
1.3.2
HTML ................................................................................................................3
1.3.3
HyTime ..............................................................................................................4
1.3.4
Hypermedia Implementations............................................................................5
1.3.5
Multimedia Scripts.............................................................................................7
1.3.6
Hypermedia Design Methodologies ..................................................................9
1.3.7
The Dexter Hypertext Reference Model..........................................................11
1.3.8
Digital Libraries...............................................................................................13
1.3.9
General Sources of Information.......................................................................14
1.4 II
Overview................................................................................................................14
Models for the Classification and Evaluation of Hypermedia ...................................18 2.1
Concepts and Issues ...............................................................................................18
2.2
The Halasz Categorization of Hypermedia............................................................19
2.2.1
First Generation Hypermedia...........................................................................21
2.2.2
Second Generation Hypermedia ......................................................................21
2.2.2.1
Node Typing...............................................................................................22
2.2.2.2
Simple Link Structures ..............................................................................22 v
2.2.2.3
Hierarchial Support....................................................................................22
2.2.2.4
GUI/Windowed Interface...........................................................................23
2.2.2.5
Extensibility ...............................................................................................23
2.2.2.6
Simple Text Query .....................................................................................23
2.2.2.7
Simple Distribution ....................................................................................24
2.2.3
Third Generation Hypermedia .........................................................................24
2.2.3.1
Integration of Search and Query Functionality..........................................24
2.2.3.2
Composite Node Types ..............................................................................24
2.2.3.3
Virtual Structures over Node Collections ..................................................24
2.2.3.4
Computation over Hypermedia Networks .................................................25
2.2.3.5
Versioning of Nodes and Subgraphs..........................................................25
2.2.3.6
Support for Collaborative Work.................................................................25
2.2.3.7
Extensibility and Tailorability....................................................................26
2.2.4
Open Distributed Hypermedia Systems...........................................................26
2.2.4.1
Open Hyperdocument and Hyperlinking Architecture..............................27
2.2.4.2
Mobile Code and Distributed Objects........................................................28
2.2.4.3
Hypermedia Document Models .................................................................28
2.2.4.4
Security ......................................................................................................30
2.3
The Dexter Hypertext Reference Model................................................................31
2.4
Summary................................................................................................................33
III Review of Hypermedia Format Languages................................................................34 3.1
SGML ....................................................................................................................34
3.1.1 3.2
Halasz Classification of SGML .......................................................................37
HTML ....................................................................................................................38
3.2.1
HTML Hyperlinking Constructs......................................................................38
3.2.2
HTML Locating Constructs.............................................................................39
3.2.3
HTML Scheduling Constructs .........................................................................40
3.2.4
Halasz Classification of HTML .......................................................................41
3.2.5
Relating HTML with HyTime .........................................................................41
3.3
HyTime ..................................................................................................................42
3.3.1
HyTime Hyperlinking Constructs....................................................................44
3.3.1.1
Contextual Link (clink)..............................................................................44
vi
3.3.1.2
Independent Link (ilink) ............................................................................44
3.3.1.3
Aggregate Traversal ...................................................................................45
3.3.2
HyTime Locating Constructs ...........................................................................47
3.3.3
HyTime Scheduling Constructs .......................................................................48
3.3.4
Other HyTime Constructs ................................................................................49
3.3.5
Halasz Classification of HyTime .....................................................................51
3.3.5.1
HyTime as Second Generation Hypermedia..............................................51
3.3.5.2
HyTime as Third Generation Hypermedia.................................................52
3.3.5.3
Conclusion: The Halasz Classification of HyTime....................................53
3.3.6
3.4
Dexter-based Analysis of HyTime...................................................................53
3.3.6.1
Correlation Between HyTime and Dexter .................................................54
3.3.6.2
Where HyTime Falls Short of Dexter ........................................................56
3.3.6.3
Where HyTime Exceeds Dexter ................................................................57
3.3.6.4
Conclusion: The Dexter-based Analysis of HyTime .................................57
Java and HyTime....................................................................................................58
3.4.1
Hyperlinking ....................................................................................................58
3.4.1.1
Java Button and HyTime clink...................................................................59
3.4.1.2
Java Choice and HyTime ilink ...................................................................59
3.4.1.3
Java Menu and HyTime agglink ................................................................60
3.4.2
Locating: Java URL and HyTime notloc .........................................................61
3.4.3
Scheduling: Java Image and HyTime event .....................................................61
3.5
Summary................................................................................................................62
IV Sample HyTime Application......................................................................................63
V
4.1
Description.............................................................................................................63
4.2
HMP DTD..............................................................................................................66
4.3
HMP Sample Document Instance ..........................................................................69
Modeling Techniques for HyTime .............................................................................72 5.1
Introduction............................................................................................................72
5.2
Related Work .........................................................................................................73
5.3
Authoring Methodology.........................................................................................77
5.4
Document Model Diagramming ............................................................................82
5.4.1
SGML Parsed Document Diagramming..........................................................84
vii
5.4.2
SGML DTD Diagramming ..............................................................................85
5.4.3
HyTime Meta-DTD Diagramming ..................................................................86
5.4.4
HyTime Document Construct Diagramming...................................................89
5.4.5
HyTime-defined Document Model Diagramming...........................................90
5.5
Meta-HyTime Constructs.......................................................................................92
5.5.1
Hyperlinking ....................................................................................................93
5.5.2
Location Addressing ........................................................................................95
5.6
Summary................................................................................................................96
VI Toward Automatic Generation of HyTime Applications ...........................................98 6.1
Introduction............................................................................................................98
6.2
HyTime Engine, Classes, and API.........................................................................99
6.2.1
Link Module...................................................................................................101
6.2.2
Schedule Module ...........................................................................................102
6.2.3
Location Address Module..............................................................................102
6.2.4
Engine Services and the Application Layer...................................................103
6.3
Application Generation Model ............................................................................103
6.3.1
Application Class Definition Code Generation..............................................104
6.3.2
Generation of Application Code for Presentation and Access.......................105
6.4
6.3.2.1
Constructor Code Generation ..................................................................106
6.3.2.2
Method Code Generation.........................................................................106
Examples..............................................................................................................107
6.4.1
HMP...............................................................................................................107
6.4.2
Java.................................................................................................................108
6.5
6.4.2.1
Hyperlinking ............................................................................................109
6.4.2.2
Locating ...................................................................................................110
6.4.2.3
Scheduling................................................................................................111
Discussion ............................................................................................................111
6.5.1
HyTime Support for Code Generation...........................................................111
6.5.2
Media Objects ................................................................................................112
6.5.3
Interaction Classes .........................................................................................113
6.5.4
Updating the Database Layers .......................................................................113
6.5.5
Semantic Assumptions...................................................................................115
viii
6.6
Summary..............................................................................................................116
VII Integrating Object-Oriented Scripting Languages with HyTime .............................117 7.1
Introduction..........................................................................................................117
7.2
Architectural Context...........................................................................................119
7.2.1
Architectural Considerations .........................................................................119
7.2.2
An Architecture For HyTime And Distributed Environments.......................120
7.2.3
A Distributed Multimedia Environment Architecture ...................................121
7.2.4
An Architecture For HyTime With Embedded Scripts..................................122
7.3
Characteristics of Object-Oriented Scripting Languages ....................................123
7.4
Integrating Scripts with HyTime Document Models ...........................................126
7.4.1
Document Extensions ....................................................................................126
7.4.2
Script Language Extensions...........................................................................128
7.5
Integrating Java with HyTime Processing ...........................................................130
7.5.1
Hyperlinking ..................................................................................................130
7.5.1.1
Button.......................................................................................................131
7.5.1.2
Choice ......................................................................................................131
7.5.1.3
Menu ........................................................................................................131
7.5.2
Locating .........................................................................................................132
7.5.2.1 7.5.3
Scheduling......................................................................................................132
7.5.3.1 7.6
URL..........................................................................................................132 Image........................................................................................................132
Summary..............................................................................................................132
VIII HyTime and the World Wide Web ...........................................................................134 8.1
Introduction..........................................................................................................134
8.2
Issues in Applying HyTime to HTML .................................................................135
8.2.1
Replacing HTML Facilities with HyTime .....................................................136
8.2.2
Legacy Documents.........................................................................................137
8.3
Approaches in Applying HyTime to HTML........................................................137
8.3.1
The Complete Revision Approach.................................................................137
8.3.2
The Additional Construct Approach..............................................................138
8.3.3
The Restructuring DTD Approach.................................................................139
8.3.4
Conversion using Overlaying Documents .....................................................140
ix
8.4
8.3.4.1
Document-wide Constructs......................................................................142
8.3.4.2
Hyperlinking Constructs ..........................................................................142
8.3.4.3
Locating the HTML Document ...............................................................144
8.3.4.4
Locating the Starting Anchors .................................................................145
8.3.4.5
Locating the Ending Anchors ..................................................................146
8.3.4.6
The Hyperlinking Document Instance .....................................................147
Presenting HyTime Documents using HTML .....................................................148
8.4.1
Converting Hyperlinking Constructs .............................................................148
8.4.2
Converting Locating Constructs ....................................................................150
8.4.3
Converting Scheduling Constructs.................................................................151
8.5
Summary..............................................................................................................152
IX Conclusion ...............................................................................................................153 9.1
Motivation ............................................................................................................153
9.2
Accomplishments.................................................................................................155
9.3
Future Work .........................................................................................................155
Literature Cited ................................................................................................................158 137.Biographical Sketch of Author .................................................................................168
x
List of Tables Page
TABLE 1: TABLE 2: TABLE 3: TABLE 4: TABLE 5: TABLE 6:
Issues for Hypermedia ..................................................................................20 Models Addressing Hypermedia Issues........................................................35 How SGML Addresses Hypermedia Issues..................................................37 How HTML Addresses Hypermedia Issues..................................................41 How HyTime Addresses Hypermedia Issues................................................51 Document Objects Used in HMP DTD.........................................................65 13 3 ,3 4 ,
xi
List of Figures Page
FIGURE 1: FIGURE 2: FIGURE 3: FIGURE 4: FIGURE 5: FIGURE 6: FIGURE 7: FIGURE 8: FIGURE 9: FIGURE 10: FIGURE 11: FIGURE 12: FIGURE 13: FIGURE 14: FIGURE 15: FIGURE 16: FIGURE 17: FIGURE 18: FIGURE 19: FIGURE 20: FIGURE 21: FIGURE 22: FIGURE 23: FIGURE 24:
The Layers of the Dexter Model Traversal Paths for Aggregate, Member, and Correspondent Traversal. ...46 Comparison of Dexter Layers and HyTime Modules ................................54 Multimedia Document Structure Corresponding to the HMP DTD...........64 Diagramming and Modeling Techniques...................................................74 Code for a Small HyTime Document ........................................................83 SGML Parsed Document Diagram Key.....................................................84 SGML Parsed Document Diagram for Figure 6........................................85 SGML DTD Diagram Key.........................................................................85 SGML DTD Diagram for Figure 6 ............................................................86 HyTime Meta-DTD Diagram Key .............................................................87 Relevant Portion of the HyTime Meta-DTD Diagram...............................88 HyTime Document Construct Diagram Key..............................................89 HyTime Document Construct Diagram for Figure 6.................................90 HyTime-defined Document Model Diagram Key......................................91 HyTime-defined Document Model Diagram for Figure 6 .........................92 Meta-HyTime Hyperlink Construct Diagram Key ....................................94 Meta-HyTime Construct Diagram for Figure 6 .........................................96 HyOctane HyTime Engine Design ..........................................................100 Partial HyOctane Class Hierarchy ...........................................................101 Partial HyOctane Engine and Toolkit Class Hierarchies for HMP..........107 HyOctane in the Distributed Multimedia Environment Architecture......121 Document Structure for Conventional Scripting Languages ...................126 A HyTime clink and its HTML Conversion ............................................149 8 6
2 3
13 3 ,3 2 ,3 4 ,
xii
xiii
1
I Introduction This introduction first gives a brief overview of the thesis work and its main results. Then the methods used for the thesis research are described. A discussion of related work is provided. Finally, an overview of the thesis’ work is given. This overview briefly describes these problems and their solutions and refers to the sections of this thesis which describe these solutions in full detail.
1.1 Context This thesis provides an evaluation and a number of new results related to a formal model for hypermedia systems. The model used in this research, Hypermedia Time-based Structuring Language (HyTime)81, encompasses the features of hypertext and hypermedia systems that were developed in the 1980s. These systems are typically classified as second generation hypermedia systems. HyTime is more comprehensive than any existing hypermedia model, and is therefore a suitable basis for hypermedia research. The emergence of hypertext as a fundamental ingredient of information systems has long been anticipated but only recently realized on a large scale, thanks to the spread of the World Wide Web (WWW). However, the hypertext capabilities of the WWW are quite limited when compared to previous second generation systems. Consequently, the support for hypertext in distributed information systems requires further research. Comprehensive hypermedia systems are referred to as third generation systems. No systems today claim to be third generation67,29. The main features of second and third generation hypermedia systems are summarized in Section 2.2.
2 The purpose of this research is to evaluate and extend a leading second generation hypermedia model. This research has the potential to benefit the design and use of distributed hypermedia systems such as the WWW. Consequently, the work is of some practical importance. Further, few researchers in the field of hypermedia have used the formal model HyTime used in this research. Also, examinations of digital libraries and other future applications of the web have revealed a need to provide more structuring of hypermedia information, which HyTime provides. Consequently, the work here is relevant to researchers who are interested in the specification of formal hypermedia models. Finally, because HyTime is an international standard, the work reported here is of potential use to the standards community with respect to future revisions of the standard.
1.2 Methodology This research is part of a larger project to develop a third generation distributed hypermedia system. This work is largely experimental and deals with a variety of issues including authoring, new document models, distribution, extensibility, and scalability. The problems addressed by this thesis were determined to be important issues in using the HyTime document architecture in the context of a distributed hypermedia system with the following features: • integration with distributed multimedia system services • support for hypermedia document models • interoperability with the WWW • efficiency and scalability in a client-server architecture A prototype system, HyOctane, which has existed since 1991, has gone through several enhancement versions. Where possible, the research results described in this thesis were
3 evaluated using the HyOctane system. Specific experiments are discussed where appropriate.
1.3 Background 1.3.1 SGML Standard Generalized Markup Language (SGML)81 specifies the basic structure of documents in general. It defines both individual documents and document classes in which they fit. Document classes are defined by SGML Document Type Definitions (DTDs). Although SGML is not designed specifically for hypertext, it has been used to specify classes of hypertext and hypermedia documents such as HTML and HyTime. SGML facilitates the defining of documents as presentation-independent, enabling the documents to be used in a wide variety of processing contexts. Additional specifications beyond SGML are typically needed to generate any presentation of a document. Document Style Semantics and Specification Language (DSSSL) is a recent ISO standard for defining how SGML documents are rendered to their presentations78. Many commercial systems exist for editing SGML documents and processing them for presentation. DSSSL is a new standard and has not been implemented in any commercial system as of this writing. SGML is described in more detail in Section 3.1.
1.3.2 HTML Hypertext Markup Language (HTML)135 is a document format for basic distributed hypertext that is defined with an SGML DTD. It is the primary format for information available through the World Wide Web (WWW)132. The WWW is a distributed hypertext environment built on the Internet that has received wide-spread acceptance and adoption.
4 There are very many HTML documents and HTML browsers available globally on the web. At the Second World Wide Web Conference, Eric Freese discussed the presentation of SGML documents in general using HTML and the WWW49. Proposals for making HTML be HyTime-conforming have been offered by Elliot Kimber87 and DeRose and Durand43. These two proposals are described and elaborated on in Section 8.2. A few commercial products exist for rendering SGML documents for display on the web. Softquad produces Panorama PRO, which is an SGML editor and web browser that renders SGML documents for the WWW123. HTML is described in more detail in Section 3.2.
1.3.3 HyTime Hypermedia/Time-based Structuring Language (HyTime) is an extension of SGML for specifying the hypermedia structure of documents81. Like SGML, it defines both document classes and instances of these classes. It also maintains SGML’s presentationindependence by defining only the general characteristics of a document’s hyperlinks and numeric orchestration of media events and leaving the specification of navigation and actual presentation up to individual document classes and applications. HyTime provides the foundation for this thesis’s work. Few implementations of HyTime appear in the literature. GMD-IPSI in Darmstadt, Germany, has performed research on using object-oriented database management systems to store and process HyTime documents16,12. Their VODAK object-oriented database system enables the authoring, storing, maintaining, and presenting of SGML- and HyTime-encoded documents. The VODAK architecture for processing HyTime constructs in the database is similar to the model used by HyOctane, which is described in Chapter IV.
5 A handful of HyTime systems are being developed in private industry. TechnoTeacher has produced and distributes a HyTime system called HyMinder. HyMinder is a library of C++ functions that perform the basic tasks of a HyTime engine. These libraries can be used in developing code for applications of HyTime. Softquad’s Panorama PRO provides its SGML documents with a HyTime-conforming representation of links123. The Text Encoding Initiative (TEI) is a project that is establishing an applicationindependent format for the interchange of documents used in academic research76. The primary result of the initiative is a very large SGML DTD defining this format. This DTD is robust and general enough to apply to a wide range of documents. Although the TEI document set is not HyTime-conforming, parts of its DTD are based on, or are designed to be compatible with, some HyTime constructs7. HyTime is described in more detail in Section 3.3.
1.3.4 Hypermedia Implementations Surveys of hypermedia have been published by Nielsen104 and Conklin39. Halasz defines three generations of hypermedia67, with current hypermedia being the second generation. Halasz identifies anticipated characteristics of third generation hypermedia. Systems cited as being of the first generation of hypermedia include NLS Augment47, FRESS, and ZOG. Systems cited as being of the second generation of hypermedia include Neptune42, Intermedia51, and KMS1. Other second generation hypermedia systems are discussed in more detail below. The characteristics of these three generations are discussed further in Section 2.2. Notecards is a hypertext application developed at Xerox PARC67. It uses a basic nodeand-link model for second generation hypertext, where a node is an arbitrary object. Navigation occurs either along hierarchical structures or along hyperlinks, which in
6 Notecards are always navigational in nature. Developed by Halasz, Notecards provides the basis for his discussion of the three generations of hypertext 67. Microcosm is an open and extensible hypermedia architecture developed at the University of Southampton65,72. It allows users to browse and query large multimedia collections. Microcosm is distinguished from many other hypermedia systems by its ability to coordinate the processing of different applications, called filters, in the processing and presentations of its documents. The Microcosm Filter Management System (FMS) can store and process links connecting to information provided by third-party applications as well as connecting to information stored in documents. Microcosm can also orchestrate multiple document viewers for a single presentation. Some hypermedia systems have used databases for storing link information such as IRIS hypermedia services66, the HB1 hyperbase management system119, and Hyper-G96,83,2,97. Hyper-G is a distributed hypermedia system developed at the Institute for ComputerSupported New Media at the Graz University of Technology. Hyper-G has become a popular distributed hypermedia system because of its more robust support for the hypertext paradigm and server management tools than WWW servers. Each Hyper-G server maintains a hierarchical organization of its documents. This hierarchy, based on a hyperdocument node type called a collection, provides a fixed structure for users to locate information. The collection node is not a visible document type, but is hardwired into the server, protocol, and clients. Dynamic views of sets of documents can also be created. The Hyper-G server architecture includes a link database and support for object-access management. Hyper-G maintains a database of links separate from the documents. This makes it possible to provide bidirectional links, so that a user can navigate both in-coming and out-going links. It also makes possible the display of a link map which shows adjacent
7 documents in the web. Bidirectional links are maintained for links which point to remote as well as local documents. The World Wide Web (WWW)132 is by far the most widely adopted current hypertext system. However, it lacks many of the features attributed to second generation hypermedia. It provides a basic hypertext document model and a format for it which is defined using SGML. This format is called Hypertext Markup Language (HTML)135. The HTML document model includes one-dimensional binary hyperlinks which provide the basis for navigation on the web. The WWW also has additional hypermedia services being continually added to it on an ad hoc basis. These services include text and structural search engines such as AltaVista44 and WebCrawler55, topical indices such as Yahoo!136, a portable multimedia scripting language and facility called Java58, and graphical and windowed browsers with extra navigational facilities such as Mosaic 99 and Netscape101. HyOctane is a HyTime engine and processing system developed in conjunction with this thesis33. This implementation allows comparison between hypermedia systems like those just discussed and one that is HyTime-based so that the contributions HyTime representation of documents provides to the field of hypermedia systems can be measured. One primary distinction between HyTime systems and the others described is the focus on the structure of the documents themselves and the separation between this structure and any presentation of the documents. This thesis discusses HyOctane and the empirical evaluation of HyTime it provides.
1.3.5 Multimedia Scripts Scripting languages are special purpose application programming languages. Examples of commercial scripting languages for multimedia applications include OpenScript used in Asymetrix Toolbook, HyperTalk used in Claris’ Hypercard, Microsoft’s VBscript, General Magic’s Telescript, Gain Technology’s GEL, and Lingo used in MacroMedia Director.
8 These languages are designed to permit authors of multimedia documents to add interactive behavior to the application without having to resort to conventional software development techniques. Though scripting languages are procedural languages, the syntax is typically simplified to make script development more accessible. A significant component of most scripting languages is the run-time environment available to the author, which includes access to databases, graphic user interface services, multimedia devices and players, and student answer evaluation and historical records. The Scripted Document System137 is built on the Etherphone system at Xerox PARC. Scripts are used in this system to control paths through multimedia documents. The scripts allows paths to be determined dynamically to build active documents. Sun’s Java programming language and its virtual machine provide an environment for distributing portable presentations, called Java applets, on the WWW58. It is a high-level language rather than a scripting language, but Java performs a role similar to multimedia scripting languages by enabling such portability of presentations. Java is similar to the C++ programming language, but with some restrictions that are intended to make its safe to download, simplify programming, and minimize the chances of programming error. Java is compiled to a virtual machine, the Java VM, which is a 32-bit stack machine. Each Java-aware browser provides an implementation of the Java VM. An HTML document includes a Java applet by reference; the applet is then automatically transferred to the browser for execution when the document is loaded. Java is described further in Section 3.4. In addition to being an elegant programming language, Java introduced a number of important ideas for extending the WWW, including: • A model for secure application delivery • Incremental delivery of applets
9 • Platform-independent delivery of applications JavaScript is a scripting language based on Java. It enables the creation of Java applets using a language that is simpler than Java’s programming language. HyTime does not specifically address the inclusion of multimedia scripts. However, as a format for defining structure around individual objects of document content, it provides for the inclusion of scripts as a potential type of media content. This thesis explores the issues involved in including and processing scripts in structured hypermedia documents.
1.3.6 Hypermedia Design Methodologies A number of research groups have built hypermedia systems, such as Notecards 67 and KMS98. Many of these systems have been concerned with user interface issues and specific hypertext models, and have not addressed the management of hypermedia document structuring information and its associated services. To address these issues, hypermedia design methodologies have been developed to represent general document structuring and processing and to associate general document components with particular presentation activities. Hypertext Design Model (HDM) was created to assist in authoring-in-the-large53. Authoring-in-the-large refers to the system- and implementation-independent depiction of document sets and their presentation styles. It is distinguished from authoring-in-thesmall, which focuses on individual documents and their specific contents rather than on general document models. Authoring-in-the-small also focuses on specific applications for document presentation. HDM is generic and is applicable to any document storage format and any presentation application. It provides a means of depicting and communicating many aspects of the design of a hypermedia system during its development, thus facilitating the development of hypermedia documents and application software.
10 Multiple ways for presenting a single document are provided for in HDM with perspectives, which define the mappings between schema components and their different presentations. Document structure is specified by links, which in HDM define hierarchical structure, join multiple perspectives of single document components, and specify general relationships between document components. HDM provides for the definition of classes of documents and applications, called schemas, individual instances of these classes, called schema instances, and the relationships between them. HDM further distinguishes between a document and its presentation by specifying a hyperbase as a document regardless of its presentation and an access structure as a sequence of possible paths for navigating a document in a particular presentation. The Object-Oriented Hypermedia Design Method (OOHDM)122 extends HDM with constructs relating specific document content and application presentations to document components. It models four areas of hypermedia development: conceptual design, navigational design, abstract interface design, and implementation. Conceptual design is the most presentation-independent phase of hypermedia development. It focuses representing the domain semantics “neutrally”, that is, regardless of the types of user and interface activities. Most of the extensions beyond HDM occur in the remaining three areas of design. During navigational design, an author establishes how the document can be traversed by the user. One, or potentially more, tree-shaped orderings of the document components are defined to represent the set of potential paths a browser can follow through the document. How the document and its navigation structure is presented to the user is specified by the abstract interface design. This specification is made in implementation-independent terms, using interface constructs that could exist on a wide variety of browsers. Finally, the navigational and abstract interface models are mapped into specific presentations in particular hypermedia browsing implementations. Multiple such mappings can be specified to multiple implementation environments.
11 These design methodologies illustrate the importance of communicating the general structure of your document as well as its content to the authors. This thesis explores means of communicating how the HyTime-encoded constructs of the document contribute to its structure.
1.3.7 The Dexter Hypertext Reference Model The Hypertext Abstract Machine (HAM) was among the first attempts to model hypermedia systems abstractly rather than create an architecture for one particular system37. This reference model provided a basis for describing individual systems and comparing them to each other. It placed structure around its file system content in terms of contexts, nodes, links, and attributes. HAM also specifies the relation between this structure and the activities of the applications presenting the documents. The Dexter Hypertext Reference Model specifies significant abstractions shared by typical hypertext systems to provide a common reference for discussing and comparing these systems68. The Dexter model has received much attention because of its development over a long period of time by a large panel of acknowledged hypertext experts. It is designed to aid in the general discussion of different hypermedia documents and applications and is not incorporated into any particular environment. The Dexter Hypertext Reference Model is discussed in more detail in Section 2.3. The Dexter model has been applied to several hypermedia systems and model extensions. Some of these applications and extensions, and the issues surrounding their creation, are discussed in a Communications of the ACM issue on hypermedia61 and at the 1996 ACM Conference on Hypertext6. The Flag taxonomy extends the Dexter model to classify different open and integrated hypermedia environments110. Systems can be categorized based on how well they can integrate different applications and formats and how well they can be tailored for
12 particular presentations styles. Another extension accounting for links embedded in documents as well as external links has also been proposed60. Both these extensions are intended solely to act as the basis for comparison and have not been used in the implementation of particular systems. Dexter has been used as an intermediate interchange format between the Intermedia and KMS systems to test its practical applicability91. This work concludes that the model should not be used specifically as the basis for interchange, and that hypermedia metamodels such as Dexter serve best as enabling general comparisons and do not apply well to implementing specific systems. However, other work has successfully applied the Dexter model to particular implementations. One application of Dexter is an engineering cooperative design environment called the DeVise project62,63. It enables large scale collaborative work. An object-oriented database is used to process objects whose structure is based on Dexter. This project raised the issues of document components being composites and containing data that is processed by different applications. Extensions to Dexter addressing these issues were designed and implemented. Another extension of the Dexter model is the Amsterdam Hypermedia Model (AHM), developed at the Centrum voor Wiskunde en Informatica (CWI)69,70. AHM extends the Dexter model into multimedia by adding to it components of the CWI Multimedia Interchange Format (CMIF) model. These extensions provide the Dexter model with more constructs for the document’s rendered presentation and for generating this presentation from the storage layer representation. An AHM-based visual authoring tool called the CMIF editor (CMIFed) has been developed at CWI, providing another Dexter-based hypermedia environment. The Amsterdam Hypermedia Model is discussed in more detail in Section 2.3.
13 CMIFed is an AHM-based authoring environment developed at CWI69,70. The document format is derived directly from the AHM superset of the Dexter model, making Dexter-based analysis of the format trivial. CMIFed is a visual authoring tool that can process a hypermedia document for editing in three ways. First, it can play the document as it would appear to a browsing user. Second, it can present the hierarchical view of a document, enabling the author to perceive and edit the document in terms of its hierarchical structure. Finally, the channel view of CMIFed displays and provides modification of the channels in a document and their timelines. CMIFed provides a visual authoring of documents in terms certain aspects of the Dexter and the Amsterdam Hypermedia models.
1.3.8 Digital Libraries A digital library is a system that replaces and extends the services of the traditional library through the digital electronic medium. Many digital libraries have been appearing on the WWW, providing organized electronic access to large collections of documents. An overview of the field of digital libraries is provided in a recent issue of Communications of the ACM focussing on the topic48. Most documentation currently on the web exists as part of small collections and do not provide many challenges for structuring and access. Digital libraries, on the other hand, have documents that exist in the context of very large collections. As more digital libraries appear on the web, more documentation will exist that provides challenges beyond that of typical current web usage. A group report on the requirements of digital library systems has listed as a critical requirement the specifications for document markup, links, and interchange conventions54. This work cites Dexter, SGML, and HyTime as significant means of addressing this need. Other work also argues the importance of structuring for digital libraries50. The particular aspects of structure discussed here are multiple media types, complex relationships between objects, abstracted levels of authoring, processing of
14 structure as well as content in determining presentation, and the modeling of the user interface. These are structural aspects addressed by HyTime.
1.3.9 General Sources of Information There a number of regular conferences and journals that present research on hypermedia. Two annual conferences focussing on the topic are the ACM Conference on Hypertext6 and the European Conference on Hypermedia Technology. The annual ACM International Conference on Multimedia covers a broad range of topics that frequently include work on hypermedia. Digital library projects are discussed at the annual ACM International Conference on Digital Libraries5. Research on and applications of WWW document processing are presented at the International World Wide Web Conference, which meets several times a year. Papers from the WWW conferences are available at a web site sponsored by the World Wide Web Consortium (W3C)132. W3C publishes its developments in web technology and protocols in the World Wide Web Journal, whose back issue articles are also available at the consortium site134. Also, the journal Hypermedia publishes hypermedia research on a monthly basis.
1.4 Overview This thesis explores the implementation of second generation distributed open document hypermedia processing and presentation in the context of existing models. Much of this work has been discussed in previous publications28,33,29,30,32,31,34,114,115,116,117,113. The following results of this research are summarized in later sections of this thesis: • an analytical evaluation of HyTime (Chapter III) • an empirical evaluation of HyTime (Chapter IV) • a new graphical modeling paradigm for HyTime (Chapter V)
15 • a method for automatically generating interactive presentation software for HyTime document models (Chapter VI) • an examination of ways to support scripting languages in HyTime document models (Chapter VII) • a detailed analysis of relating and interchanging Hypertext Markup Language (HTML) and HyTime (Chapter VIII) The empirical basis for much of the work done for this thesis involves the HyOctane HyTime engine design and implementation113. Recent extensions to the engine include modifying it to operate within a distributed client/server environment instead of basing its data storage and processing in a local database management system30. Design extensions were made to facilitate application generation, discussed in Chapter VI, and script incorporation, discussed in Chapter VII. Also, an alternate presentation format has been written for http clients for display on the World Wide Web 45. This thesis’ contribution to these efforts was in the creation of the document models and instances involved and in designing the processing environments. The bulk of the coding was performed by John Rutledge and Chetan Gopal of the Distributed Multimedia Systems Laboratory. This project and a sample HyTime application defined within it are presented in Chapter IV. Modelling techniques have been developed that facilitate the authoring of documents, the structuring of document classes, and the creation of browsing applications 53,52. These have proven to be helpful for working with large and complex document structure classes and relating them with their instantiations in the documents, their presentation by browsers, and their general hypermedia semantics. As an SGML architecture, HyTime further complicates this process because corresponding constructs between the architectural level and the document class level need to be distinguished from each other and also related to document instances and presentations in terms of both raw structure
16 and the hypermedia semantics represented. This thesis introduces a modelling diagram technique that facilitates the authoring and application generating process for HyTime systems118. It is a bottom level version of the more general modelling techniques that is applied specifically to HyTime, accounting both for HyTime as an architecture and as having constructs with hypermedia semantics. This modelling technique is discussed in Chapter V. The creation of applications that process and present open hypermedia documents typically follows the design of those documents’ models. Given an environment and architecture for general open hypermedia processing that has a set of presentation and interaction scriptware for browsing applications, the creation of applications for document sets often follows the same pattern. Once this pattern is understood for a particular environment, the generation of applications within it can often be partially automated. To explore the automatic generation of hypermedia applications, a design for such generation in the context of the HyOctane environment has been created31. This design is discussed in Chapter VI. The application generator works for documents defined using HyTime and presented in the HyOctane hypermedia environment. The inclusion of scripts in hypermedia documents provides the author with much power in sculpting the document’s presentation. Chapter VII discusses the inclusion of scripts within HyTime documents and HyTime processing environments. This discussion includes: • issues of integrating scripts in hypermedia environments • mechanisms for extending HyTime documents with scripts • ramifications for the HyTime processing environment • implementation experience
17 Chapter VIII establishes a relationship between HyTime and HTML as the basis for a hypermedia environment that uses both. This relationship enables the use of HTML browsers to present HyTime-encoded documents114. It also enables the representation of existing HTML documents with HyTime constructs so they can be included in a more general HyTime processing environment117.
18
II Models for the Classification and Evaluation of Hypermedia This chapter reviews the models for the classification and evaluation of hypermedia that are utilized in this thesis. These include the Halasz classification and the Dexter Hypertext Reference Model. Section 2.1 introduces some key hypermedia concepts. Section 2.2 describes the Halasz classification of the three generations of hypermedia. The Dexter Hypertext Reference Model is discussed in Section 2.3.
2.1 Concepts and Issues Hypertext is the establishment of associative structure for documents. One consequence is that hypertext can provide access to document components that is not strictly linear, such as with tables of contents and indices. The term hypertext was coined by Ted Nelson100. The original concepts of hypertext are attributed to Vannevar Bush 35. Hypermedia is the application of hypertext concepts to documents containing all media types including text. All the text and media objects that make up a document’s content are placed in a structure assigned to the document. The context of the media content in this structure affects the order and context of their presentation to the user when the document is processed. A document model is a generic structure or template which a document conforms to. A class of documents may share a general structure that an individual document’s structure fits. This enables authors to work with categories of documents, as well as with specific documents. A hypermedia document model is a generic model for documents that contain hypermedia structure or multimedia content. Open hypermedia document systems allow the unified processing of documents fitting any model. An open document environment
19 enables an author to create a new document model without needing to create or adapt application software for processing and presenting it. Presentation-independence is a desirable characteristic of document structures in open hypermedia environments. A document’s presentation has a structure that is potentially distinct from the structure of the document itself. Further, a single document can have many different presentations with different structures. When a document’s structure reflects that of a single potential presentation, its processing becomes limited to that presentation style. When a document is structured independently of any potential presentation, it is more freely processed into a wider variety of presentations. Consequently, in some formats such as SGML, document structure specification excludes presentation characteristics. However, rendering a presentation-independent document structure into a given structure of presentation requires more processing than a document whose structure more directly mirrors that presentation. Because of this, presentation independent processing environments benefit significantly from providing a unified means of rendering any document of any model into any style of presentation. Authors in open presentationindependent environments create not only documents and their models but also the specification of how documents of each model are rendered to each style of presentation.
2.2 The Halasz Categorization of Hypermedia The identification of generations of hypermedia systems is due to Halasz67. These generations are listed and described in Table 1. First generation hypermedia focus on the presentation of text and the interlinking of text document components. Such systems typically present documents as text, with certain portions of the text highlighted. Selection of each of these highlighted portions by the user causes another segment of text to be presented. Halasz puts existing contemporary systems in the category of second
20 TABLE 1: Issues for Hypermedia
3rd Generation Open Distributed
29
Third Generation
67
Second Generation
67
1st
67
Issue
Description
Meta-linear Structure Over Text
• Non-linear relationships established between text portions • Typically navigational links
Node Typing
Non-text node types (i.e. graphics); cannot contain other nodes
Simple Link Structures
Binary; bidirectional; not typed; whole node or point anchors
Hierarchical Support
Browsers display diagrams aiding navigation through networks
GUI/Windowed Interface
Multiple windows; links activated by mouse
Extensibility
• Versatile document structure does not prevent flexible use • Programmer’s interface to applications for extensions
Simple Text Query
Slow, full-text string search
Simple Distribution
Multi-user; limited concurrency control
Integration of Search and Query Functionality
• Integration of information retrieval and DBMS facilities • Pattern matching languages for hypergraph manipulation
Composite Node Types
In addition to content nodes, has container or collection nodes
Virtual Structures over Node Collections
Document structure does not become outdated with changes in the network
Computation over Hypermedia Networks
Integrated computational engines available to the application
Versioning of Nodes and Subgraphs
Maintaining change history at both node and graph level; version identification as an attribute in search and query
Support for Collaborative Work
Support for shared access to hypertext, and group protocols
Extensibility and Tailorability
Easier customization of hypermedia system by end user
Open Hyperdocument and Open Hyperlinking Architecture
• Support the delivery of any document type • Supporting linking between any application mediated content or content type • Use of content-based addressing • Integration with hyperapplications
Mobile and Distributed Objects
• The ability to deliver active behavior integrated with static documents • Allow objects to be dynamically deployed anywhere in the system
Interactive Time-based Hypermedia Document Models
• The ability to represent and deliver hypermedia documents which include rich interactive time-based hypermedia presentation • Integration with continuous media system services
Security
• Transaction security; safety of mobile objects
21 generation. Second generation hypermedia extend the first generation by incorporating more non-text media (primarily graphics) and by using windowed display environments. These also provide navigation overviews to utilize the increased graphics and window capabilities for aiding the user in traversing the documents’ links. Halasz also identifies seven areas of functionality which third generation hypermedia systems should address, which are listed and described in Table 1.
2.2.1 First Generation Hypermedia The primary structure of any document is linear in nature. Words appear, one after another, from the beginning to the end of a document. With the advent of hypertext, document structure became analyzed in terms that went beyond this linearity. A passage in one part of the document could be associated with a non-adjacent passage. Examples include the association of a line in a table of contents with the beginning of a section and of an index term with its discussion in the main text. With electronic documents, such associations could be presented as the first passage being selectable by the user interface, and by having such a selection display the second passage. Such presentations are typical of first generation hypermedia. These presentations rarely include graphics or other nontext
media,
and
they
pre-date
the
use
of
windowed
interfaces.
2.2.2 Second Generation Hypermedia Second generation hypermedia extends the first generation by adding non-text media and the use of windowed interfaces. The following subsections describe the main features of second generation hypermedia. Many of these features are derived from the proliferation of computer graphics and graphical user interfaces.
22 2.2.2.1 Node Typing A node is a portion of document content. It is the atomic construct of document structure. Node typing extends first generation structuring by allowing nodes to be of different types. Typically, each type represents a different medium along which a node’s content is conveyed. Node typing helps extend the types of media a document can contain by labelling document content by its medium. Nodes cannot contain other nodes, so there is no hierarchical structure. All nodes are considered to exist on the same level.
2.2.2.2 Simple Link Structures The association of document portions is extended in second generation hypermedia with the link structure. The document portions associated with each other are called anchors. Second generation links have the following characteristics: Binary: There are always exactly two anchors per link. Bidirectional: The association between the anchors can be equally considered starting with either node. Not Typed: Unlike second generation nodes, links are not distinguished from each other by the assignment of different types. All links are considered to have the same meaning and define the same association. Whole Node or Point Anchors: A link’s anchor is either an entire node or a point between two nodes. An anchor cannot be a portion of a node.
2.2.2.3 Hierarchial Support Documents in many hypermedia networks exist in hierarchies because containers exist holding documents and other containers. An application will often traverse such a network hierarchy to access a particular document within it. Second generation hypermedia
23 systems offer special support for the traversal for such hierarchies, both for the applications and for the user.
2.2.2.4 GUI/Windowed Interface When graphical user interfaces (GUIs) and windowed screen displays were developed and adopted, hypermedia was extended to utilize them. In the second generation, document presenters can presented multiple windows to allow the user to access multiple documents simultaneously. Also, point-and-click interfaces allow users to specify links for traversal more quickly than in text interfaces.
2.2.2.5 Extensibility Documents are regularly added to and updated within libraries. Also, new browsers can be developed to view previously existing documents. It is important that a hypermedia system be extensible. Documents should be easily added to and modified within the system without disrupting previously documents and their display. Also, new techniques for presenting documents should be easily written in new browsers or added to existing browsers without disrupting the display of previously existing documents.
2.2.2.6 Simple Text Query A document’s structure and its links often do not provide the user with enough information to access the desired portion of the document. In such cases, the user may wish to perform a query on the document’s components to find a particular location within it. In the second generation, the mechanism for accomplishing this that is available to the user is slow and simple: a string search through the document’s contents. A string is specified, and the location of each copy of the string in the document is displayed for the user.
24 2.2.2.7 Simple Distribution The basic facilities for providing simultaneous access of a document network to multiple users is provided in the second generation of hypermedia systems. Extra features beyond the basic requirements are typically not offered.
2.2.3 Third Generation Hypermedia Third generation hypermedia extends the second generation by incorporating seven features identified in response to second generation limitations67. These seven issues are discussed in the following subsections. 2.2.3.1 Integration of Search and Query Functionality The third generation provides more capability for the user to access a document portion using information beyond that provided by a document’s links. As with the previous generation, the document content can be searched for string matches. Here, however, a document’s structure can also be queried against. The user can, for example, look for a particular word within a certain node type. More complex patterns of structural components and their content can also be queried for. 2.2.3.2 Composite Node Types Here document nodes can contain other nodes as well as media content. This provides for a hierarchical structure of the document itself, as well as enabling a group of document nodes to be referred to as a single entity. This adds richness to the structures that authors can provide their documents. It also extends second generation whole anchor link specification by allowing anchors to be groups of nodes. 2.2.3.3 Virtual Structures over Node Collections Many aspects of the structure of a document or library of documents change over time. References to a document or document portion can become obsolete if defined in a
25 manner that is susceptible to such changes. Virtual structures are defined to be immune to these changes.
2.2.3.4 Computation over Hypermedia Networks Second generation hypermedia systems offer a browsing user access to documents created by human authors. Third generation systems can automatically compile a document to present to the user based on information the user provides. This action is called computation over hypermedia networks. The user can enter a request in the form of a query, and a computational engine within the hypermedia system searches the network of documents to find the information requested by the query and format it for presentation to the user.
2.2.3.5 Versioning of Nodes and Subgraphs Documents are regularly modified by their authors. It is often useful to record each modification of a document as a version and enable authors to access any previous version of a document as well as the most recent one. This enables authors to undo changes that were discovered later to be undesired. Versioning also facilitates the updating of other documents in the library to account for the changes in the modified document. This usefulness is extended when versioning is applied to individual nodes rather than simply to the entire document. Third generation hypermedia systems provide node-level versioning to their documents.
2.2.3.6 Support for Collaborative Work Multiple authors often work simultaneously in creating new versions of documents in a collection. Such collaborative work presents difficulties not present in single-user systems. These include the need for • preventing simultaneous multiple changes to the same document portions
26 • notification of teammates when changes are made • annotation by one author of content created by another • communicating the coordination between authors of changes made and to make Third generation systems provides facilities that meet these needs.
2.2.3.7 Extensibility and Tailorability Hypermedia systems often structure their documents and their presentation in generic terms. This provides needed flexibility. However, particular document sets and document browsers typically use more specific structures. Producers of these document sets and browsers are assisted in this process by hypermedia systems that • do not limit the applicability of their document structures and presentation styles • provide tools for relating the general composites of the document structure and browsing schemes with particular presentation styles. Hypermedia environments are extensible and tailorable if they have these characteristics. Third generation systems are extensible and tailorable.
2.2.4 Open Distributed Hypermedia Systems Distributed hypermedia is the provision to a group of users of logically transparent access to heterogeneous hypermedia information repositories that are distributed across local and wide-area networks29. The minimal access mode is browsing via associative addressing and authoring; searching, annotating, and collaborative access are examples of other modes that may be supported. An important aspect in the design of distributed hypermedia systems is
27 scalability to large amounts of multimedia information, large numbers of users, and large numbers of repositories. Halasz’ issues involve enriching the hypertext paradigm, but do not address its use in a network of distributed heterogeneous hypermedia systems. Consequently Buford and Rutledge29 add to this list four more issues, which are listed and described in Table 1. Third generation distributed hypermedia systems permit interoperability between servers providing heterogeneous document types. These systems are open not only to a specific category or type of document model, but provide an open hyperdocument architecture. Using distributed object management facilities and an open hyperlinking mechanism, hypermedia documents will link to or include embedded hyperapplication content as well as arbitrary stored media. Powerful mobile object virtual machines permit a range of dynamic behavior to be delivered, from simple interaction to agents to full applications. Content addressing and search capabilities include content-based retrieval for multimedia data types. Hypermedia document services are integrated with multimedia system services for real-time scalable delivery of temporally composed multimedia content. The system provides security services for access control and authentication to support transactions, safe mobile objects, protection of intellectual property, and privacy, in addition to the collaboration support described by Halasz.
2.2.4.1 Open Hyperdocument and Hyperlinking Architecture The concept of hyperlinking is strikingly simple: the ability to create and traverse associations between arbitrary information elements. Despite the generality of this paradigm, few if any hypermedia systems have been developed which can claim a universal link service — the ability to support hyperlinking between any two objects, no matter what the content model, application interface, or computing environment. The ability of a hypermedia system to be inclusive of arbitrary information sources is a
28 powerful feature for users. With this ability, associative access becomes an intrinsic facility of the user’s desktop. Without it, the user must consider and maintain separate spaces of information access. 2.2.4.2 Mobile Code and Distributed Objects A distributed hypermedia system forms an information structure over which useful computation can be performed for users, including search, dynamic document creation, event generation, generalization, and discovery. The efficient performance of these activities depends on the available computing and network resources throughout the system. Integration of the hypermedia system with distributed computing environments such as those described in the previous section provides a flexible foundation from which such computation can be performed. Mobile code refers to the ability to relocate applications or applets for execution at any server or client in a distributed system, possibly for the purpose of performing computation locally or for managing resources. 2.2.4.3 Hypermedia Document Models Today’s distributed hypermedia systems have simplistic use of hypermedia data types both because of the limited semantics of document models and because of the lack of underlying system support for delivery of hypermedia information. Integrating a hypermedia document model with a hypermedia system increases the representational power of the system and the generality of the system design. Putting hypermedia modeling semantics in the hypermedia system allows many hypermedia applications to leverage this functionality and adds generality to the system design. For discussing hypermedia document models in this thesis hypermedia semantics are divided into the following three areas: • hyperlinking • locating
29 • scheduling This division is based on the hypermedia document model provided by HyTime81, which is discussed in Section 3.3. Hyperlinking: Hyperlinking is the establishment of relationships between document components that are not implicit in the document’s hierarchical structure. It provides the basis for meta-linear structuring. These relationships are not restricted to being navigational in nature, although this is the typical significance of hyperlinks in early hypermedia. Any type of relationship with any significance can be established with hyperlinks. Locating: In order for a hyperlink to be established between document portions, what those portions are must also be established. Locating is the ability of a hypermedia system to define a portion of document or a segment of any data so that it can be referred to from elsewhere in the system’s library of documents. This includes the capacity to define as a located object a data whose boundaries are not delimited as a single component within a document’s hierarchical structure. Scheduling: Hypermedia documents rely on measurements and numbers to describe many aspects of their components. Although typically referring to the timing of document object presentation, scheduling in the context of this thesis is the use of numbers and measurements in general to describe document objects and their relationships to each other. Scheduling can result in the specification of when events are to occur on a timeline. Other examples of applied scheduling include the placement of graphic display windows on a workstation, the generation of geographic maps, and the creation of indexed tables.
30 2.2.4.4 Security The requirements for security in distributed hypermedia systems are varied because of the wide range of activities for which the system might be used. The following are examples of services that open distributed third generation hypermedia systems should be able to provide. 1. Confidentiality: Private information is delivered over insecure networks only to those who are authorized to access it 2. Intellectual property rights: Publishers wish to deliver licensed material and desire a technical solution to preventing unauthorized redistribution 3. Authentication: To enforce access rights, systems wish to verify the identity of the user 4. Privacy: Users wish to access information without disclosing their identity, or may wish that information about their browsing history, search, and information access is restricted. 5. Access rights: Groups of users may use the system for collaboration, requiring synchronized shared access and different modalities of access, such as read, write, update, annotate, create, and delete. Search and browsing of documents by users might be limited to only publicly accessible files, or those that have the proper access level 6. Transaction integrity: The hypermedia system is integrated with a database system which uses transactions when performing updates to guarantee atomic, consistent, isolated, and durable results 7. Authentication, Secure communication: The hypermedia system is used to carry financial transactions which require secure data entry, secure transmission, secure
31 authentication of all parties, etc. 8. Non-repudiation: A user cancels a financial transaction and requires verification that the cancellation request was received. 9. Safety: Applets or agents can be downloaded to servers to perform computation on behalf of users and extend the functionality of the hypermedia system. Mobile objects might intentionally (viruses) or inadvertently lead to denial of service for other users. 10. Management: The system’s security policies must be unambiguously specified. The policies must be applied consistently as new users, information, and services are added to the system. The distributed hypermedia system should base its security facilities on the security services provided by the operating system software wherever possible. Additional services needed beyond that of the operating system to satisfy the above examples should also be provided by the distributed environment.
2.3 The Dexter Hypertext Reference Model The Dexter Hypertext Reference Model specifies significant abstractions shared by typical hypertext systems to provide a common reference for discussing and comparing these systems68. It divides hypertext processing into three layers: the run-time, storage, and within-component layers (see Figure 1). The run-time layer is concerned with the final presentation of documents, including how the user interacts with the interface and what activities occur during presentation. The within-component layer contains the nodes, or the atomic containers of the media objects inside a document. The storage layer provides the hypertext structure of a document. The storage layer is related to the withincomponent layer through anchoring, which specifies where in the hypertext structure the media content nodes are placed. The presentation specifications define how the contents of
32 the storage layer are to be processed when the user browses through them. The Dexter model focuses on the storage layer and its connections with each of the other two layers, it maintaining independence from particular presentation applications and source document formats. The Dexter model is designed to aid in the general discussion of different hypermedia documents and applications and is not incorporated into any particular environment. One significant characteristic of the Dexter model is its distinction between links and anchors. In Dexter, links establish relationships between objects in the document. They are an integral part of the document’s general structure and reside in the storage layer. Anchors in the Dexter model encapsulate individual portions of document content, specifying what the ends of a link are. Dexter anchors reside in the anchoring layer, forming the bridge between the storage and within-component layers. They enable storage layer links to establish linkends in a manner that is independent of the particular network distribution and file systems that store and deliver their content. The Amsterdam Hypermedia Model (AHM) extends the Dexter model into multimedia by adding to it components of the CMIF multimedia document model69,70. CMIF channels are implemented, which provide independently controlled presentation of document components. Composites for defining synchronization are added so that
Run-time
Presentation Specifications
Storage
Anchoring
Within-component
FIGURE 1: The Layers of the Dexter Model68
33 different components can be related to each other within the context of a timeline for presentation. Further, the model can describe how the context of a link’s activation is to be affected by the link’s traversal. And finally, AHM specifies the behavior of the presentation traversal of links that are to nonpersistent data. These extensions provide the Dexter model with more constructs for the document’s rendered presentation and for generating this presentation from the storage layer representation.
2.4 Summary This chapter detailed some key hypermedia issues, the Halasz generational classification scheme, and the Dexter Hypertext Reference Model. The next chapter uses this material to classify and evaluate the hypermedia document format languages used in this thesis.
34
III Review of Hypermedia Format Languages This chapter reviews the document format languages SGML, HTML, HyTime, and the hypermedia presentation programming language Java. The document format languages are classified and evaluated using the models discussed in Chapter II. The issues of open documentation have been addressed by Standard Generalized Markup Language (SGML), which is presented in Section 3.1. The most widely used distributed hypermedia system today is the World Wide Web (WWW), whose documents are primarily represented using Hypertext Markup Language (HTML). HTML is discussed in Section 3.2. Second generation hypertext and hypermedia systems have been reviewed by a number of authors39,104. Many of the key ideas of current hypermedia systems have been incorporated into the document format called Hypermedia/Time-based Structuring Language (HyTime)81. An overview of HyTime is provided in Section 3.3, focussing on the constructs that appear in this thesis. These document formats are analyzed in terms of the Halasz classification of Hypermedia. Table 2 provides an overview of which Halasz hypermedia issues are addressed by which of the three main document models used in this thesis. How each document format meets these requirements is discussed in the sections ahead. HyTime is also analyzed and evaluated in terms of the Dexter Hypertext Reference Model. Finally, Java is described in Section 3.4, and correlations between Java and HyTime are given.
3.1 SGML Standard Generalized Markup Language (SGML) is a syntax for representing the general structure of electronic documents. SGML provides a small set of atomic constructs that group document content and attach descriptive labels to it. SGML also
35 specifies, using Document Type Definitions (DTDs), the general document structures of classes to which individual documents can conform. By defining both documents and their classes, SGML addresses the needs of open document processing. Generic, presentationindependent structures can be defined for documents, and classes for such documents can be created.
SGML defines a set of constructs for building the markup that is placed with the content of a text document file. This markup delimits the text content into containers called elements. In addition to text, elements can contain other elements or a combination of other elements and text. An element also has a generic identifier (GI) that assigns a name TABLE 2: Models Addressing Hypermedia Issues Issue
SGML
HyTime
HTML
Meta-linear Structure Over Text
yes
yes
yes
Node Typing
yes
yes
yes
Simple Link Structures
no
yes
yes
Hierarchical Support
yes
yes
no
GUI/Windowed Interface
no
no
no
Extensibility
yes
yes
no
Simple Text Query
no
yes
no
Simple Distribution
no
no
no
Integration of Search and Query Functionality
no
22 yes
no
Composite Node Types
yes
yes
Virtual Structures over Node Collections
yes
yes
Computation over Hypermedia Networks
no
no
22
22
22 22
no no no
Versioning of Nodes and Subgraphs
no
no
Support for Collaborative Work
no
some
Extensibility and Tailorability
yes
yes
Open Hyperdocument and Open Hyperlinking Architecture
yes
yes
no
Mobile and Distributed Objects
no
some
no
Interactive Time-based Hypermedia Document Models
no
yes
some
22
no no no
36 to an element describing its type. SGML also defines attributes that are associated with and describe elements. Each attribute has a name and a value. Two particularly useful types of attributes are the unique identifier (ID) and the unique identifier reference (IDREF). An ID attribute gives its element a unique name within the document. An IDREF attribute has as its value the ID of some other element in the document, thus representing a reference to the element. Together these constructs provide the meta-linear and hierarchical structure of a document and descriptive information about portions of the document. External entities in an SGML document are references to data stored outside the document, typically in a file located on a file system on a network. This enables any type of data, including binary, located anywhere, to be included in a document. SGML-defined notations augment this capability by assigning names to segments of data, including external entities, that describe what format that data is in, and thus suggesting how it is to be processed for personation. External entities and notations, along with generic identifiers, provide the basis for second generation hypermedia node typing. Sets of documents for use with particular applications are defined by SGML Document Type Definitions (DTDs). Each SGML document must be associated with a particular DTD. A DTD defines a set of element types that can be used in conforming SGML documents. A content model is defined for each type saying what elements of that type can contain. Content models are defined mostly as regular expressions of element types, saying that content can be certain patterns of element sequences. A DTD also defines the set of attributes that can be used with each element type. SGML implies the functionality of the tool which processes it: the SGML parser. Applications of SGML use this tool for the initial processing of documents. This processing generates information about the application-independent and presentationindependent structure of a document. This information is further processed by an
37 application to determine how the document is to be presented. Applications define their own SGML-encoded constructs and apply presentation semantics to them. While SGML parsers are developed for use by multiple applications, the applications themselves must be individually crafted by developers. Passing a document instance through an SGML parser with its DTD enables the parser to determine the validity of the document format. A parser extracts information from the DTD as well as the document proper and provides this information to the application.
3.1.1 Halasz Classification of SGML While SGML is a format for general documentation and not hypermedia, it has properties that can be applied to addressing certain issues of hypermedia. HTML and HyTime, both hypermedia document format languages, use SGML as their base. SGML constructs for general documentation are the building blocks for HTML’s and HyTime’s constructs for hypermedia documentation. Table 2 shows which Halasz generational hypermedia issues are addressed by SGML. Table 3 shows what SGML facilities address these issues.
TABLE 3: How SGML Addresses Hypermedia Issues Issue
How SGML Addresses
Meta-linear Structure Over Text
Elements, unique identifier referencing
Node Typing
Generic identifiers, external entities, notations
Hierarchal Support
External entities
Extensibility
Open document philosophy
Composite Node Types
Hierarchical containment of elements
Virtual Structures over Node Collections
External entities
Extensibility and Tailorability
Open document philosophy
38 SGML supports first generation hypermedia with ID referencing, which forms the basis for establishing relationships between document components that are not derived from the document hierarchical containment structure. Second generational node typing is provided by SGML generic identifiers, which declare document elements as being of different types. Hierarchical support is provided by external entities. The open document philosophy behind SGML’s use facilitates extensibility by providing a common framework and integrability for documents of different document classes. SGML addresses a few third generation hypermedia issues. The defining of composite node types is provided by SGML DTD content models, which state for each element type what types of elements they can contain. SGML external entities provide some specification of virtual structures over node collections by enabling documents to refer to each other and to other external non-SGML data collections. The extensibility and tailorability of SGML-based hypermedia systems is enabled by SGML’s open document philosophy. This philosophy also provides the basis for addressing open hyperdocument and open hyperlinking architecture.
3.2 HTML Hypertext Markup Language (HTML)135 defines a format that is specified with an SGML DTD. HTML consists of a particular set of markup tags that can be interspersed with text document content to delimit, structure, and describe that content. The primary HTML tags that influence textual display are headers and paragraphs. HTML also provides in-line inclusion of document objects of non-text media.
3.2.1 HTML Hyperlinking Constructs The interactive behavior of HTML documents is defined by HTML anchor (a) tags. An anchor specifies a portion of the document to be part of a link: either a link beginning, a link end, or both.
39 If the anchor tag specifies a hypertext reference (href) attribute, then the anchor is the beginning of a link, and the value of the attribute specifies the link’s end. The contents of such elements are considered hotspots. Their selection by the user causes another document or document portion to be displayed. This relationship between a hotspot and the display its selection creates is typical of hyperlinks. Such links in HTML are onedirectional: selecting the hotspot brings up another document segment, but that segment does not necessarily act as a hotspot back to the original location. Also, each anchor can only link to one other document object. If the anchor tag specifies a name attribute, then other anchor href attributes can use that attribute’s value to specify this anchor as a link end.
3.2.2 HTML Locating Constructs Although HTML uses SGML, it uses neither entities to reference external files nor SGML IDs to reference portions of the same document. There are attributes in HTML, however, that use HTML-specific schemes for such references. These attributes can have their values in Uniform Resource Locator (URL) notation. This notation specifies the locations of document objects on the WWW. URL notation provides a common, unified means of accessing files and information stored across the web. One of these attributes can also have as its value the pathname of a file relative to the current document’s pathname on its file system. In either case, the located file is considered to be the object referenced. One such attribute is the source (src) attribute of the in-line image (img) element type. An img element specifies the occurrence of an in-line image within the display of a text document. The src attribute locates the file containing that image. The href attribute of the anchor element also performs such a reference. This attribute specifies what is presented when the element content is selected. A file located by this
40 attribute can be of any medium type, as long as the system on which it is to be presented is configured to present that medium type. If its value is a valid URL, then the ending point of the anchor is some file elsewhere on the WWW. If its value is not a URL and does not begin with “#”, then the value is the name of a local file that is the linkend.
In addition to the value possibilities described above, the href attribute can also reference a portion of an HTML document. Such a portion would be defined as the contents of an anchor element with its name attribute assigned. If the anchor tag specifies a name attribute, then other anchor href attributes can use that attribute’s value to specify this anchor as a link end. An anchor’s name would appear after a ‘#’ character in an href attribute. The ‘#’ and anchor name could appear by themselves to indicate the destination of this link is a named anchor in the same document. Putting a local file pathname or URL specification before the ‘#’ would locate the named anchor in the specified HTML document file.
3.2.3 HTML Scheduling Constructs There are not many scheduling-related semantics in HTML. HTML defines no timelines for presentation and no measured coordinate systems for screen displays. One HTML construct that suggests scheduling semantics is the image map. An image map is an img element that is defined as a hotspot (that is, as the content of an anchor element with its href attribute assigned) and has its ismapattribute assigned. Different portions of an image map can be clicked on to access different objects. The URL accessed by clicking on an image map is the href attribute value followed by a ‘?’ character and two numbers giving the coordinates of the image where the mouse click occurred. The server specified in the URL can then process the address with the coordinate suffix to determine the appropriate document object to return.
41
3.2.4 Halasz Classification of HTML HTML is typically considered first generation hypermedia, though it has a few characteristics attributed to hypermedia of later generations. However, the various facilities on the World Wide Web (WWW)132 provide HTML’s use with more of the later generation characteristics. Table 2 shows which Halasz generational hypermedia issues are addressed by HTML. Table 4 shows what HTML facilities address these issues. Because it is defined as an SGML DTD, HTML provides meta-linear structure over text and second generational node typing. However, also as a DTD, the set of HTML node types is closed. Second generation hyperlinks, which consists of simple binary, onedirectional links, is provided by HTML anchor elements. HTML also represents some of the semantics required of interactive time-based hypermedia document models. One example is the inline image, which defines the inclusion of non-text media, though in HTML only images can be included as such. More complex hypermedia document modeling semantics are provided by HTML image maps, which use coordinate systems to break a single image into multiple active areas. However, most semantics of hypermedia modeling are unaddressed by HTML.
3.2.5 Relating HTML with HyTime This thesis identifies semantic overlaps between HTML and HyTime. These are constructs or composites of constructs in both languages that perform similar, equivalent, TABLE 4: How HTML Addresses Hypermedia Issues Issue
How HTML Addresses
Meta-linear Structure Over Text
SGML DTD-defined document structure
Node Typing
SGML DTD-defined document structure
Simple Link Structures
anchor elements
Interactive Time-based Hypermedia Document Models
• In-line images, image maps • No modeling of time-related processing
42 or identical functions in representing hypermedia aspects of a document. Understanding these overlaps facilitates the creation of hypermedia systems that use both languages for different but complementary aspects of document and presentation processing. These semantic overlaps and their application to coordinating HTML and HyTime usage are discussed in Chapter VIII.
3.3 HyTime Hypermedia/Time-based Structuring Language (HyTime) is formal model for defining the hypermedia structure of electronic documents81. The hypermedia concepts represented by HyTime include multi-directional and multiply anchored hyperlinking, descriptive, flexible, and powerful document object locating, and the scheduled placement of document objects along measured axes. Although there are a few other research models for hypermedia document architectures69,70, HyTime has generated attention for its perspective on fundamental hypermedia modeling concepts and its status as an international standard, particularly with respect to its relationship with SGML102. HyTime is an SGML architecture, which uses a meta-DTD to define a subset of SGML documents that is more flexible than that which would be defined by a regular DTD. A DTD defines exactly what element types can appear in a document, and exactly what attribute names can exist for each element type. An architecture’s meta-DTD, on the other hand, defines collections of attributes, called architectural forms, that can be assigned to elements in a document. An element with architectural form attributes can have other attributes as well, and can have any generic identifier. The structure of the meta-DTD is characterized by the existence of two types of constructs: element type forms (ETFs) and attribute list forms (ALFs). Both of these types of forms have SGML-defined declarations for the architecture’s attributes. There are a number of relationships that exist between forms of these types. Each ETF can contain a
43 certain pattern of elements conforming to other ETFs. The textual representation of this pattern is based on SGML content model notation. Each ETF uses the attributes of particular ALFs. Common attributes list forms are ALFs that are used by all ETFs. An SGML element is recognized as an instance of a HyTime element type form if a HyTime architectural form (HyTime) attribute is assigned to it. Usually this attribute’s name is “HyTime”. Its value is the name of the form to which the element conforms. When an element is a form instance, particular HyTime attributes can be assigned to it. These attributes provide information about the hypermedia nature of that element. Elements conforming to forms are called HyTime elements. An architectural form and the values of its attributes assigned to a particular HyTime element describe the hypermedia semantics that element has and how that element contributes to the hypermedia structuring of the document as a whole. As with SGML, HyTime does not represent the actual media content of a document. Instead, a document’s media objects are placed in HyTime containers which exist in a hierarchical document structure. HyTime, with SGML, defines this structure. These media objects can also be specified by HyTime as the endpoints of hyperlinks. Furthermore, these objects can be contained in HyTime-defined events which have a placement in relation to those of other object-containing events within a coordinate system. As an open document architecture, HyTime does not enforce a single document model. HyTime does not specify a single DTD but instead defines constructs that can exist in documents of different DTDs. While the diverse DTDs define separate models, their documents can all share the use of HyTime constructs that are uniformly recognized. HyTime defines the functionality of the tool which processes it: the HyTime engine. Applications of HyTime use this tool with an SGML parser for the initial processing of documents. A HyTime engine accepts the output of an SGML parser, identifies and
44 processes HyTime elements, and provides applications with both the results of the SGML parse and the information defined by the HyTime elements. Applications specify their interpretation of SGML and HyTime constructs into presentation constructs. While HyTime engines are developed for use by multiple applications, the applications themselves must be individually crafted by developers.
The following subsections describe the constructs of HyTime that are utilized in the work behind this thesis.
3.3.1 HyTime Hyperlinking Constructs The HyTime hyperlinks module provides architectural forms that perform hyperlink structuring for a document. Elements of these forms reference groups of document objects and describe a relation that exists between them.
3.3.1.1 Contextual Link (clink) The HyTime contextual link (clink) element type form establishes that a simple hyperlink exists between two document objects. One of these objects is the clink element itself, thus making it contextual. The other object is specified with an attribute named linkend.
3.3.1.2 Independent Link (ilink) The HyTime independent link (ilink) element type form establishes that a hyperlink exists between two or more document objects. It provides a more complex definition for hyperlinks than the clink. For example, with ilinks, one of the linked objects does not have to be the ilink element itself, enabling it to be independent instead of contextual. Also, when used in conjunction with certain location addressing constructs, an ilink can define a cascading sequence of hyperlinks that can be traversed one at time or all at once.
45 The semantics ilinks define include the link type, the anchors, the significance of each anchor in hyperlink, and the link’s traversability. The link type is a single word that describes the class of the link. In SGML/HyTime syntax this word is the generic ID of the ilink element. The anchor roles (anchrole) attribute assigns to each anchor a name describing its role in the hyperlink. This attribute is the same of all links of the same type. The anchors themselves are determined by the link ends (linkends) attribute as a list of ID references to the anchors. This number of IDREFs must either equal or be one less than the number of anchors established by the anchrole attribute. If the numbers are the same, then the link is independent and none of the anchors is the link element itself. If there is one less link end than anchor, then one of the anchors is considered the ilink itself, making the link by nature contextual although it conforms to the independent link form. The significance of each anchor can be further specified by the link end terms (endterms) attribute, which references for each anchor a document subtree describing that anchor. Finally, the allowable directions of traversal through the anchors are determined by the external access traversal rule (extra) and internal access traversal rule (intra) attributes.
3.3.1.3 Aggregate Traversal An aggregate location link (agglink) is a location element that locates multiple objects and has its aggloc attribute set to “agglink”. It is considered a HyTime link that joins the located objects. Such links are also contextual; their anchors are the located objects and the link object itself. The value of an ilink’s aggregate traversal (aggtrav) attribute assigned to an anchor that is an aggregate determines how that anchor is handled. The aggtrav attribute contains a list of tokens, with one token for each anchor or one token qualifying all anchors. The possible values for these tokens and their meanings are as follows:
• “agg” (aggregate traversal) specifies traversal to the aggregate itself
46 • “mem” (member traversal) specifies direct traversal to the aggregate members • “cor” (correspondent traversal) specifies traversal between the corresponding members of this and the other correspondent traversal anchors Aggregate and Member Traversal: The significance of aggregate and member traversal is relatively straightforward. Aggregate traversal specifies that everything the aggregate location locates is a single anchor and is traversed to as a unit. Access of any agglink anchor requires a subsequent traversal of the agglink. With member traversal, on the other hand, traversal through the ilink to the anchor results in the direct and simultaneous access of all the anchors specified by the aggregate location link, as shown in Figure 2. Correspondent Traversal: Correspondent traversal is more complicated then the other types of traversal for aggregates. When an ilink has correspondent traversal anchors, direct traversal can occur between the corresponding anchors of the different agglinks. Each correspondent traversal anchor of an ilink must specify the same number of agglink anchors. The agglink anchors are ordered so that each anchor of an agglink is considered to correspond the anchors of the same ordination in the other agglinks, as shown in Figure 2.
aggregate traversal
member traversal = hyperlink anchor = ilink
correspondent traversal = agglink affected by aggtrav = traversal path
FIGURE 2: Traversal Paths for Aggregate, Member, and Correspondent Traversal.
47
3.3.2 HyTime Locating Constructs The HyTime location address module provides architectural forms that specify the locations of document portions. Elements of these forms associate an SGML ID not with a single element but potentially with more complexly defined document objects, including portions or sequences or non-adjacent collections of elements, and including document objects that vary with time or can only be determined at run-time. When an IDREF is made to a location address element, the reference is not to the element with the ID but instead to the document object the element’s address locates. HyTime provides many different addressing techniques that can be used to form an anchor. These forms can be directly applied to any text content. An application can also define its own content specific notations and create HyTime anchors which use these notations. A set of different addresses can be aggregated (aggloc) in various ways, and a set of addresses can define the span of an anchor (spanloc).
HyTime introduces the notion of a location ladder which generalizes the concept of an anchor. Any location reference can refer to an already existing location, and an arbitrary series of location references can be chained together. This allows a newer anchor, perhaps created in a different document owned by another user, to build upon existing anchors.
Notation Location (notloc): The notation location (notloc) element type form uses a string in a specified non-HyTime notation to address a document object.
Name Space Location (nameloc) and Name List Specification (nmlist): The name space location (nameloc) and name list specification (nmlist) element type forms enable the location of objects in external SGML documents using their IDs or entity names. SGML by itself allows elements and entities in the same document to be referenced by their ID or entity name.
48 Name Query (nmquery) with the HyTime Query Notation (HyQ): The name query (nmquery) element type form and HyTime query (HyQ) notation define the locating of document objects using HyTime-defined queries on properties those objects possess. HyQ can be used to form dynamic links and to define the contents of a document by a search for document property or attribute. The HyQ query language coupled with the underlying structured document model available through SGML makes it possible for a HyTime system to immediately address several of Halasz’ issues for third-generation systems — search and query functionality, virtual structures over node collections, and computation over hypermedia networks.
Data Location Address (dataloc): The data location address (dataloc) element type form resolves to a portion of data content rather than to an SGML construct. It defines a portion of an element’s content as accessible to HyTime constructs. Without location ETFs such as dataloc, a part of a document could only be referenced if it was an element with an ID attribute assigned to it. A dataloc element is required to have an id attribute. It also must have a location source (locsrc) attribute, which specifies an element within the document to which the dataloc’s address is to be applied. This address is the dataloc’s text contents, which specify numbers indicating the tokens of a location source’s text contents are to be located. The quantum attribute specifies how these tokens are to be established and counted. A HyTime dimension list can be contained in a location element such as a dataloc to specify the measurements defining a portion of an element’s data.
3.3.3 HyTime Scheduling Constructs The measurement, scheduling, and rendition modules of HyTime define architectural forms that define measured coordinate systems and the placement of document objects within those systems. A timeline for presentation could be such a coordinate system. HyTime could then specify particular document portions as occurring at specified times. A
49 coordinate system could also map the areas of a display screen. Images could be specified as document objects that are displayed at particular locations on the screen.
Event Schedule (evsched): The event schedule (evsched) element type form specifies a collection of objects with measured placements in one instance of a coordinate system. An event schedule could represent a single screen display or one possible timeline of a multimedia presentation.
Event: The event element type form represents the placement of a single object in an event schedule. It assigns one set of coordinates to that object. The contents of the event element define the object being placed in the schedule.
Extent Specification (exspec): The extent specification (exspec) attribute list form is used by an event element to define its coordinates.
Dimension Reference (dimref): The dimension reference (dimref) element type form defines one coordinate of an event’s placement in terms of another coordinate, potentially that of another event in another schedule.
Accessed Anchor List (accanch): The accessed anchor list (accanch) element type form defines schedules in terms of the hyperlink anchors through which objects placed in the schedules were traversed during presentation.
3.3.4 Other HyTime Constructs HyTime has facilities that do not fit in the three categories described above. Most of these are defined in the HyTime base module. Many provide foundations for building constructs in the other modules. Others perform unique tasks that simply do not fit in the other categories.
50 HyTime Document (HyDoc): The root element of any HyTime document must conform to the HyTime document (HyDoc) element type form. It establishes the document as using HyTime constructs and makes it susceptible to HyTime processing. Suppress HyTime (sHyTime): The suppress HyTime (sHyTime) element type form establishes an element as having no HyTime semantics and not requiring HyTime processing. There are no restrictions on what sHyTime elements contain. Such elements do, however, still use the attributes of certain attribute list forms. Identification Attributes (all-id): The identification attributes (all-id) common attribute list form defines the HyTime id attribute, which functions the same as the SGML ID attribute. The HyTime attribute names (HyNames) attribute of the all-id ALF can reassign for its element an SGML attribute with a particular name as being a HyTime attribute of a different name. ID Reference Resolution Control (all-ref): The reference type (reftype) attribute of the all-ref common attribute list form places restrictions on the types of elements that an IDREF attribute can reference. If this attribute is declared in the DTD as fixed, then the same restriction applies to all instances of that element type in the document. Activity Tracking: The activity tracking facility and its constructs allow the author to specify who is to be notified when particular activities are performed on parts of a document. This provides the basic structures needed for specifying the collaborative organization around the collective authoring of a document. It also provides some, but not all, of the information in a document needed for a secure environment around it. Activity tracking only specifies the monitoring of a document’s use, not who can use it and when.
51
3.3.5 Halasz Classification of HyTime HyTime satisfies all second, and some third, generation hypermedia generic document structuring requirements. Table 2 shows which Halasz generational hypermedia issues are addressed by SGML. Table 5 shows what HyTime facilities address these issues.
3.3.5.1 HyTime as Second Generation Hypermedia HyTime satisfies the requirements of providing a generic and presentation-independent document model for second generation hypermedia. HyTime inherits node typing from SGML. Unlike HTML, HyTime is defined as an SGML architecture, not a DTD, so it maintains the ability to define new node types for its applications. As an architecture, HyTime also inherits SGML’s facilitation of extensions. SGML’s external entities and TABLE 5: How HyTime Addresses Hypermedia Issues Issue
HyTime
Meta-linear Structure Over Text
Hyperlinks module
Node Typing
Inherits from SGML
Simple Link Structures
Hyperlinking
Hierarchical Support
Name space locations
Extensibility
Inherits from SGML
Simple Text Query
Query location, HyQ
Integration of Search and Query Functionality
Query location, HyQ
Composite Node Types
Inherits from SGML
Virtual Structures over Node Collections
Hyperlinking, Location addressing
Support for Collaborative Work
Activity tracking
Extensibility and Tailorability
Inherits from SGML
Open Hyperdocument and Open Hyperlinking Architecture
Inherits from SGML, hyperlinking
Mobile and Distributed Objects
Location addressing
Interactive Time-based Hypermedia Document Models
Hyperlinking, scheduling module
Security
Activity tracking
22
22 22
22
52 HyTime’s name space locations enable documents to reference external data, providing support for document hierarchies. HyTime hyperlink constructs meet and exceed second generation link requirements. HyTime’s name query form and HyQ notation meet and exceed second generation simple text query requirements.
This use of the Halasz classification applies only to HyTime as a document format, which is only one aspect of a hypermedia system. HyTime’s domain as a hypermedia solution is also narrowed by its presentation-independence. The second generation insufficiencies of the standard listed in Table 2 exist primarily because of this intended scope. HyTime can not be said to provide a GUI because it represents nothing about the documents appearance when presented. Similarly, HyTime can not be said to provide simple distribution. These aspects of hypermedia must be provided by components of the system outside of HyTime processing.
3.3.5.2 HyTime as Third Generation Hypermedia HyTime addresses some of the requirements of third generation hypermedia. The third generation evaluation provided here is based on previous work 22. SGML provides HyTime with the ability to create composite node types. It also provides some extensibility and tailorability through its open document philosophy and, through HyTime’s architectural definitions, the ability to create customized document sets that can be integrated with each other. HyTime querying enables integration of search and query functionality. HyTime querying, with hyperlinking and location addressing, can define complex document structures that are computed at runtime, providing some virtual structuring over node collections. Activity tracking can be used as part of collaborative work support, but HyTime does not satisfy most needs of this third generation characteristic. HyTime provides no constructs for defining versions of documents.
53 HyTime’s failure to provide computation over hypermedia networks is, as with some second generation characteristics, a result of its intentionally focussed domain of presentation and processing time independence. This unprovided facility is a runtime activity performed by system components that fall outside of HyTime processing. HyTime provides no constructs that integrate external processing in particular with document structure. However, the notation location form could be used, in conjunction with an extraHyTime system component, to enable external processes to be referenced as part of document structure.
3.3.5.3 Conclusion: The Halasz Classification of HyTime HyTime provides most second generation and some third generation facilities as specified by the Halasz classification of hypermedia. HyTime’s neglect of some of these facilities is intentional. HyTime is a format language for structuring documents in a manner that is independent of their runtime processing and presentation. Because of this, it would be inappropriate for HyTime to address such areas as defining graphical interfaces, providing file distribution, and integrating runtime processes. All second generation requirements are either satisfied by HyTime or fall outside its domain. Thus, HyTime could be considered a complete solution for the structuring of documents in second generation hypersystems. Some third generation hypermedia facilities not provided by HyTime could be considered part of its domain. This suggests HyTime does not provide a complete solution for structuring third generation hyperdocuments. These neglected facilities are versioning of documents and providing support for collaborative work.
3.3.6 Dexter-based Analysis of HyTime The Dexter model was created to describe hypertext systems so that they have a common basis for evaluation and comparison. It is designed to represent those aspects of
54 hypertext that are common to all implementations. Dexter provides a means of analyzing HyTime in comparison with other hypertext solutions. It also enables comparison of HyTime with the established needs of generalized hypertext modeling because Dexter defines these needs. If HyTime fulfills these needs, it can be said to be complete. If there are aspects of the Dexter model HyTime fails to address, then shortcomings of HyTime are revealed. Aspects of hypertext that HyTime defines that Dexter does not suggest areas where HyTime may be too rigid and specific for generalized hypertext structuring. An analysis of HyTime in terms of the Dexter model is given here, and is based on previous work22. 3.3.6.1 Correlation Between HyTime and Dexter HyTime is represented well by the Dexter model. Its division of modules is similar to Dexter’s division of layers, as shown in Figure 3. This similarity includes a focus on the facilities of the storage layer. The Dexter module provides its most precise definitions for the storage layer and its translation to the other two layers. The run-time and withincomponent layer have very little specification in the Dexter model. HyTime also provides little specification of either document appearance, which is represented by the run-time
Dexter Layers
Corresponding HyTime Modules
Run-time
Presentation Specifications
Storage
Anchoring
Measurement, Scheduling, and Rendition
Hyperlinks and Base
Location Address
Within-component
FIGURE 3: Comparison of Dexter Layers and HyTime Modules
55 layer, or exactly how the document content is stored, which is represented by the withincomponent layer.
The storage layer of the Dexter Model defines the overall structure of the document itself, independent of the document’s presentation or the storage of its contents. The storage layer corresponds with the hyperlinks and base modules of HyTime. The HyTime base module provides the building blocks for general document structure that are typically components of the more semantically specific constructs from the other modules. These building blocks by themselves can define some linear and hierarchical structuring beyond that provided by SGML. The hyperlinks module establishes non-hierarchical and nonlinear relationships between document components. The structures defined by this module correspond very closely with links in the Dexter model. The ability of these two modules to link document components with each other in a web of relationships closely mirrors that of the storage layer.
Neither Dexter nor HyTime define much about how a document’s contents are stored in local file systems because such information is platform-specific. However, the Dexter anchoring layer provides an interface between the storage and within-component layers by providing storage with referenced anchors that locate document component items and portions of data within those items. Anchors can also refer to collections or contiguous spans of data in the within-component layer. These facilities mirror that of the HyTime location address module. Location addressing makes document objects, including media objects, referenced with SGML ID attributes. Further, the module specifically defines collections of objects and spans of data as referenced document constructs. Quite often, HyTime location addresses are used to define the endpoints of HyTime hyperlinks. The relation between location addresses and hyperlinks in HyTime is similar to that between anchors and links in Dexter.
56 Neither Dexter nor HyTime define much about how a document appears upon presentation. Dexter places this aspect of document processing in the runtime layer, which is little defined because the possibilities of document appearance and interaction are considered too broad to represent with one general model. HyTime declines to specify document appearance and interaction for similar reasons. These reasons include the desire to keep the aspects of structuring defined by HyTime equally applicable to all different current and future means of presentation. Dexter’s presentation specifications provide communication from the storage layer to the run-time layer. This allows the storage layer to define some general aspects of the document’s presentation when they depend on other constructs defined on the storage layer. The functionalities of HyTime’s measurement, scheduling, and rendition modules fit within the presentation specifications layer. They define the numeric aspects of document structuring, specifying the placement of document objects within measured coordinate systems such as mapped images or timelines. While HyTime does not defined the specifics of presentation, these numeric constructs do provide much information needed for processing a document into its presentation. These modules provide transitional services that fit within presentation specification.
3.3.6.2 Where HyTime Falls Short of Dexter HyTime provides less of the representation functionality of the run-time layer than Dexter does. Both models keep their focus away from document presentation to enable presentation-independence, but HyTime’s lack of specification for this area is more pronounced. HyTime’s main shortcoming for supporting run-time processing is its lack of constructs that enable the definition of document structure in terms of variables that are only determinable at run-time. For example, while HyTime’s dimension referencing allows defining the indirect definition of numbers in the measured placement of objects to be in terms of the placement of other objects or in terms of mathematical formulas, it does
57 not provide for such referencing to be done in terms of object dimensions that are only determinable at run-time. An example of such run-time referencing is having one MHEG video clip start when another ends when the length of the clips are not known during authoring. One exception to this lack of runtime support is that certain structures are defined in terms of what anchors of a hyperlink have been accessed during presentation in traversing through the hyperlink web to a particular place in the document. The concept of access anchors is used by the internal and external traversal attributes of the ilink form to restrict the possible traversals through a link. This concept is also used by accessed anchor lists in the scheduling module, which define the runtime rendering of schedules in terms of the hyperlink anchors through which objects placed in the schedules were traversed to.
3.3.6.3 Where HyTime Exceeds Dexter HyTime addresses hypermedia in general whereas Dexter addresses hypertext in general. Since hypermedia is a superset of hypertext, HyTime provides constructs for representational semantics that Dexter does not. The primary structural requirement that hypermedia has over hypertext is the use of numbers and coordinate systems. This is because non-text media are frequently defined in terms of numeric measurements, such as image dimensions and sound sample duration. Furthermore, the presentation of such media objects are often coordinated relative to each other using numbers. Examples of this are the placement of media displays on a timeline and the showing of multiple images at different locations on a screen at once.
3.3.6.4 Conclusion: The Dexter-based Analysis of HyTime As a model enabling the description and comparison of the general class of hypertext systems, Dexter serves as a good base for evaluation of other models that seek to represent hypertext usage in general such as HyTime. For the most part, the Dexter model and the
58 model implied by HyTime constructs are symmetric and perform the same representational tasks. That indicates that HyTime succeeds in addressing the needs of general hypertext structuring. HyTime exceeds the representation power of Dexter by defining the use of numeric coordinate systems in document structuring. However, this overlap is not due to overspecifying hypertext by instead to the fact that HyTime seeks to represent general hypermedia, not just hypertext as Dexter does. These extra semantic constructs are due to the large domain HyTime addresses. The comparison of HyTime and Dexter does reveal the shortcoming that HyTime does not provide many facilities that Dexter does for guiding a document’s rendering to a presentation based on attributes that are evaluated only at the time of presentation.
3.4 Java and HyTime Java is not a hypermedia document format language but a programming language for creating portable presentations using a particular GUI. However, the functionalities and presentation interface provided by Java have aspects that are inherent to hypermedia processing. In this section, an overview of Java is given and an exploration of these hypermedia characteristics is made. Further, the semantic overlaps between these Java constructs and corresponding HyTime constructs are described. These semantics overlaps form the basis for discussion in Section 6.4.2 and Section 7.5, which discuss, respectively, the use of Java to present HyTime documents and the inclusion of Java applets within HyTime documents. These HyTime-encompassing Java constructs have been divided into the areas of hyperlinking, locating, and scheduling in the subsections below.
3.4.1 Hyperlinking Java’s Abstract Window Toolkit (AWT) provides the following components for defining a document’s presentation and interface38. These interface components provide the user with navigational capabilities. Defining the navigational structure of documents is the
59 primary potential application of HyTime hyperlinking. This is the premise for contrasting Java navigational constructs with HyTime hyperlinking constructs in the subsections below. 3.4.1.1 Java Button and HyTime clink The AWT button provides the same interface functionality as buttons on most windowed system: the user clicks on it with the mouse and an action occurs. This thesis associates the HyTime clink form with the AWT button. The clink’s contents are associated with the display of a Java button. The clink’s endpoint is associated with what is displayed when the button is pressed. An alternate HyTime equivalent could be an ilink defined as contextual, with only one link end and two anchor roles. 3.4.1.2 Java Choice and HyTime ilink AWT’s choice appears as a line of text in a rectangle, much like a button does. Clicking on the choice rectangle, however, enables the user to select one of a list of text strings, and the selection made appears in the rectangle afterwards. Often in Java applets, a choice box is combined with a button so that pressing the button activates the choice made. This allows the user to select one of multiple possible destinations for the button. This thesis associates this choice/button combination with ilink forms having more than two anchors. The potential for more than one endpoint for an anchor in an ilink corresponds with the multiple possibilities given in the choice box. The strings given as choices could correspond with the hyperlink’s anchors to which traversal can occur from the current location. Each text string would be the value of the corresponding anchor’s role, or perhaps be specified by the link end term for each anchor. Pressing the button leads to the presentation of the anchor whose role was selected by the user through the choice component. This combination is appropriate for member traversal but not for aggregate traversal because the choice component only allows one level of selection, which corresponds here with a hyperlink for which all anchors are accessible in a single step of
60 traversal. If an anchor of an ilink is an aggregate link, then traversal to that anchor must be specified as member traversal, not aggregate traversal. As member traversal, all objects located by that agglink would be considered as anchors immediately accessible through the ilink. Each member of the agglink would become an immediate option in the corresponding choice box. 3.4.1.3 Java Menu and HyTime agglink While the button enables only one action, and the choice shows multiple possible actions in one level of indirection, the menu provides for the cascading selection of choices. The selection of one menu option could provide another menu of new options. This enables a large number of options to be organized in a tree-shaped hierarchy for easy traversal and access, providing more selective power than buttons and choices. This cascading effect is not possible with choice boxes, which offer only one level of options. One limitation of menus is they can only appear on the menu bar of the window displaying the Java applet. Menus cannot be integrated with the main GUI display like buttons and choices can. The cascading selection of actions corresponds with the HyTime aggregate link (agglink) form. An agglink is an element that locates a group of objects and specifies them as anchors that can each be traversed to. When an agglink element is specified as the anchor of another hyperlink, and aggregate traversal is specified for that anchor in the link, then traversal from the other link to any anchor in the agglink is considered to happen in two steps instead of directly. In a Java menu, an option that displays a list of more options is the equivalent of an anchor that is an agglink for which aggregate traversal is specified. The extra step of traversal is represented by the fact that selection the first option results in the display of the list, and a subsequent selection must be made for an option in that list.
61
3.4.2 Locating: Java URL and HyTime notloc URLs can be used in Java applets to specify files that are accessible through the web. The openStream() method can take a URL and make the contents of its file available to the applet for, as an example, presentation of the file to the user. An SGML public identifier for using URL notation to reference web-accessible files has been proposed, as has its use with HyTime notation location (notloc) elements to make web files addressable with IDREF attributes87. The use of the public identifier and notloc is discussed further in Section 8.3.1, in the context of converting HTML’s use of URLs to HyTime. The presentation of a file addressed with a URL is often the action resulting from activating a Java applet’s link, specified with either a button, choice, or menu as described above. This interaction has a corresponding HyTime representation as a link having a starting anchor be the button, choice option, or menu option, and the ending anchor specified as the ID of the notloc addressing the file with the URL.
3.4.3 Scheduling: Java Image and HyTime event One Java construct that suggests scheduling semantics is the image. The Graphics class takes as parameters for its drawImage() method the coordinates for the image’s upper left corner and the image’s width and height for its placement in the Component area. An equivalent HyTime encoding would be to contain the image in an event, provide the event with the two-dimensional placement coordinates of the corner placement and image size, and put the event in an event schedule representing the Component area. The HMP HyTime application developed in conjunction with this thesis defines its display of images using two-dimensional event schedules with events containing the image media object, as described in Section 4.1.
62
3.5 Summary This chapter reviewed the primary document format languages utilized in this thesis in terms of the Halasz classification and the Dexter model. It also reviewed the Java language and its potential relationship to HyTime. SGML was shown to address some of the issues of the first and second generations of hypermedia but primarily provides a foundation for open document structuring. HTML, as one application of SGML designed specifically for hypertext presentation, addresses more of these hypermedia issues but still falls short of second generation requirements. HyTime, an architecture of SGML, addresses most of the second generation issues, some of the third, and maintains the open document modelling capabilities of SGML. An analysis and evaluation of HyTime using the Dexter model was provided. HyTime was shown to closely mirror Dexter in general, supercede it by specifying the numeric aspects of multimedia structuring, and fail to provide the equivalent of Dexter’s ability to influence document presentation.
63
IV Sample HyTime Application 4.1 Description Much of this thesis research involves the HyTime environment being developed at the Distributed Multimedia Systems Lab (DMSL)33,113,32,31,30. This environment centers around a HyTime engine named HyOctane and a HyTime application called Hypermedia Presentation (HMP) (formerly called SlideShow). The use of HMP demonstrates HyTime’s applicability and provided empirical study of its use. The initial design of this environment was the focus of an earlier thesis113. Part of this thesis’ work has been the design of extensions to HyOctane and HMP. Unless noted otherwise, this chapter describes HMP before these extensions. HMP is an interactive presentation program. It presents slides where each slide is a backdrop image and a series of media object presentations. These media objects can be images, animations, audio playbacks, and script playbacks. They can be positioned on the screen and presented at particular times. Also displayed with a slide are buttons, each of which when pressed by the user’s mouse triggers the presentation of another slide or terminates the program. HMP processes a particular set of documents, defined by an SGML DTD. Section 4.2 contains the DTD used by HMP. Figure 4 diagrams this DTD. Descriptions of the
64 document objects used in the DTD are given in Table 6. This DTD and application demonstrate HyTime’s ability to establish hyperlinks and to place document objects in relation to one another in a shared space. Elements of type slidesho act as root objects for their documents. Elements of type slides contain multiple elements of type slide and define a presentation space for them. This presentation space consists of a workstation screen, with an x and y axis, and a timeline. Elements of type slide represent the presentation of a slide, including its background, its component presentation units, and its buttons. An element of type button associates the display of button pressing feedback with
in DTD only
HyDoc
axis axismeas = “virspace” axisdim = “1024”
HyTime architectural name archform form (if an instance of a form) attribute HyTime attribute attribute non-HyTime attribute attribute = fixed value (if fixed in DTD) attribute element reference contents of contents of element (or) element ?+* (sequence) ? = 0 or 1 + = 1 or more * = 0 or more
fcs axisdefs = “x-axis y-axis timeaxis”
axis axismeas = “virspace” axisdim = “1024” axis axismeas = “SIsecond” axisdim = “1024”
file accessed as SGML external entity in SGML declared notation element name notation
evsched
evsched
extlist
clink
clink
linkend
linkend transitn
dimref flip selcomp axisref extref elemref
event
event
event
event
exspec
exspec
exspec
exspec
a signed non-zero integer
HyOctane = “script”
gif
fli
ulaw
HyScript script text
FIGURE 4: Multimedia Document Structure Corresponding to the HMP DTD33,34,32,31
65 the selection of a particular screen area. These button screen areas are specified by button event (butnevnt) elements. An element of type button press (butnpres) schedules the feedback of a button selection and schedules the occurrence of a button done (butndone) event which triggers the linking to a new slide. A background (backgrnd) element is a screen-sized background image display. A slidevnt element can contain one of four media object element types, and it assigns a time and place for that object’s display. A media object element is an SGML external entity, thus specifying an external file and its format. The timing and positioning of background, slide event, and button event elements are specified by placment elements which are reference by the exspec attribute. Each coordinate in the three-dimensional space is specified either directly by a number or indirectly as a dimension reference (dimref) element. Dimension reference elements
TABLE 6: Document Objects Used in HMP DTD33,34,31
SGML Element Type
HyTime Architectura l Form
HyTime Semantics
HMP Semantics
slidesho
HyDoc
Contents are whole hyperdocument
Root object of document
slides
fcs
Definition of a finite coordinate space
Defines context of slide presentations
slide
evsched
Group of events in same fcs instance
One slide
backgnrd, slidevnt, butnevnt, butndone
event
Existence of an object in an fcs
Existence of a media object presentation in a slide
button, nextslid
clink
Link from contained object to another object
When contents are selected present slide linked to
xaxis, yaxis, timeaxis
axis
Reference for object position and dimension
The workstation screen dimensions and seconds after start of slide
placment
extlist
Position of an object along several axes
Where and when an object is presented in a slide
dimref
dimref
Reference to another object’s position for relational placement
Reference to another object’s position for relational placement
66 specify a coordinate number as some function of a particular coordinate of use of scripts as media types in HyTime documents is discussed in Chapter VII.
4.2 HMP DTD This section contains the Document Type Definition specifying the model for the document set used by the Hypermedia Presentation application of HyTime.
slides - O (slide+,butnpres*)> slides NAME #FIXED fcs and time axes (defined below) are used -CDATA #FIXED "xaxis yaxis timeaxis">
67
68 id
ID
#REQUIRED>
69 