An Overview of the REX Software Architecture Jeff Magee, Jeff Kramer, Morris Sloman and Naranker Dulay
Department of Computing, Imperial College of Science, Technology and Medicine, 180 Queen's Gate, London SW7 2BZ, UK. Email:
[email protected]
Abstract This paper describes the software architecture currently under development for REX, a project supported by the European Economic Community under the ESPRIT II initiative. The architecture is aimed at supporting the construction of reconfigurable and extensible parallel and distributed systems. The main principle underlying this architecture is that systems should be described, constructed and modified as a structural configuration of interconnected component instances. The structure should be described by a separate explicit configuration language allowing components to be programmed in a range of heterogeneous programming languages. The paper gives an informal description of the three elements of the architecture namely, an Interface Specification language, a set of communication primitives and a language for describing overall system structure called Darwin. Examples of the use of these are given together with an overview of how they integrate to support construction and reconfiguration of distributed systems.
1
1.Introduction The REX project, Reconfigurable and Extensible Parallel and Distributed Systems, aims to develop an integrated methodology and associated support tools for the development and management of parallel and distributed systems. Ten European partners from industry and university research groups are involved. The project started in May 1989 and is to run for 5 years. Two large demonstrators in the telecommunications and Computer Integrated Manufacturing (CIM) areas act as a focus for the work and as a means for demonstrating the techniques and tools developed. Interface described by REX ISL
Rex Component
Implemented in range of Programming Languages ( C, Ada, Pascal etc) + REX Communication primitives
Figure 1 - Program Components Central to the REX project is the view of distributed and parallel systems as interconnected sets of component instances. Components are objects which encapsulate state and have well defined interfaces specified by an Interface Specification Language (ISL). The functionality of a component may be implemented in a range of programming languages, however, interaction with other components utilises communication primitives supplied by REX (Figure 1). Components in REX are types (c.f. classes) from which more than one component instance can be created. Component instances execute in parallel. The sets of instances which exist in a system together with their interconnections are specified in a separate configuration language - Darwin (Figure 2).
Figure 2 - Systems In addition to describing overall system structure, Darwin permits systems to be structured hierarchically by allowing the specification of composite components which are configurations of either program or other composite components. The interface to a composite component is specified by the ISL in an identical way to program components (Figure 3). In this way, program components may be replaced by composite components and vice-versa without affecting the rest of the system structure. Finally, Darwin is also used as the language in which to express system 2
extension and modification by structural changes.
Figure 3 - Composite Component In the following, the Interface Specification Language is described in Section 2, the communication primitives in Section 3, and the configuration language Darwin in section 4. The features of each of these is demonstrated by example together with a design rationale. The final section of the paper discusses some of the outstanding issues in the design of the REX software architecture together with its relationship to the remainder of the project.
2. Interface Specification Language (ISL) The ISL is used to clearly define the datatypes which can be transferred across a component interface. Since components can be implemented in different programming languages, the ISL provides a common specification for datatypes to enable type checking when binding together the interfaces of heterogeneous components. In addition, the ISL must define the points at which interactions with other components occur. The requirement is for context independent components which can be reused both in the same system and in other systems[6]. This requirement means the interface must specify both the services that a component uses as well as those that it provides. In the following these interaction or service points are termed ports. The ISL can be thus be considered to have two elements: datatype definition and port definition. 2.1 Datatype Definition The datatype definition part of the REX ISL is loosely based on the ANSA[1] and XEROX[2] ISLs. The usual scalar types and constructors for arrays, records and variants are provided (figure 4).
3
Scalar Types signal = a zero information message type - used for synchronisation messages bool = FALSE | TRUE card8 = 8 bit unsigned integer card16 = 16 bit unsigned integer card32 = 32 bit unsigned integer card64 = 64 bit unsigned integer int8 = 8 bit integer int16 = 16 bit integer int32 = 32 bit integer int64 = 64 bit integer real32 = 32 bit floating point number (IEEE single precision Format) real64 = 64 bit floating point number (IEEE double precision Format) Standard char int real
Aliases = card8 = int32 = real64
/* ASCII characters */
Structured Types Arrays vector = [80] real32 screen = [24, 80] char Records student = ( refno: int32; age:int16; male:bool) mess = ( i:int16; v: vector) Variants ordinal = < i: int16 | c:char | b:bool | s:signal>
Figure 4 - Scalar and structured types. Note that variant records have an implicit tag which can be queried by a tag function which returns the tag value current for a variant variable. This is safer than explicit tags which can be incorrectly set by a user. The REX ISL also permits type definitions to be parameterised so that the specification of the size of arrays can be delayed until variables or ports of these types are declared e.g. string (N) = [ N ] char. A novel feature of the REX ISL, not found in [1,2], is the inclusion of type extensions. Record and variant types can be extended to form new record or variant types respectively (figure 5). This permits the implementation of general components which can be applied to any type which has been defined to be an extension of the base type which that component handles. Extendible types are crucial in allowing evolution and reuse. Wider extended type values can always be passed to narrower variables of a parent type. Record extensions were first introduced by the language Oberon[3] and have also been implemented in a version of Conic[4]. Record extension /* example */ studentmarks = student + ( grades : gradeT; degree : degreeT ) Variant extension /* example */ scalars = ordinal + < r : real >
4
Figure 5 - Type Extensions. To permit complex pointer data structures to be transferred between components, the ISL provides a linked datatype as shown below. list = item = stack =
^ item ( data : datatype; next : list ) linked list
When a pointer is passed to a variable or port of linked type all data reachable from that pointer is copied. Effectively, the data structure is “flattened” for transmission. 2.2 Port Type Definition Ports represent at an interface both the services used by a component and the services provided by a component. Port types are the signatures of these services. For example, the declaration: get = port signal return char
declares a port type get which defines a service taking a signal input and returning a character value as its output. Ports may specify multiple input and return types as in the following example: push = port stack, item return stack, length
The above examples define interfaces to services which are invoked synchronously i.e. they have a return part. The ISL also permits the definition of services which are invoked asynchronously as in the following example: put = port char
Interfaces To describe complex interfaces consisting of many ports, the ability to construct complex port types is required in a way analogous to datatypes. The REX ISL provides two port type constructors, arrays and port sets. Port arrays are declared in an identical way to ordinary arrays e.g: pushes = [10] push
Port sets are analogous to the record constructor for ordinary datatypes, however, the type matching rule for port sets is different from that of records. The type matching rules for interfaces will be discussed in section 4. An example of a port set is given in Figure 6.
5
define filedefs = { fileid = int data = [512] char length = int errcode = int16 fileio = { read: port fileid return length , data write: port fileid, length , data return length open: port string (64) return fileid close: port fileid return errcode invert error: port errcode }
} fileio read write open close error
Figure 6 - Port Sets. Port sets are the way the ISL describes the interface to a component. Since we wish to use the same definition for both the client which uses an interface and the server which provides it, port sets do not specify invocation direction. However, we may wish to specify that the invocation for some ports is in the opposite direction. The keyword invert in the above example specifies that the port error has the opposite direction to the rest of the ports. A client component which used this interface would thus make calls on read, write, open and close and accept calls on error. A server component which provided the service would accept calls on read, write, open and close and make calls on error. Port sets can be extended in the same way as record types (Figure 7). newfileio = fileio + {seek:port int return errcode} newfileio read write open close seek error
Figure 7 - Port Set Extension. Program Components Program components provide the functionality of REX systems. As discussed before, port sets describe the interface to these components. The syntax to associate a port set with an implementation is given in Figure 8.
6
component fileserver (id : int) = entry fileio + { use filedefs: fileio /* can also declare additional ports which extend the fileio set. Component is implemented in a programming language or configuration language */ }
fileserver
read write open close error
Figure 8 - Program Component fileserver . The keyword entry indicates that the component fileserver will accept calls from the ports included in the fileio port set and make calls on the “inverted” ports. The component may declare additional ports which are regarded as extending the fileio set. Components are types which may be multiply instantiated. In the above example the formal parameter id will be resolved when an instance of fileserver is created. The example shown in Figure 9 shows the client use of the fileio interface. component client = exit fileio + { use filedefs: fileio /* Implemented in some programming language */ } read write
client
open close error
Figure 9 - Program Component client . The keyword exit indicates that the component client will make calls on the ports included in the fileio port set and accept calls from the “inverted” ports. Again, the component may declare additional ports which are regarded as extending the fileio set. A component need not necessarily extend an existing port set definition but may form a new port set as in the example of figure 10 which declares two ports of type fileio to enable the component to access two file system servers. In this case, the ports would be referenced by the implementation part of the component as filesys1.read, filesys1.write etc.
7
component client2 = { use filedefs: fileio port exit filesys1:fileio exit filesys2:fileio /* Implemented in some programming language */ }
read write open close error
client2
read write open close error
filesys1
filesys2
Figure 10 - Program Component client2 .
3. Communication Primitives The previous section has outlined how the REX ISL is used to specify the interface to components in terms of the datatypes which can be transferred and the ports through which these transfers occur. To access this interface the component implementation uses the REX communication primitives. In the examples given in the following, the implementation language is assumed to be C although it is intended that the communication primitives can be embedded in a wide range of programming languages. The communication primitives fall into two classes: call primitives used to invoke services via ports and accept primitives used to service invocations on ports. 3.1 Call Primitives The simplest form of call is the asynchronous invocation shown in figure 11. This call does not block the enclosing components execution. Note that in the example, the port type has not been imported from a file of ISL definitions but has been directly declared. The value passed by the call must be of the same datatype as that declared with the port.
8
component foobar = { port exit outch : port char -----
outch
call outch('a'); ------}
Figure 11 - Asynchronous Call primitive. The synchronous call shown in figure 12 blocks the enclosing component until a result is returned. In the example, the values 12 and 3 are transmitted and the component blocks until the result (hopefully 4.0) is returned. The synchronous call is essentially a remote procedure call (RPC). component foobar = { port exit divide : port int,int return real ----double a; -----
divide
call divide(12,3) wait (a) ------}
Figure 12 - Synchronous Call primitive or RPC. Component instances in REX may be either colocated on the same processor or remotely located on processors connected by a communication system. There is thus the possibility that a call may fail due to communication failure, remote processor failure or called component failure. In the above example, if the call fails then the enclosing component will also fail (halt its execution). In the circumstances where a programmer wishes to handle the failure the form of the synchronous call shown below can be used: double a; int r; -----
call divide(12,0) wait (a) fail r; if (r!=0) error("Divide failed for reason %d",r);
The fail clause indicates that if the call fails it returns a non-zero result code r indicating the reason for the failure, rather than halting the enclosing component. A successful call is indicated by a zero return code. Result codes indicating communication failure etc., will be system defined and as discussed in the following the callee may also return user defined fail codes. This simple failure handling approach has been adopted to facilitate the integration of the REX communication primitives into a range of programming languages. In addition, a timeout may be associated with a call by means of the primitive: valid(divide,100)
9
which sets a timeout value of, in this case, 100ms on the divide port such that a call on divide will fail if it does not complete within 100ms. It seems reasonable to associate the timeout with the port rather than the call as in Ada[5] or Conic[6] since it is indicating a required property of the service rather than a particular invocation. Note that the timeout is on completion of the call at the caller as in Conic rather than acceptance of the call by the callee as in Ada. To allow parallelism between caller and callee during a synchronous call it is possible to separate the call from the wait as shown below: call divide(12,3); /* statements */ wait divide(a);
This form allows a number of services to be invoked in parallel i.e call port1; call port2; ..; wait port; wait port2, to increase the parallelism and thus performance of a system. It avoids the proliferation of “messenger” tasks found in Ada programs. This ability to defer waiting for the results of a call is further enhanced by the handle mechanism which allows a stream of calls to be made to single ports before waiting for results. This facility is useful in increasing performance where communication latency is high between caller and callee. The following example illustrates the mechanism: handle H1,H2; /* declares handles H1 & H2 */ -----call divide:H1(12,3); call divide:H2(12,2); ------wait divide:H1(a); wait divide:H2(b);
The handle is set by the call and is the capability to wait for the return value associated with that call. In the example a will be the result of 12/3 and b the result of 12/2. The handle mechanism is similar in intent to the promise mechanism outlined in [8]. 3.2 Accept Primitives In the RPC model [7] each call creates a new thread at the callee. Inter-thread synchronisation is then supported by semaphores, locks or monitors. In the REX software architecture, the service of a call is explicitly associated with an execution thread as in [5,6]. The explicit approach is preferred since it avoids the need for additional synchronisation primitives. Consequently, the accept primitive blocks the thread of execution of its enclosing component until a call arrives. When an asynchronous call is accepted, the communication transaction is complete. The three ways of completing a synchronous call at a component are outlined in figure 13. The normal return passes the result of the call back to the caller. The abort passes a fail code back to the caller which is either caught by the fail clause or causes the caller to halt. Lastly, the call can be passed on to a third party by the forward primitive. The component to which the call is forwarded returns the result directly to the original caller.
10
divide
component arith = { port entry divide : port int,int return real exit bigdivide : port int,int return real ----#define DIVZERO 23 int a,b; -----
bigdivide
accept divide(a,b); if (b==0)
abort divide(DIVZERO); else if (a
