Cover Page - CiteSeerX

10 downloads 44039 Views 26KB Size Report
and server components; they monitor interactions to enforce client and server ... client or the state of the server are best implemented by client-side and ...
Cover Page

Mediators and Obligations: An Architecture for Building Dependable Systems Containing COTS Software Components Vicraj Thomas, SriKumar Kareti, Walter Heimerdinger, Sunondo Ghosh Honeywell Technology Center, Minneapolis, MN 55418 thomas,skareti,walt @htc.honeywell.com 

Keywords Dependable systems, middleware, software architecture, soft real-time, CORBA.

Contact Information Dr. Vicraj Thomas MN65-2600 Honeywell Technology Center 3660 Technology Drive Minneapolis, MN 55518 Email: [email protected] Phone: +1 612 951 7470 Fax: +1 612 951 7438

1

Mediators and Obligations: An Architecture for Building Dependable Systems Containing COTS Software Components Vicraj Thomas, SriKumar Kareti, Walter Heimerdinger, Sunondo Ghosh 

Honeywell Technology Center, Minneapolis, MN 55418 thomas,skareti,walt @htc.honeywell.com 

1 Introduction



























































Client



 

 

 

 

 

 







































































Mediator



Mediator













































Client-side mediator





























































Server

This paper describes an architecture based on mediators and obligations for building dependable distributed systems that include some Commercial Off-The-Shelf (COTS) components. This architecture enables the construction of reliable and trustworthy systems even if some components of the system are COTS components with widely varying robustness and trustworthiness. A noteworthy feature of this architecture is that it does not depend on the availability of source code for the COTS components. It is assumed that the systems of interest are soft real-time systems. Faults occur in the system when one or more components fail to meet their obligations. These obligations include timely response to service requests, bounds on demands for service from other system resources, correctness of results, and legitimacy of interactions. Faults may also occur due to unauthorized or illegal interactions between components. Mediators are software entities that detect and contain faults by monitoring interactions between client and server components; they monitor interactions to enforce client and server obligations. For example, clients are obliged to make a bounded number of well-formed requests to authorized servers. Servers are obliged to return correct, well-formed results to an authorized client within a predetermined time frame. Client and server obligations are enforced by client-side and server-side mediators respectively. Clientside mediators may monitor requests from clients and the responses to these requests. Server-side mediators may monitor incoming service requests and outgoing responses. This concept is illustrated in Figure 1. Mediators provide the architectural benefit of separating concerns about component functionality from concerns about dependablity—system functionality concerns are addressed by system software components while dependablity concerns are addressed by the mediators. Different mediators may be applied to different components in the system depending on the trustworthiness of the component and the criticality of the component’s interactions with other components. Moreover, some mediators can be composed by chaining them together.

Server-side mediator

Figure 1: Mediators that enforce client and server side obligations 1

2 Mediator Functions Possible uses of mediators are described in this section. Enforcing timing obligations. Timing obligations of servers ensure that clients do not block for unpredictable amounts of time waiting for a request to be serviced. These obligations may be enforced by clientside or server-side mediators; if a service request does not complete within a specified time the mediator can generate and return a failure notification to the client. In addition, a server-side mediator may be able to respond with recently computed values if new values cannot be computed on time. A mediator that enforces timing obligations is described in Section 4. Fault-management. Fault-management techniques such as server replication and automatic restart of failed services are used to increase system dependability. These techniques may be implemented by mediators in a manner that is transparent to other components of the system. A mediator may replicate service requests and forward it to multiple servers or it may monitor a service for failures and restart it on failure. Security. Security mechanisms such as access control lists (ACLs) and encryption are often used to implement policies that determine which clients can access particular services and how secure these interactions must be. Both client-side and server-side mediators are necessary for implementing security protocols. Client-side mediators attach signatures to outgoing requests. Server-side mediators use the signatures to identify the client and then look-up ACLs to verify its authorization to make the request. The mediators may also encrypt communications between clients and servers. Flow Control. Flow control policies are designed to provide all components fair access to services. These policies may be implemented by both client-side and server-side mediators. For example, a client-side mediator may enforce limits on the number or the rate at which a client makes requests on a server. Similarly, a server-side mediator may limit the number of outstanding requests for service by rejecting new requests or requests from low-priority clients. Acceptance tests for inputs and outputs. Acceptance tests are used to ensure that clients do not make erroneous or ill-formed service requests and servers do not return erroneous or ill-formed responses. These tests may be implemented by both client and server mediators. Naturally, tests that depend on the state of the client or the state of the server are best implemented by client-side and server-side mediators respectively. Controlled introduction of new software. Long-lived systems must provide for the controlled introduction of new versions of critical servers. These new versions go through a verification period where they execute in parallel with the old version. Results produced by the new version are compared with results produced by the old version and, in case of disagreement, results from the old version are used. A serverside mediator may be used to support the controlled introduction of a new server: the mediator broadcasts services requests to both the old and the new servers, compares the results produced by the two, and sends the appropriate results back to the client. It may also log outputs produced by the new server. System Monitoring/Instrumentation. System monitoring and instrumentation are useful techniques for identifying performance bottlenecks and for detecting abnormal interaction patterns among components. Mediators may be used to collect information such as the number and frequency of requests made by clients,

2

average service times for servers, and utilization of resources such a memory and bandwidth. One such mediator is described in Section 5.

3 Mediator Implementation Techniques such as wrappers on system calls [RB00], operating system call interceptors [MR95], object request broker proxies, CORBA 2.2 interceptors, and application level server proxies may be used to implement mediators. Wrappers on system calls are appropriate for implementing client-side mediators. However they require clients to be re-linked with the new libraries or even recompiled if libraries are statically linked. Operating system call interceptors are used by the Fault-Tolerant Mach [MR95] system to ensure that applications make system calls with well-formed and reasonable arguments. These interceptors, called Sentries in the Fault-Tolerant Mach system, may also be used to implement mediators. Sentries intercept all inter-process communication and implement the desired mediator functionality. Proxies are commonly used in distributed object systems such as CORBA and Microsoft DCOM. Some systems allow user-defined code to be introduced into these proxies. Such proxies may therefore be used to implement client-side mediators. The CORBA 2.2 specification defines interceptors that can be used to intercept requests to a server before the request is serviced. This mechanism may be used to implement a server-side mediator. Section 5 describes a mediator built using this technique. Finally, an application level server proxy is an application component that acts as a front-end to the real server. Clients make service requests to this proxy which forwards the requests to the real server. Such a proxy can be used to easily implement a server-side mediator without having to trap system calls or communications within an ORB. Sections 4 and 5 describe the implementation of two mediators as application level proxy servers.

4 Flight Simulator Application We demonstrated the mediator and obligations based architecture by building a simple distributed flight simulator application. This application simulates an aircraft flying low over uneven terrain—the aircraft maintains a constant separation from the terrain by increasing or decreasing throttle settings. The application consists of two CORBA components. The first component is an auto-pilot that controls the throttle. The second component is a navigation component—it has a model of the terrain over which the plane is flying and computes throttle settings necessary to maintain constant separation from the terrain. The autopilot periodically invokes a method on the navigation component to get a throttle setting; the navigation component computes a new throttle setting and returns it to the auto-pilot. The algorithm for computing the throttle setting is time-consuming and takes a non-deterministic amount of time to converge. It is however a monotonically converging algorithm. Because the computation of throttle settings takes non-deterministic amounts of time the auto-pilot may not get new throttle settings in time to avoid crashing into the terrain. In fact, in our simulation the aircraft invariably crashed after a while because the navigation component did not respond fast enough. To make the navigation component deterministic we introduced into the system a server-side mediator to enforce timing obligations on the navigation component. The mediator intercepts requests to the navigation component and, if the component fails to respond within a specified time, raises an exception. This exception is caught by the auto-pilot that then invokes on the navigation component a method that returns an approximate throttle setting. The value returned is the one that was in the process of being computed but had not yet converged. The introduction of this mediator keeps the aircraft from crashing into the terrain.

3





















Mediator

Navigation



Auto-Pilot













































































4 1 2

3

CORBA BUS

Figure 2: Mediator that enforces timing obligations of the Navigation component The mediator for the flight simulator is implemented as a proxy server. The mediator, along with other components of the system, is shown in Figure 2. The arrrows in the figure show a call from the auto-pilot component to the navigation component being intercepted by the mediator.

5

Instrumentation

We demonstrated the use of mediators for instrumentation using a simple CORBA-based instrumentation server that receives and processes timing events from other components. An instrumentation mediator intercepts method calls to CORBA servers and generates instrumentation events for the instrumentation server. The system can be used to measure response times of any CORBA server. We have two versions of the instrumentation mediator: one implemented as a CORBA interceptor and the other as a server proxy. The inteceptor implementation requires the addition of one line of code to the server being instrumented and a recompilation of the server. The proxy server implementation needs no changes to the server.

6 Mediator Performance Overhead The performance overhead of CORBA-based mediators was measured using mediators that intercept null method calls from a client to a server but do no additional processing on the calls. All measurements were made using the Iona Orbix ORB running on Windows NT 4.0 systems. A proxy server type of mediator adds about 2.6 milli-seconds to the cost of a method invocation; this is equivalent to the cost of a null method invocation. However, this overhead is a very small fraction of total method execution times for typical distributed applications. An interceptor type of mediator has an insignificant overhead—it adds about 27 micro-seconds to each method invocation.

References [MR95] Z. Segall M. Russinovich, Fault-tolerance for off-the-shelf applications and handware, The 25th International Symposium on Fault-Tolerant Computing—Digest of Papers, June 1995. [RB00] N.M. Goldman R.M. Blazer, Mediating connectors: A non-bypassable process wrapping technology, DISCEX 2000: DARPA Information Survivability Conference and Exposition, DARPA, January 2000.

4