Components of Problem Solving and Types of Problems ∗ Joost Breuker † University of Amsterdam Department of Social Science Informatics Roeterstraat 15 NL–1018 WB Amsterdam, the Netherlands email
[email protected] tel (31) 20 525 3494/6798/6789 fax (31) 20 525 6896
Abstract A typology of problems is presented that is used for indexing and accessing reusable problem solving components in a library that supports the CommonKADS methodology for building knowledge based systems. Eight types of problems, such as planning, assessment etc., are distinguished, and their dependencies are explained. These dependencies suggest that the typology is to be viewed as a “suite” rather than the usual taxonomy of “generic tasks”. Developing the suite has lead to some new insights and elaborations of [Newell and Simon, 1972]’s theory for modeling problem solving. • Tasks are distinguished from problem definitions. A tasks is constructed by finding and configuring problem solving methods (PSMs), which are suitable for solving the (well-) defined problem. Tasks and PSMs therefore have a one to one correspondence ([O’Hara and Shadbolt, 1993]), while there is a one to many corresponce between a problem definition (type) and PSMs. • Three phases are proposed that turn spontaneous, ill-defined problems into well-defined ones, respectively. into problem solving tasks. • A complete solution consists of three components: a case model, an argument structure and a conclusion. The conclusion is a sub-part of both other components. • Tasks (PSMs) package recurring chains of dependent types of problems in variable ways. • The availability of behavioural models, or of structural/behavioural models in a domain determines to a large extent which types of problems can be posed and solved. ∗
From: L. Steels, W. Van de Velde & G. Schreiber (eds) A Future for Knowledge Acquisition, Proceedings of the EKAW-94, Springer-Verlag, Berlin, 1994, p 118 – 136 † The research reported is partially funded by the Esprit programme of the European Commission, P-5248, KADS-II. I would like to thank Andr´e Valente for commenting on this paper. A different, and far more elaborate version of the framework presented this paper can be found in [Breuker, 1994b]
1
1
Introduction
In the CommonKADS methodology for constructing knowledge based systems, a Library of reusable problem solving components takes a central role in top-down support of knowledge acquisition [Valente et al., 1993, Breuker and de Velde, 1994]. It performs the same role as the KADS-1 library of “interpretation models”, but differs in content and structure [Breuker et al., 1987]. The new Library’s content no longer consists of only highly abstract inference structures — one for each type of task — but of linked components of different grain-size levels, allowing for refinement by selection and combination to fit the application domain. The components come in a large variety. They may range from a full generic model that needs only to be instantiated by domain terms, to a simple inference function. The components are expressed in CML, the Conceptual Modeling Language [Wielinga et al., 1993], and can be semi-automatically translated into M L2 , the formal language of CommonKADS [van Harmelen and Balder, 1993]. The most abstract description of a component is a function. A function has input- and output roles. This Library has two divisions: the Task division, where problem solving methods (PSMs) are stored, and the Domain division, containing reusable ontologies. The Domain and Task divisions are linked by features (assumptions), representing interactions between PSMs and domain knowledge. In this paper, we are concerned with the Task division, and more precisely with one of its major accesses: a typology of problems. This typology differs in two major respects from current approaches, such as “role limiting methods” [McDermott, 1988], Generic Task [Chandrasekaran, 1983], [Chandrasekaran and Johnson, 1993], [Clancey, 1985]’s taxonomy, or the KADS-1 one [Breuker et al., 1987, Schreiber et al., 1993]. First, the typology is about “well defined” problems, and not about tasks. As will be explained below, a task consists of a problem definition and a configuration of PSMs. The problem definition is used to find an appropriate PSM in the Library. Second, the typology is not a taxonomy (see Fig. 1), but a “suite”, consisting of rational dependencies in the breakdown of any problem. See Fig. 7 for what is meant by a suite. It is based on dependencies between problems instead of a classification of their properties. What started as a revising the taxonomy of KADS-1 (Fig. 1), ended in redefining many taken for granted concepts such as task, solution, problem space, etc. To a large extent, this paper can be taken as revisiting Newell & Simon’s classical problem solving paradigm that is still prevailing in AI [Newell and Simon, 1972]. The results of this exploration are rather elaborations of this paradigm, than complete revisions or an alternative view. Some of these results are new. For instance, it appears that the solutions are not simple answers to questions, but complex objects, consisting of two strongly related parts or sides: a case model and an argument structure. This double side of a solution is also reflected in problem solving methods (PSMs) and the their “generate & test” structure.
2
- assessment - monitoring
- classification system analysis Q Q s prediction Q
- diagnosis - repair
- system
modification
- remedy
basic task
- system
synthesis
- elements
- configuration - scheduling
- structure
- refinement design
- s&e
- transformational
- none
- planning - open endeddesign
design
Figure 1: Taxonomy of “basic tasks”: the revised KADS-1 taxonomy of generic tasks [Aamodt et al., 1992, App.C.]
2
Problems and Tasks
The classical definition is that problems are some conflict between a current state and a goal state [Newell and Simon, 1972]. However, many (everyday life) problems are not so obviously goal related: An old lady wants to visit her friend in a neighbouring village. She takes her car, but halfway the engine stops after some hesitations. On the side of the road she tries to restart the engine, but to no avail. In this story, the goal is to visit a friend, and one may assume that the old lady’s plan to do so may take a multitude of intermediary subgoals, but none of these is in conflict with the current state of the malfunctioning engine. The term “goal” would become highly inflated if it should cover any expected state, including those assumed states as subsumed under the ‘frame problem’. The old lady’s assumption about a correctly functioning car is violated, when she perceives that there is no longer a correspondence between her foot’s pressure on the accelerator and the car’s speed. Therefore, it is more appropriate to speak about norm states than about goal states. Identifying a discrepancy between a current state and a norm or expected state leaves us with an ill defined problem. The discrepancy itself does not say anything in which direction a solution should be found. Problems are well-defined when there is a simple test to conclude that a solution is a solution. This view stated by [McCarthy, 1956] is at the basis of Newel & Simon’s description of these problems by their problem space [Newell and Simon, 1972, p. 72-78]. This notion has been widely accepted, but further discussions on this theme are rare. [de Velde, 1988] elaborates the notion 3
of problem space, as a triple Ω =< P, S, solution >, where P is the set of problems, S is the set of solutions, and solution is the relation that should hold between both sets. This definition explains the competence in problem solving, because it focuses on the relation between input (P ) and output (S). The meaning of this relation (solution) is generally that a solution (s) satisfies some problem (p). A diagnosis solution, consisting of a faulty component that explains the malfunctioning of some device, is an example. Besides ‘satisfies’, ‘is-consistent-with’ may be an interpretation of the solution relation (see section 3). From a performance perspective, the solution set P may not be known, because the problem space has not been completely explored, or the set may be infinitely large. Moreover, it takes a rather strong interpretation of McCarthy’s prescript of testability. Of course the simplest and most complete test to see whether a particular solution s is a solution is by finding it as a member of the set of possible solutions S. However, the test may imply no more and no less than whether some attributes are part of the solution s. From a performance perspective, requirements on solutions are available rather than sets of solutions. Therefore it appears here that a problem is defined by some abstract description of a solution, rather than some (enumerable or typical) set of solutions. When the solution is given in abstracto, problem solving is a process of refinement of an abstract solution with the help of more specific, but still generic domain knowledge, and of ‘filling’ the structure of the solution with the specific data from the current state. [Clancey, 1985] was the first to propose this combination of abstraction and refinement as the major operators in the heuristic classification method. [de Velde, 1988] took this view of Clancey, which is blurred by so many other interesting observations, in a more generic and principled way, and showed that all problem solving can be characterized by abstraction and refinement procedures, which come in two major but mixable flavours: classification and construction. We will return to this issue in the last section of this paper. A problem may be well-defined but that does not make it a task. For instance, the old lady may define her problem as one that should lead to an outcome that is some repair of her car. Note that many alternative problem definitions may have occurred. She might have, for instance, decided to find alternative means for transportation, or to give up her current plan. The difference between a well-defined problem and a task looks rather subtle. A problem can be solved, and a task is executed. The distinction is hardly noteworthy if we see in solving a problem the controlled execution of problem solving methods. However, with task we also mean something that can be repeated, that can be distributed and that contains a plan, i.e. a configuration of PSMs. A task differs therefore from a problem, because in a task the problem solving methods are known, and for a problem these still have to be found. Of course, in a more dynamic view where problem solving consists of defining new subproblems and solving these, the distinction is less sharp. However, in knowledge engineering the problems to be automated are so routine that they are tasks, rather than problems for human agents. People may complain that they have problems; not that they have tasks at their work. This means that problems are deviations and complications, and tasks are not. Going from an ill-defined problem to a task can be described as an ontogenesis of three steps (see Fig 3 for a summary). Problem Identification. The output of the first stage is a conflict between a current state and a norm state: these two states are inputs to problem identification. The conflict is a “sponta4
neous” problem: it has the role of a ‘complaint’. The complaint is the inducement or trigger for a (well defined) problem, a p in Van de Velde’s formula. The conflict is not the cause of the problem. Finding the cause of a problem is a problem by itself (diagnosis, or reconstruction). The conflict describes the difference between what is the case, and what should have been the case. What should have been the case, i.e. the norm, may have two different meanings. The first one is an expectation, i.e. a state that is predictable on the basis of knowledge about the world. The malfunctioning car of the old lady is an example. The second type of norm is a desire, i.e about states in the world that ought to be the case. The distinction between these types of norms is of importance, because different types of inference are involved (e.g. [Sergot and Jones, 1992], [Breuker, 1993], [Valente and Breuker, 1994]). Problem Definition. This second step takes as its input the conflict, and produces as its output some abstract solution, or, as we will see in the next section, a generic conclusion. A problem definition turns a spontaneous problem into a well-defined one. It establishes the problem space, Ω. A problem definition is a necessary step to provide sense to a problem. For instance, the old lady of our story may decide to have first her car diagnosed and repaired, and may then continue her journey. An alternative is, that she may have a taxi pick her up and have a garage take care of her car. Or she may give up the goal — visiting her friend — altogether. Problem definition is a planning activity, which is guided by the dependencies between abstract solutions (= problem types: see Fig. 7). The sense of a problem definition is the fact that the chain of the potential solutions in the plan aim at a goal. Task Specification. Well-defined problems change into tasks by finding or constructing appropriate PSMs. In organizations, recurring well-defined problems are planned as tasks, and distributed over agents and other resources. These scheduled tasks have often lost their relation with spontaneous problems, which means that tasks have not necessarily some violated norm as their direct origin. 1 . Task specification has as its inputs a defined problem, and one or more PSMs which should solve the problem. The controlled configuration (decomposition) of PSMs forms the ‘task-body’, while the problem definition is its ‘header’. Fig. 2 shows the structure of task knowledge. The “assumptions” slot of the task structure refers both to the required data, and to the required (generic) domain knowledge.
task HH H ) problem HH H task-body (PSMs) j definition type HH Z ~ Z ) HH j ? control specification = assumptions additional structure ? roles data
generic BBN
- obligatory solution control task - optional elements goal
Figure 2: Structure of task knowledge 1
Indirectly they have, as the goals of a task are the operationalization of some ‘desire’
5
PSMs goal
- task specification
problem type
- task HH j data
problem definition
conflict normproblem identification current state Figure 3: Three steps in going from problem to task
3
What’s in a Solution?
Problems are defined by their generic solutions. This is also the philosophy behind the various taxonomies of problem/task types refered to in Sec. 1. Newell & Simon observe that the term solution may mean different things [Newell and Simon, 1972, p. 76]. They distinguish between solution-objects, solution-paths, and solution-action sequences . The first is what we normally would see as a solution: some outcome or conclusion of the reasoning. For instance, the solutionobject in diagnosis is the set of faulty components. Solution paths take the reasoning itself as its objective. In finding a proof in a geometry problem we are not so much interested in the outcome — that was already given or hypothesized — but in the way it can be argued. Solution-action sequences are plans or instructions that lead to the required solutions: they are plans, or rather: problem solving methods (PSMs). In later literature one will find no explicit elaboration on this observation, but rather varied descriptions of what solutions consist of. A good example can be found in the work of Clancey. In [Clancey, 1985] he shows that in analytic types of problems, the solution is part of the knowledge base, i.e. the solution is to be found by classification rather than by construction. For instance, in medical diagnosis, the solution may be the name of one or more known diseases. The solution is here the conclusion of applying a PSM, and coincides with Newell & Simon’s “solution object”. However, in later publications, notably [Clancey, 1992], he convincingly argues that this is too meager a view, and that a solution is rather constructed as a situation specific, or case model (cf. [Steels, 1990]). At the end of the article (sec. 7) the emphasis is no longer on modeling, but on the way evidence supports the case model components. This is a “process model relational structure”, which “can be interpreted like a proof”. From these three distinctive aspects of solutions of problems, it is possible to synthesize the notion of a complete solution to a problem. We propose that a complete solution has three components. (1) The case model represents the understanding of the problem. (2) The conclusion is the answer to the question posed by the problem definition, while the supportive evidence forms (3) the argument structure that states what evidence supports the conclusion (see Fig 4). The conclusion is a part both of the argument structure, and of the case model. From the perspective of the argument structure, a conclusion is the “sentence” or theorem which is supported by all the evidence. Evidence may be empirical (data, requirements), or based on common knowledge,
6
complete solution
Z Z Z Z
case model
conclusion
argument structure
Figure 4: Components of a solution which may range from simple facts to as-valid-accepted structures or procedures of argument. From the perspective of a case model, the conclusion is that part of the case model which is of interest (input) in solving a next problem (see Sec. 6 for an explanation). The argument structure is not independent of the case model. It refers to facts or entities in the case model. The argument structure is linked to the case model by making references to its behaviours and/or components. Complete solutions are rare, or rarely made explicit, because one may be interested in only one component (e.g. conclusion, argument strcucture), or because it is assumed to be known by the ‘user’ (e.g. (large parts of the) case model). The roles of these three components of a solution with respect to communication to the user are the following : Explanation: The CASE MODEL has the role to explain the data. Justification: The ARGUMENT STRUCTURE justifies the conclusion. Result: The CONCLUSION is an input for another problem definition, or (sub)task. This (sub)task may be executed by another agent than the one who produced the solution (see also [Breuker and de Greef, 1993])
3.1
Problem Type and Components
The way the conclusion covers the other two components is not fixed, but depends on the problem type. For instance, if the problem is to find a proof, the conclusion coincides fully with the argument structure. In design, the conclusion — some structure — covers completely the case model, i.e. there is no distinction between these two components. 2 The argument structure consists of the set of explicitely satisfied requirements. For diagnosis problems, the conclusion consists of the identification of one or more faulty components, while the case model includes also the correct ones and their structural description [de Kleer et al., 1992]. Here the argument structure is built up from the tests performed to show that the faulty components are indeed faulty, and the correct ones correct — or can still be assumed to be working correctly. It should be noted that the argument structure is not simply the trace of some problem solving process: it is largely its restructuring in terms of argument relations between case model components and evidence. Assessment is concluded by a decision class or metric attributed to a case model [Valente and L¨ockhoff, 1994]. The argument structure for the complete solution of this type of problems is very thin. It suf2
In Fig 4 the dashed box ‘conclusion’ should then cover the box ‘case model’ completely.
7
fices to show that the terms used in the norms of the decision class are applicable to the case [Breuker, 1993]. Usually the case model is described as a situation specific version of generic domain models (e.g. [Steels, 1990]). Similarly, the argument structure can be viewed as a problem specific version of the rationale of a PSM (see [de Velde, 1993] about the “competence” of a PSM). The rationale of a PSM describes how and under what assumptions a method yields valid results.
3.2
Problem Solving Methods and Components
A good example of the different contributions to the solution components is provided by PSMs for solving diagnosis problems of devices. The conclusion of such a diagnosis consists of the set of abnormal component(s). This set is the only relevant one for the dependent problem: the repair of the device. However, some of these PSMs may as well be aimed at the construction of a more complete case model by identifying the normal components as well. Requirements on the completeness of the solution has direct consequences for the cost-effectiveness of a PSM. It is cheaper to identify only faulty components, instead of also the correctly working ones, because a single deviant value is sufficient evidence for a fault, whereas establishing the correct working of a component may involve checking its full behavioural repertory. Minimal solutions may be tractable, where a slightly more complete solution may easily make the problem too complex [de Kleer et al., 1992]. PSMs may also differ whether they reach the conclusion by modeling or by argumentation. This can be illustrated by the two major approaches in model based diagnose. 3 In consistency based diagnosis (e.g. GDE [deKleer and Williams, 1987]), the abnormal components are identified when their behaviour is inconsistent with the model of their correct behaviour. These conclusions are reached in a deductive way, and primarily by argumentation. The behaviour of the smallest set of remaining components that cannot be explained by models for normal components “must be” abnormal components. The price payed is that we have an incomplete case-model, because we do not know what the faulty behaviour of the abnormal components is. Note also that the other components are assumed to be working properly, because no falsifying evidence has been obtained in testing. Therefore, also the argument structure is incomplete, but the conclusion — the set of abnormal components — is completely justified. In abductive diagnosis, the abnormal components are not only identified, but they are modeled as well. Generic models of faulty behaviour are matched against the data to obtain explanations (covers). The case model, therefore, explains the abnormal components [Console and Torasso, 1991]. Abductive PSMs can only be justified by assuming the domain to be a closed world. It is easy to see how consistency based diagnosis can be extended by the use of fault models. In this way abductive and consistency based approaches are complementary (e.g. [Console and Torasso, 1991], [de Kleer et al., 1992]). 3
We will use the convention to denote tasks or methods by a verb — e.g. (to) diagnose —, while problem types will be indicated by nouns, e.g. diagnosis. Sometimes, this convention is hard to maintainfor PSMs, because of the use of adjectives to denote a specific method, e.g. in heuristic classification.
8
4
Problem Types
Problems can be characterized by their (minimal) solutions, i.e. their generic conclusions. A generic conclusion is an abstract description of an object that covers the set of conclusions. The generic conclusions can be thought of as types, and there exists also a fair consensus on a number of these types. Diagnosis, design, planning are well established ones. We will identify another five types: assessment, monitoring, modeling, assignment (including scheduling and configuration) and prediction. We do so, not only because some of these (assignment, monitoring) are still the largest common denominators in the various typologies proposed for knowledge acquisition (see also [Breuker, 1994]), but also because these additional types fill up holes in the systematic view we will develop in the next section. These definitions are still tentative and will be the object for further study, in particular by formalization. For some problem types formalizations can be found in the literature: [de Kleer et al., 1992] for diagnosis; [Poeck and Gappa, 1993] for assignment. Modeling is concerned with the identification of what is a system and what is its environment, or more precise: what is its interface with the environment. This interface is consists of behaviour (input/output state dependencies): a behavioural model. The model may be a partial one, as in functional specifications. Design has as its conclusion a structure of components and their connections. The components may be objects or processes, physical or symbolic. Planning delivers sequences of causally/teleologically connected states (behaviour) in the world, which may be filled by actions. If the final state(s) are not intended future states (goal states), but states in the past, the planning problem is called reconstruction. Assignment distributes (additional) elements (components, actions) over a structure. If the structure is a plan, the assignment problem is generally called scheduling. If the structure is a design the problem is often called configuration. Prediction delivers the resulting states of a system over time, starting with an initial state. When one derives an initial state from some output states one may speak of postdiction. Monitoring yields a discrepancy between a predicted state and an observed state. Assessment provides a measurement that classifies behaviour according to norms. Diagnosis finds components or structures which conflict with their behavioural model or design. Table 1 summarizes the typology.
5
A Framework for Analyzing Tasks
By finding relevant distinctive properties, these generic conclusions may be categorized in a taxonomy. The advantage of a taxonomy is that it provides coherence (abstraction; exclusion). However, for a typology of problems, a single taxonomy appears to work as a Procrustian bed. There 9
major type synthesis
modification analysis
type of problem modeling design planning/reconstruction assignment (scheduling, configuration) prediction monitoring diagnosis assessment
generic conclusion behavioural model structure of elements sequence of actions distribution/assignments state of system discrepant states faulty element class/grade attribution
Table 1: Type of problem and corresponding generic conclusions is a wide consensus on the first major distinction, i.e. between synthesis and analysis problems. [Clancey, 1985] gives a most eloquent view on this difference by arguing that analysis problems deal with operations on known systems — “models of systems in the world” — while in synthesis problems there is not yet a model-of-a-system available. However, beyond this distinction, further refinements become problematic, because distinctive properties represent different views. For instance, synthesis problems can be distinguished by the number and nature of dimensions of the generic conclusion. A plan is one dimensional, over ‘time’. A design is multi-dimensional and generally involves ‘space’. Moreover, these problem types may also be distinguished by the nature of internal requirements, i.e. whether the elements and/or the structure of the system to be designed or planned are given, or have to be found (see Fig. 1). A third problem with these taxonomies is that when they go deeper than one ply, consistency is easily lost. Therefore, we have looked for another principle to arrive at a collection of problem types, which has the same properties as taxonomies (exclusion, covering), but not the problems that are the consequence of ‘multiple inheritance’. These have been found in looking at the the way problems are related to one another by the fact that the results of one problem are necessary to define a dependent problem. These dependencies, i.e. the generic conclusions, between well-defined problems are a more consistent organizer of problem types than a taxonomy, and have moreover the advantage of a practical analytic tool for identifying configurations of problems and tasks, as they occur in real life.
5.1
Some Typical Tasks and Their Problems
The suggestion to look rather for the dependencies between problems as they are packed in a task, than for a set of distinctive properties of generic conclusions that could serve as the basis for a taxonomy, comes from experiences with the KADS-1 taxonomy of generic tasks ( see e.g. [Breuker et al., 1987, Ch. 8, Real Life Tasks]). In real-life applications, a problem type hardly ever comes alone, but rather in chains that reflect the fact that the conclusion of one problem is a required input to solve a next problem. These dependencies between problem types — i.e. their generic conclusions — can be easily overlooked because task decompositions (PSMs) may break up the task in sub-tasks which may cover more than one problem, and one problem may be solved by more than one subtask. In the next sections we will provide evidence for the view that by distinguishing the problems from the tasks, common and rational dependencies between problems 10
become visible.
Problems in Diagnose Tasks The minimal solution of a problem is a conclusion. Conclusions may be needed to resolve a next problem. The conclusion of a diagnosis problem — a set of faulty components — is a necessary input for a repair task. The repair is dependent on the results of a diagnosis problem. In well organized, routine problem solving, the problems and their dependencies are packed into tasks and task decompositions. This packing is provided by a PSM, and each PSM may pack problems in a different way. Here we will discuss some examples from diagnose tasks. A very simple version of a diagnose task is exemplified by the a common practice in finding faulty computer components by replacing the boards in a systematic way. This replacement is a repair, and can be conceived as a very simple assignment problem in which a new component is assigned to the position of a failing or suspect component. In this simple task, the diagnosis and assignment problems are configured in such a way, that repairs are used to test diagnosis hypotheses, and the solutions to the diagnosis problem are used as hypotheses. This alternation between simple hypothesis generation and test-by-repair stops, when a repair yields a correctly functioning device (see Fig 5). diagnose-task (2) "
generate/ discriminate hypotheses
" " evidence hypothesis
l l l -
test hypothesis assignment
diagnosis
Figure 5: Decomposition of a simple diagnose task with implied problem types (in italics). The test-task consists of solving a repair problem However, this configuration will only work under specific conditions (cost of replacement, easily decomposable device etc.) and assumptions (single fault). In most other practical diagnose tasks the solution of repair problems is postponed until this task has been definitely concluded, for instance, when a single fault has been identified. In the diagnose-by-repair task there is still a one to one correspondence between the problems to be solved and the subtasks. The generate-hypotheses subtask covers the diagnosis problem, and the repair subtask stands for solving the assignment problem. Another, more common structure of a diagnose task with a different set and distribution of problems is the following (see Fig 6). A diagnosis problem can only exist, when some malfunction or complaint has been observed. A malfunction or complaint is a discrepancy between what a system should do according to its design/model and what is observed. Stating how the system should behave — given some input — is the result of some prediction. This prediction problem can be solved by derivation from
11
structural/behavioural models, or by instantiating a behavioural model (e.g. a causal model, or an associational model based on empirical correlates). However, in this diagnose task one is not free to choose from either option. If the diagnose task indeed implies a real diagnosis it should lead to pointing out which components in the structure are the cause of the complaint. It should take a structural model as its road map, and behaviour models of the components as input for prediction, or testing. The requirements of the diagnosis problem dominate the dependent test subtasks. A combination of prediction and monitoring of the actual behaviour of the components of a device provides the primary ingredients for a test task. The test task, or rather the conclusion of the monitoring problem in this task, may result in a discrepancy. This discrepancy may play two roles. The first one is the input-role for a diagnosis problem, as a complaint. In this way a diagnose task can be triggered, because a fault has been monitored. The second role is an ‘evidence’ role, and is also an input to a diagnosis problem. In that case, the test has become a proper subtask in a diagnose task, rather than a ‘front-end’ task to a diagnose task. This distinction can be expressed in the control over the test (sub)task and diagnose task. In the first role, the test task is followed by a diagnose task; in the second role, the test task is a subtask and is used in an iterative way to refine the hypothesized diagnoses (output of the diagnosis problem). The conclusions of the diagnosis problem take the role of hypotheses. The diagnosis problem does not have a single solution, but all solutions. However, the diagnose task uses the hypothesis generation and empirical testing cycle — a PSM — to bring this number down to a minimal set that explains the malfunction. It is the control over these problems that constitutes the task and provides roles to the conclusions of these problems. Note that the generic nature of the problems remains the same, while for each iteration the specific problem to be solved is a different one. diagnose-task (1) " l " " l l generate/ evidence/complaint test discriminate - hypothesis hypotheses hypothesis " @ " " diagnosis @ " @ - compare predict XXX observe XX X :
prediction
monitoring
- dependency
Figure 6: Decomposition of a typical diagnose task with implied problem types (in italics) In Fig 6 the relation between this diagnose task decomposition, and the implied problems is presented. Whatever the packing of problems in these diagnose tasks (and its monitoring “front-end”, respectively assignment (repair) “back-end”), a common chain of dependencies between the types of problems can be abstracted: prediction → monitor → diagnosis → assignment Other types of problems may enter the diagnose task as well, in particular in providing evidence 12
(test roles). We have already mentioned the assignment problem, that can be used in two roles: as a test, and as a “back-end” to the diagnose task proper. However, more complicated versions of diagnose may occur. For instance, when observations are difficult or impossible to obtain in a direct way, test may be performed by reconfiguring a system. For instance, a cheap test for stereo-audio equipment is to switch channels, when the complaint is in one of the channels. The configuration problem that is solved is not part of a test, and does not feed in one of the problems of the test, but it changes the diagnosis problem, because the system description of the device has been changed. This dependency with the configuration problem can be simply added to the chain above (we will use here the standard term assignment for configuration): assignment → prediction → monitor → diagnosis → assignment Another type of problem that can be observed in real life diagnose tasks is planning. The sequence (or tree) of hypothesis testing can be governed by a plan or schedule e.g. as to maximize the effectiveness and minimize costs of tests (cf. MOLGEN, [Stefik, 1981]). The chain of problems is as follows: planning/assignment → prediction → monitor → diagnosis → assignment
Model and Design Tasks; Levels of Understanding Before problems with a system can be identified and diagnosed, the system has to be understood. This means that a (case)model has to be constructed. Generic versions of such a model may already be available, thus reducing the model task to an instantiation or adaptation of this generic model of a system. For instance, we may know the global structure and behaviour of amplifiers, which we may use to diagnose a particular type and exemplar. Understanding is not an all or nothing process. For all practical purposes, a system can be understood at three levels: • By classification: the properties of a system are known, which distinguish a type of system from another type. For instance, a taxonomy of bacterial diseases is a very simple model of malfunctions of the body caused by bacteria. • By understanding its behaviour, i.e. by modeling how changes in the environment, i.e. input to the system, affect the state of the system. • By understanding its working. This implies not only to have a model of its behaviour, but also a description of what causes this behaviour. There are two major types of causal models: – The causal model describes the behaviour of a model by intermediary, internal processes or states of a system. For instance, in describing diseases in terms of processes (syndromes), intermediary states such as “immunosuppressed” may provide explanations to account for bacterial diseases which normally would not interfere with the normal processes in the body. 13
– The deepest form of understanding consists of relating the causal model to a (physical) structural description that explains by its connections how causes flow through a system. The components of such a system are viewed as subsystems, for each of which a behavioural model may be available, for which again a structural description may be used for more detailed understanding. An example of a structural model is the wiring diagram of an electrical circuit. When we use the term causal model, we mean in general only the first type of model, while the one that has a mapping between causal chains and physical connections or components is more precisely denoted by the term structural/behavioural model. A structural model of a system is the result of solving a design problem. In the construction of artifacts (devices), the design problem is solved by a design task. 4 However, we use the term to model, when it concerns the design of causal or structural models of natural systems. Both model and design tasks appear to produce the same kinds of models, and there are more striking similarities. A design or a model is tested. A design is tested against its functional requirements, i.e. against the behaviour that it is intended to exhibit in interacting with its environment (including a user), while an empirical model is tested against its predicted behaviour. These tests look exactly the same as the tests performed in diagnose task; the difference is that this testing is not focussed on explaining a particular discrepancy, but may imply the full behavioural repertoire of the designed or observed system. There is a growing literature on design and model tasks. A typical example of the PSMs for these tasks can be found in the work of Chandrasekaran. As a first decomposition of these tasks, the following subtasks are identified: “Propose, Verify, Critique, Modify (a design)” [Chandrasekaran, 1990]. Other versions of the same decomposition can be found in the “propose & revise” PSM for configure tasks [Marcus and McDermott, 1988], or the “generate, test & debug” paradigm of [Simmons, 1992] for plan and reconstruct tasks. The Propose subtask is the only task that contains the design problem. The Verify, Critique and Modify subtasks represent process control cycles, which check whether the current state of the design is sufficiently close to (a subset) of the requirements. Therefore, the design task appears to imply the following chain of problems, where Verify implies monitoring, Critique diagnosis and Modify an assignment:
design → prediction → monitor → diagnosis → assignment A design problem is solved by the construction of a structural model. The nature of the structure may be explicitly required — internal requirement —, or may be part of a design option. The structure is created to satisfy functional/behavioural (external) requirements. These are the result of solving a modeling problem. Within a model/design task one may often find the design problem and the next dependent assignment problem quite entangled, because the design problem delivers the structure, while the assignment fills it with elements. Because of interaction problems [Simmons, 1992], it is far more efficient to solve both problems in the same refinement cycles. A similar weaving is found in schedule tasks where the planning and assignment problems are often 4
Here we intend the verb “to design”; not the substantive “design” which we use for problems.
14
solved in a combined way [Valente, 1994, Sundin, 1994]. The same can be the case for diagnose and repair (assignment) [Friedrich, 1993]. Therefore, the full chain implied in model or design tasks looks as follows: modeling → design → assignment → prediction → monitor → diagnosis → assignment/design
Similar analyses as those for diagnose, model and design tasks can be performed for other kinds of tasks. In [Breuker, 1994], other tasks are described. For instance, in process control and device manipulation one finds the follwing chains: planning/assignment → prediction → monitor → diagnosis → assignment
planning/assignment → prediction → assessment → assignment Similar chains are to be found in test, evaluate and remedy tasks. and diagnose tasks which are aimed at the reconstruction of actions that have lead to a failure.
6
A Suite of Problem Types
These chains of problems which are identified in a large variety of tasks exhibit congruent patterns, which can be integrated into a graph: a suite of problem types (see Fig. 7). synthesis modification
analysis
behavioural view (system*environment) assessment
planning >
assignment prediction HH 3 j H @ R @
modelling Q s Q
-
design
monitoring - diagnosis-
structural view (system)
Figure 7: A suite of problem types, dependencies and views The graph can be divided into two overlapping perspectives: a behavioural and a structural one. A behavioural perspective is taken when its is not possible or feasible to control a system by encapsulating causal effects into physical structures, or to know how causal effects ‘propagate’ through the physical structure. In that case we have to plan or assess actions. By planning we 15
make a spatio-temporal arrangement that enables causal propagation for a specific situation. For instance, a travel plan (schedule) consists of local and temporal contingencies (“connections”) and actions (components). Therefore, the perspectives are relative, and the graph may be collapsed into a single chain. However, we think that the split is highly relevant, because it reflects two ways to control our world: by material technology where we pack causality in (increasingly complex) devices, which leads to mechanization and automation, and by enabling chains of actions: the latter being the default option. In a very abstract sense we can now explain the ontogenesis of problems and their packing into tasks by PSMs, as a tracing — mainly backtracking — along the suite. Spontaneous, ill-defined problems are identified by solving (permanent) monitoring or assessment problems. The spontaneous problem is an indication that somewhere earlier in a dependent problem a solution is no longer valid. This backtracking in problem definition has a minor or a major scope over the suite, dependent on the nature of available domain knowledge and information (data) in the situation. The minor type of problem definition consists of a refinement of the conflict (ill defined problem) to allow a minimal, local change — modification — to a system (repair) or a course of action (remedy): both modifications involve an assignment problem. The required refinement may be obtained by solving a diagnosis problem. In diagnosis faulty components instead of the faulty system are identified. Refinement may also be obtained by assessing the conflict, e.g. in terms of classes of malfunctions. If the problem cannot be refined further or cannot be solved by assignment (repair; remedy), more drastic problem solving is required, i.e. one has to start at the modeling entry of the suite. In this way, a problem definition may consist of more than a single type of problem. As we have seen in Sec 5.1, a task covers in general more than one problem type. This is for two different reasons or rationales. The first one is based upon the fact that more than one dependent problem may have to be solved, as in the case of a diagnosis and an assignment (repair). We have seen the packing of combinations of problems by a single method/task in particular for reasons of efficiency (see e.g. [Friedrich, 1993]). However, we have also found other chains of problems. In all those cases these problems are packed into tasks because of the different roles that are involved in arriving at a complete solution. The rationale for these additional tasks is not that they are intermediary steps towards some goal, but that they constitute a test or evaluation cycle that builds the argument structure of a solution. Despite the fact that this control cycle is never a main task, but a rather universal subtask, it also follows the dependencies in the suite. For a further elaboration of the consequences of the reappearence of the ‘generate & test’ paradigm, the reader is refered to [Breuker, 1994].
7
Conclusions
These new conceptualizations of the elements of solutions, and the problem typology have a number of advances: • By distinguishing problem definitions from tasks a smaller, less variant set of types can be identified. Tasks and their decompositions hide the problems they are to solve. More16
over, the names and ‘real life’ appearances of tasks such as ‘operation control’, ‘(medical) diagnosis’, ‘trouble shooting’, ‘verification’ etc. obscure the their commonalities and distinctive features. The typology of problems is at least more parsimonious and appears thus far sufficient to characterize the problems that are solved by (real life) tasks. • It is assumed that this parsimonious typology also provides a more strict classification of problem solving methods (PSMs). Ideally, for each problem type exclusive PSMs should be identified. There is not yet enough evidence to warrant such a claim, but we have at least shown that by disentangling problem types from (sub)tasks we get a better view of what problem(s) a PSM is supposed to solve, and what task-goals it is aimed to accomplish. For instance, it can be understood why many PSMs for medical diagnose are not diagnostic methods at all, but rather aimed at assessment (normative classification) (e.g. [Clancey, 1985]). • The view that a complete solution to a problem has two major components or ‘sides of the same coin’ — a case model and an argument structure — clarifies why some PSMs justify conclusions (partial solutions) — e.g. tests which check observations against hypotheses — and other PSMs focus on a coherent interpretation — e.g. causal abduction. More in general, the ‘generate and test’ paradigm that is observed in all complex tasks reflects the separation of the construction of the case model and of the argument structure. • Conclusions are that part of a case model and/or argument structure, that answer the question implied by a problem. Conclusions are necessary inputs to solve a ‘next’, related problem, and for that reason sufficient as a minimal solution to a problem. • The pattern of dependencies between problem types forms a “suite”, i.e. rational sequences of conclusions, which start in principle with modeling, but may be triggered by monitoring or assessing (one’s own) activities. The suite avoids the inconsistencies in (single view) taxonomies of problem, or task typologies proposed in the literature. Moreover, it provides guidance for understanding the structure of real life tasks. • The genesis of problems and tasks may be modeled by three steps: – A problem is identified as the discrepancy between some norm(al) state and an actual state. – The ill-defined problem is turned into a well-defined one, i.e. one for which a generic conclusion is found. The problem type suite reflects the typical (sequences of) generic conclusions. – In task-specification for a well-defined problem appropriate problem solving methods (PSMs) are found or constructed. Besides these practical consequences of the rather theoretical exercise presented here, the more specific application is of course in the Common KADS library tools, as currently under development. Further research is going on, aimed at a formalization of the problem types to assess the detailed structure of the problem types (generic conclusions) and the consistency of the dependencies.
17
References [Aamodt et al., 1992] Aamodt, A., Bredeweg, B., Breuker, J., Duursma, C., L¨ockenhoff, C., Orsvarn, K., Top, J., Valente, A., and van der Velde, W. (1992). The CommonKADS Library. Deliverable 1.3. KADS-II/T1.3/VUB/TR/005/1.0, KADS Consortium, Free University, Brussels. [Breuker, 1993] Breuker, J. (1993). Modelling Artificial Legal Reasoning. In Aussenac, N., Boy, G., Gaines, B., Linster, M., Ganascia, J., and Kodratoff, Y., editors, Knowledge Acquisition for Knowledge Based Systems, Proceedings of the EKAW-93, pages 66 – 78, Berlin. Springer. [Breuker, 1994] Breuker, J. (1994). A suite of problem types. In Breuker, J. and de Velde, W. V., editors, The CommonKADS Library for Expertise Modeling. IOS-Press, Amsterdam. [Breuker and de Greef, 1993] Breuker, J. and de Greef, P. (1993). Modelling system-user cooperation. In Schreiber, G., Wielinga, B., and Breuker, J., editors, KADS: a Principled Approach to Knowledge Engineering, pages 47 – 70. Academic Press, London. [Breuker and de Velde, 1994] Breuker, J. and de Velde, W. V., editors (1994). The CommonKADS Library for Expertise Modeling. IOS-Press, Amsterdam. [Breuker et al., 1987] Breuker, J., Wielinga, B., van Someren, M., de Hoog, R., Bredeweg, B., Wielemaker, J., Billault, J.-P., Davoodi, M., and S, H. (1987). Model Driven Knowledge Acquisition: Interpretation Models. Esprit P1098 KADS A1, University of Amsterdam. [Chandrasekaran, 1983] Chandrasekaran, B. (1983). Towards a Taxonomy of Problem Solving Tasks. AI Magazine, 4(1):9 – 17. [Chandrasekaran, 1990] Chandrasekaran, B. (1990). Design problem solving: a task analysis. AI Magazine, pages 59–71. [Chandrasekaran and Johnson, 1993] Chandrasekaran, B. and Johnson, T. (1993). Generic tasks and task structures. In David, J.-M., Krivine, J.-P., and Simmons, R., editors, Second Generation Expert Systems, pages 232 – 272. Springer, Berlin. [Clancey, 1985] Clancey, W. (1985). Heuristic classification. Artificial Intelligence, 27:289–350. [Clancey, 1992] Clancey, W. (1992). Model construction operators. Artificial Intelligence, 53:1– 115. [Console and Torasso, 1991] Console, L. and Torasso, P. (1991). A spectrum of definitions of model based diagnosis. Computational Intelligence, 7(3):133 – 141. extension of paper from the ECCAI-90 Proceedings. [de Kleer et al., 1992] de Kleer, J., Mackworth, A., and Reiter, R. (1992). Characterizing diagnoses and systems. Artificial Intelligence, 56(2–3):197 – 222. [de Velde, 1988] de Velde, W. V. (1988). Inference structure as a basis for problem solving. In Kodratoff, Y., editor, Proceedings of the 8th European Conference on AI, pages 202 – 207, London. Pitman. 18
[de Velde, 1993] de Velde, W. V. (1993). Issues in knowledge level modelling. In David, J.-M., Krivine, J.-M., and Simmons, R., editors, Second Generation Expert Systems, pages 211 – 231. Springer, Berlin. [deKleer and Williams, 1987] deKleer, J. and Williams, B. (1987). Diagnosing multiple faults. Artificial Intelligence, 32:97–130. [Friedrich, 1993] Friedrich, G. (1993). Model-based diagnosis and repair. AI Communications, 6(3/4):187 – 206. [Marcus and McDermott, 1988] Marcus, S. and McDermott, J., editors (1988). Knowledge Acquisition for Expert Systems. Kluwer, Reading MA.
Automating
[McCarthy, 1956] McCarthy, J. (1956). The inversion of functions defined by Turing machines. Automata Studies, Annals of Mathematical Studies, 34:177 – 181. [McDermott, 1988] McDermott, J. (1988). Preliminary steps towards a taxonomy of problemsolving methods. In Marcus, S., editor, Automating Knowledge Acquisition for Expert Systems, pages 225–255. Kluwer Academic Publishers, The Netherlands. [Newell and Simon, 1972] Newell, A. and Simon, H. (1972). Human Problem Soving. Prentice Hall, Englewood Cliffs, NJ. [O’Hara and Shadbolt, 1993] O’Hara, K. and Shadbolt, N. (1993). Knowledge Acquisition, 5(4):449 – 481.
Locating generic tasks.
[Poeck and Gappa, 1993] Poeck, K. and Gappa, U. (1993). Making role-limiting shells more flexible. In Aussenac, N., Boy, G., Gaines, B., Linster, M., Ganascia, J., and Kodratoff, Y., editors, Knowledge Acquisition for Knowledge Based Systems, Proceedings of the EKAW-93, pages 103 – 122, Berlin. Springer. [Schreiber et al., 1993] Schreiber, G., Wielinga, B., and Breuker, J. (1993). KADS: a Principled Approach to Knowledge Engineering. Academic Press, London. [Sergot and Jones, 1992] Sergot, M. and Jones, A. (1992). Deontic Logic in the Representation of Law. Artificial Intelligence and Law, 1(1):45 — 64. [Simmons, 1992] Simmons, R. (1992). The role of associational and causal reasoning in problem solving. Artificial Intelligence, 53(2–3):159–207. [Steels, 1990] Steels, L. (1990). Components of expertise. AI Magazine, 11(2):28–49. [Stefik, 1981] Stefik, M. (1981). Planning with constraints (MOLGEN: Part 1). Artificial Intelligence, 16:111 – 140. [Sundin, 1994] Sundin, U. (1994). Assignment and scheduling. In Breuker, J. and de Velde, W. V., editors, The CommonKADS Library for Expertise Modeling. IOS-Press, Amsterdam. [Valente, 1994] Valente, A. (1994). Modeling components for planning problems. In Breuker, J. and de Velde, W. V., editors, The CommonKADS Library for Expertise Modeling. IOS-Press, Amsterdam. 19
[Valente and Breuker, 1994] Valente, A. and Breuker, J. (1994). A commonsense theory about normative systems. In Breuker, J., editor, Proceedings of the ECAI Workshop on Artificial Normative Systems. ECAI. [Valente et al., 1993] Valente, A., Breuker, J., and Bredeweg, B. (1993). Integrating modelling approaches in the Common KADS library. In Sloman, A., Hogg, D., Humphreys, G., Ramsay, A., and Partridge, D., editors, Prospects for Artificial Intelligence, Proceedings of the AISB-93, pages 121 – 130. IOS Press, Amsterdam. [Valente and L¨ockhoff, 1994] Valente, A. and L¨ockhoff, C. (1994). Assessment. In Breuker, J. and de Velde, W. V., editors, The CommonKADS Library for Expertise Modeling. IOS-Press, Amsterdam. [van Harmelen and Balder, 1993] van Harmelen, F. and Balder, J. (1993). M L2 : a formal language for KADS models of expertise. In Schreiber, G., Wielinga, B., and Breuker, J., editors, KADS, a principled approach to knowledge based system development, pages 169 – 201. Academic Press. [Wielinga et al., 1993] Wielinga, B., de Velde, W. V., Schreiber, G., and Akkermans, H. (1993). Expertise model definition document. Esprit P5248 KADS-II/M2/UvA/026/1.1, University of Amsterdam. date May 24 1993.
20
