Supplementing Process-Oriented with Structure-Oriented Design Explanation within Formal Object-Oriented Method
Lemai Nguyen*
[email protected] Paul A. Swatman*
[email protected] Graeme Shanks**
[email protected] *School of Management Information Systems, Deakin University, Vic, Australia **School of Information Management and Systems, Monash University, Vic, Australia Abstract This paper reports the results from an action research project which studies the benefits of documenting the evolution and the rationale for the evolution of a requirements specification. Earlier research into the use of ad hoc design explanation, in which design decisions were documented using the IBIS notation (Issue Based Information System) as they were made, found both benefits and weaknesses of the approach. The weaknesses suggested an investigation into the complementary use of a post hoc notation--QOC (Question-Option-Criteria). This study was undertaken with a view to understanding the potential contribution of supplementing the IBIS approach with additional rationale reported using QOC. This study demonstrated that both notations were useful. The study resulted in a deeper understanding of the process of documenting the evolution of the requirements and the rationale for the evolution. The study also led to a few suggestions about how to use both the QOC and IBIS notations effectively.
1. Introduction This paper reports the results from an action research project which studies the effects of
documenting both the evolution and the rationale for the evolution of a requirements specification. We were encouraged in our objective for a number of reasons. There are indications from the literature that design explanation1 may offer designers many benefits, such as: • improvement of decision making. Design rationale may raise crucial issues, rationalise discussions and avoid missing important considerations about requirements by each designer as well as by a group of designers [8, 5, 6]. • reuse of rationale arguments in similar situations and related projects [25, 6, 8]. We believe that this avoids solving the same problem of making the same decisions over and over again as the specification evolves. • explanation of a specification. It is important to understand a specification in terms of its relationship to possible alternatives and/or other similar projects [16]. • assistance in communication between various participants including stake holders, users, specifiers and other developers [6, 16].
1
Generally, design rationale is information which explains and justifies the reasoning behind a specification. In our approach, design rationale is used with emphasis on explanation.
Earlier research into the use of ad hoc design explanation, in which design decisions were documented using the IBIS notation (Issue Based Information System) [5] as they were made, (reported in [17]) found both benefits and weaknesses of the approach. The benefits included the following: • IBIS provides a useful basis for the description of the FOOM process, including discussions and decisions about requirements, • the clear structure of IBIS elements, the network representation of requirements evolution and the classification of arguments and their attachment to the FOOM specification and the evolution network support the audit trail of FOOM specific requirements including class/object identification and their evolution, • the IBIS notation supports communication between participants2. The weaknesses of the approach, however, were found to include: • the implicit evaluation of IBIS arguments. Although there may be a number of Positions3 for a particular Issue, each Position is assessed in isolation by its own pro and con Arguments rather than by a common set of Arguments. • the locality of arguments. Because IBIS was used non-intrusively within the requirements engineering process, each Issue discussed only a partial requirements problem. The logical analysis of the overall problem could not be performed. Each Issue is solved in isolation and with little consideration of the Context around the Issue and/or of other related Issues. This leads to great difficulty when later evaluating different FOOM specifications. These weaknesses suggested an investigation into the complementary use of a post hoc approach to design explanation---QOC (Question-Option-Criteria [16]. The research reported in this paper concerns an action research study undertaken with a view to understanding the potential contribution of supplementing the extended IBIS approach of our earlier work with additional rationale reported using QOC. The paper structure is as follows: • a description of the project, • our observation and interpretation of what happened,
• our reflection upon the benefits of QOC and IBIS and how to use them effectively, and • a conclusion drawn from the findings. Our interpretation and reflection will show how QOC was used, how it addressed the weaknesses of IBIS and what new problems were raised from using both the notations.
2. Background for the project 2.1. Formal Object-Oriented Method The requirements engineering process used within the project was Formal Object-Oriented Method (FOOM) [23]. Briefly, FOOM is based on a synthesis of socio-organisational theory [4], the object-oriented approach MOSES [12] and the mathematical formal specification language Object-Z [7]. An introduction to FOOM is included as appendix A. For the purpose of this paper, however, it is sufficient to understand that the specification developed during the FOOM process is definitively represented using the formal language Object-Z [7] and illustrated with text and structured diagrams based on MOSES [12], a predecessor to OML. The issue of generalisation of research outcome was considered when choosing FOOM as a specific requirements engineering method. However, at an abstract level, all major requirements engineering methods are based on the principle of an evolving requirements model and therefore, while the details may vary, this central principle remains constant. From this point of view, we believe that the research could be extended to other requirements engineering methods.
2.2. The post-hoc design explanation---QOC Suggested by findings from earlier research [17], QOC was used to complement IBIS in this research. Here, we will describe the notation and explain why it was chosen. More details about design rationale and the IBIS notation are included in appendix B. Option
Question
Criterion
Criterion
Option
Criterion
2
We use participants to refer to all people involved in the delivery of a complete FOOM specification including specifiers and users. 3 Issue/Position/Argument with capital letters refers to element of an IBIS Issue, argument/issues with lower case letter are used in general meaning.
Question
Figure 1. The QOC notation [16]
QOC directly addresses the requirements of the artefact (by criteria) and emphasises a post hoc structuring of the design space through specific questions and a comparison of investigated alternatives delineated by Options (see figure 1). Design space analysis does not represent the historical record of the design but is a co-product of design [16]. The fact that the two design rationale approaches were desirable and their two notations would be used within FOOM imposed the selection of a simple notation to learn and use and yet one sufficiently expressive. QOC is simple to learn and use and there have been a growing number of intensive research projects into QOC mainly at Rank Xerox Cambridge EuroPARC (previously known as Rank Xerox Research Centre Cambridge Laboratory). Their findings indicate the usefulness of QOC in the construction of design space analysis. MacLean et al. [16] proposed and demonstrated how QOC can be used to structure design space. Bellotti et al. [1] learnt how to generate good design questions. Research into the cognitive dimension of design explanation by [20] suggested using a tree of Criteria for better evaluation of different Questions in a design project. Shum et al. [22] found that QOC most supports designers with critical problems while it can be a distraction with well elaborated ones. A suggested process model of using QOC is described, and its use illustrated, in [15]. A common conclusion is that QOC is shown to be a very useful tool but should not be a full description of design space. Research by Johnson [13] showed that QOC could be used to overcome the limitation of formal specification in communication and representation of design arguments and alternatives. In other research by Khushalani [14] and by Boyle [2] QOC was used to structure observational data of development projects. Briefly, QOC was chosen to be used in our study due to its simplicity, popularity and usefulness proved by intensive research into QOC. Our project differs from previous research into design explanation. It shifts the focus of study from a prescribed design explanation approach for general requirements analysis to a selected but not restricted approach and its possible adaptation to suit and improve a specific requirements engineering method, namely FOOM. Specifically, this research does not study QOC in isolation but in order to supplement IBIS.
2.3. The situation --- A FOOM CASE tool The requirements engineering project for the action research study is to specify requirements for a FOOM CASE tool. This will be a tool to manage the evolution
of a FOOM specification and to record the design explanation relating to this evolution. Therefore, learning from the action research study also directly contributes to the understanding and improvement of the requirements. According to a current version of the specification, the FOOM requirements evolution process is viewed as a network of specification versions. As shown in figure 2, each version is an intermediate state of a FOOM specification at a specific moment (static state). These versions are linked together by transitions (dynamic evolution) according to the development process. One of the requirements is to keep track of these intermediate states of a FOOM specification and the transition from one to another state using an approach similar to Resource Control Systems (RCS) or Source Code Control System (SCCS). Another is to attach decisions and arguments into this model of evolution to explain why each version has a certain form and why a transition occurs. The CASE tool has to support monitoring the FOOM process, provide explanation and therefore improve understandability of the specification to both the analysis team and later the design and implementation team.
Arguments
Figure 2. A FOOM evolution network
2.4. Research approach The research approach chosen is action research. The research process reported in this paper can be described as in figure 3. The process and experience of applying both IBIS and QOC within FOOM is observed and for interpreted/explained. This results in new issues and foci further research. According to Checkland [3], action research can be described in terms of an intellectual framework (F) within which the learning is defined, a methodology under study (M) and a real-world application (A) where
M is applied. From this view, our research can be described as consisting of: • declaration of F, M, A and foci for the following observation. In a subsequent cycle, F, M and A may be evolved through learning from the previous cycle, • application of (maybe new) M and (interpretive) observation on the process of applying M, • interpretation and analysis of observational data collected. Fram e w o rk
M e thod Planning
A c t io n
N e w c y c le
R e f le c t i o n A p p lic a t io n P r o d u c t
Figure 3. Action research cycle in the project In our research, F is the framework of ideas within which systematic documentation and representation of arguments made during the requirements engineering process may be understood to explain the evolution of requirements specifications and improve the understandability and traceability of the requirements specifications. The benefits of supplementing IBIS with QOC within the FOOM process will be investigated. This is M in our case. A is the specification of requirements for a FOOM super-structure CASE tool to manage the evolution of the FOOM requirements, which is described in the previous subsection. Rather than describe the research fully, this paper focuses on only two issues: whether both QOC and IBIS are useful, and how the two notations can be used together effectively. Particularly, it investigates if and how QOC overcomes the weaknesses of IBIS found in earlier research. New problems raised during the research will also be highlighted for future research. In the next section, we will explain the observation and interpretation of what happened.
3. Observation The project was undertaken in the Centre for Information Systems Research, a university research and development environment. A small team of three, a FOOM specifier/researcher, a FOOM developer and an expert in design explanation, were involved. During the process of development, a number of academics and PhD students from the Object-Oriented Requirements Engineering group were also involved at some stages. This investigation took place over 12 months. During this period, all requirements arguments were recorded and structured either in IBIS or in QOC and all were attached to a particular model or a transition under discussion. The process of using the two notations was observed. Figure 4 illustrates a process model of what happened. According to our observation and interpretation, the process was rather cyclic and each cycle consisted of requirements elicitation, modelling and evaluation. In the modelling activity, arguments about requirements were recorded and structured in IBIS as they were made. These IBIS arguments were a good source for structuring logical analysis arguments in QOC. In the evaluation, QOC arguments were used in isolation for the evaluation of different candidate solutions represented in FOOM models (Object-Oriented diagrams and Object-Z). Unsolved or incomplete IBIS and QOC arguments lead to further requirements elicitation. The specifier was the creator of the IBIS and QOC arguments. IBIS and QOC arguments were also used to communicate with other participants in two ways: for assessment of requirements criteria for validation of solutions, and for further elicitation of old or new requirements. read Elicitation read
M o d e lling add read read IBIS arg
Evaluation
IBIS arg IBIS arg
read
...
QOC arg QOC arg ...
Conversion
Figure 4. The process of using IBIS and QOC
4. Reflection 4.1. The usefulness of IBIS and QOC Overall, evidence from this study confirms the findings of our previous research about the usefulness of IBIS. Furthermore, the two notations provide a wellstructured and traceable description of the FOOM process and explanation of FOOM specifications and assist the communication of ideas between participants. Particularly, analysis of observation data shows that design explanation information: • provides the FOOM specifier with both a chronological record of requirements arguments, structured in IBIS, and a logical restructure and evaluation of requirements models, structured in QOC, • supports better understandability of FOOM specification which is written in different forms of semi-formal diagrams and formal documents in Object-Z, • improves the FOOM specification process if integrated with the design environment. To explain these points, we will demonstrate and discuss how both IBIS and QOC were used to solve a specific requirements problem. Firstly, we will show how three IBIS arguments were useful in recording a flow of discussions about a particular requirements problem. Secondly, we will point out some limitations of the IBIS arguments in solving that problem. Next, we will demonstrate how a QOC argument solved these limitations of the IBIS arguments. Fourth, we will show that the connection between IBIS elements and Object-Z specifications (as illustrated in the IBIS arguments) assisted in understanding FOOM models. Lastly, we will affirm that design explanation would improve the FOOM process if integrated with a FOOM workbench. IBIS was very suited for recording the ongoing process of either confirmation of a selected model or making changes to that model. Indeed, IBIS arguments showed that the modelling process tended to be based on a selected or current specification version and involved discussions about some components of the version until a new relatively stable and satisfactory specification was achieved. The following is an example. The version MAW7.24 (see figure 5) was selected for a debate about a definition of the FOOM specification version and its relations to Model5, Documentation and Transition. At 4
MAW is abbreviated for Meta-Argumentation Workbench. MAW is used for a historical link to the inspiration for our project. 5 Italic words denote classes in the requirements model for the FOOM CASE tool (see section 2.3).
that time, each Documentation was attached to a Model and each Transition was attached to two Models, the from and to sides of the transition. Each Project was a graph of Models. Each Model represented a snap shot of the FOOM specification at a specific time. There were a number of issues related to this debate. Appendix C provides details of three such issues which form the basis for the following discussion. They are discussions between a FOOM specifier and two participants. Issue A (see appendix C) lead to the creation of the class Specification and a need to document the evolution of requirements at two levels of the complete FOOM specification and component model.
Project
Model
Documentation
Transition
Argument Decision
Figure 5. Structure model MAW 7.2 is selected for discussion The creation of Specification lead to an important change to the selected structure diagram. New problems included: • how to specify relationships between the new class and other existing classes in the current diagram taken up in Issue A, and • how to equate the previous definition of Model as a version of the new class Specification taken up in Issue A, and a node-link diagram in its own right. Issue B (see appendix C) generated three Positions for consideration, see figure 6(a), 6(b) and 6(c) of which the third was rejected and the others were left undecided. Being constructed from the second perspective (of equation of Model to Specification) issue C defended the Position 6(c) which was rejected in Issue B. Issue C, however, rejected the Position 6(b) which was left
undecided by Issue B. It is obvious that Issue C argued in favour of not changing the relationships. Specification
Model
Documentation Transition
Argument
Decision
a. Position 1 Specification
Model
Documentation Transition
Argument
D e c isio n
b. Position 2 Specification
Model
Documentation Transition
D e c isio n
Argument
c. Position 3 Figure 6. Three possible positions for Issue B There were also a number of other related Issues about definitions of Specification, the relationship
between Specification-Model, description of FOOM CASE tool browser and how to document changes to versions, models and components. All these Issues represented a flow of FOOM modelling activities (see also figure 4) which ranged from the creation of the class Specification to understanding different possible specifications of the relationship between this class to Model, Documentation and Transition. Each issue addressed one aspect of the change to the definition of FOOM version. During the modelling, Positions were discovered and elaborated as new IBIS arguments were recorded. Therefore, a group of some IBIS arguments represented a chronological flow of requirements discussions on some related topic. Each argument either confirmed a selected model or recommended changes to it. The IBIS information was very desirable and valuable to the FOOM participants, especially for the specifier, in exploring and examining different positions carefully, in preventing losing ideas, arguments and alternative changes and in refreshing previous discussions between participants. In fact, during the modelling process, participants often asked the specifier why a certain modelling choice was made. For example, “Why did we select the acyclic structure for the evolution graph?” or “Why is the involuted link from a particular argument's element to itself not allowed?”. In addition, our observation also strongly confirmed the usefulness of IBIS for communication of ideas. In fact, IBIS was often used to sum up a discussion to avoid misunderstanding between participants. Therefore, on one hand, the usefulness for the description of the FOOM modelling process of IBIS is confirmed. However, on the other hand, IBIS also showed its limitations as found in earlier research. They were carefully examined as follows. The limitations lie in the implicit evaluation of Positions and the locality of Issues. Due to these limitations, the assessment of different Positions within and across Issues is very hard. Firstly, the fact that each Position is assessed by its own set of pros and cons leads to the rather implicit evaluation of different Positions. The resolution of an Issue cannot be solved merely because of numbers of pro and con Arguments for each Position. Secondly, Positions may be expressed and/or solved differently in different Issues. However, they may represent the same modelling option. The third Position of the issue B and the first Position of the issue C illustrate such cases. They were assessed differently while representing the same option of figure 6(c). Moreover, an Argument may address a Position differently in different Issues and contexts (versions).
For example, the Argument which supports the definition of Model as a subclass of Node is no longer valid after the creation of the class Specification. This confirms the importance of the newly added element Context. Q: How arguments and design decisions explain the evolution of FOOM requirements.
O 1 : A t t a c h Documentations a n d Transition s t o b o t h Models a n d Specifications a n d t r a n s i t i o n s b e t w e e n t h e m . Arguments record only FOOM components under discussion.
O 2 : A t t a c h Documentation a n d Transition t o o n l y Specifications a n d t r a n s i t i o n s b e t w e e n t h e m . Arguments r e c o r d b o t h F O O M Models a n d components under discussion.
C1: To be able to explain why a particular FOOM specification/version evolves
C2: To be able to explain why a particular FOOM model evolves
C3: To be able to explain why a particular FOOM component evolves
C4: Expressiveness of the specification
O 3 : A t t a c h Documentation a n d Transition t o o n l y Models and transitions between t h e m . Arguments r e c o r d o n l y FOOM components under discussion.
C5: To avoid redundancy
Assessment Argument: IBIS Issue B, P3 (AO)
Figure 7. A QOC argument example QOC was used to overcome these limitations. We created a retrospective analysis and evaluated accurately all alternative models against a common set of criteria in a high degree of abstraction. IBIS arguments were used as a useful and structured source for the construction of QOC argumentation. Figure 7 represents a QOC argument constructed for the review of the above group of Issues. The question in this QOC argument represented a requirements problem of the group of IBIS arguments. Analysis of these IBIS arguments led to three possible options to the question of how to document the evolution of a FOOM specification/model. With QOC, all three options were evaluated against a common set of criteria. Therefore, all these options were evaluated explicitly and accurately. These criteria were derived from all Arguments supporting and objecting to Positions of IBIS Issues. However, rather than representing a concrete argument, each Criterion represented a requirement at a higher level of abstraction. From our experience, the evaluation of an Option against a Criterion can be explained by an Assessment Argument [16] which was often an Argument in an IBIS issue. Putting it in other words, Criteria represent clients' requirements while Assessment Arguments explain why a certain Option supports a Criterion. We find this organisation of Argument and
Criteria manifests a better link between clients' requirements and specifications. Therefore, QOC arguments provide logical analyses around a specification while IBIS arguments provide a historical record of the specification process. The attachment of some Object-Z fragments and/or object-oriented diagrams to IBIS Positions is illustrated in the previous IBIS examples (see appendix C and figure 6). This attachment assisted us in two ways. First, relevant fragments of the specification to a Position were available. Second, supporting and objecting Arguments for this Position assisted in understanding this particular part (which is sometimes complicated) of a FOOM specification. However, not every Position was provided with both Object-Z and graphical explanation. In fact, the “formal” explanation was created only for complicated Positions and Issues. Lastly, this application of both IBIS and QOC confirmed the usefulness of design explanation within FOOM. It encouraged us to look for alternative FOOM specifications for a requirements problem and supported the exploration of issues/questions, and therefore, improved the decision making activity. It is evident that IBIS and QOC have improved the FOOM requirements modelling process. It is desirable to have a CASE tool which would • assist in recording both the graph of requirements evolution and related arguments, structured in IBIS and QOC, • allow the creation and edition of FOOM requirements models and arguments. We believe that because of this, design explanation would assist in better capturing rationale information and management of the FOOM process.
4.2. How to use IBIS and QOC within FOOM effectively? (How best to organise design explanation information?) Our analysis of observation data was also focused on how best to organise design explanation information, particularly how to use both ad hoc and post hoc design explanation techniques and notations effectively. Overall, learning from this research strongly confirms most findings from previous research. Furthermore, analysing design explanation information assists us in understanding the FOOM modelling process and reflecting on the IBIS and QOC notations. This analysis also leads to a comparison of IBIS and QOC arguments. With regard to understanding the process of documenting FOOM requirements evolution using both ad hoc and post hoc techniques, the learning:
• leads to a process model of structuring the graph of requirements evolution and documenting the rationale for the evolution, • confirms the classification of design decisions from the previous research, • suggests recording only arguments and the current version of the requirements model but not recording intermediate versions. Analysing the IBIS and QOC information reveals a deeper understanding of the FOOM modelling process at a micro-level (see figure 8). The FOOM modelling process can be seen as rather iterative. Each iteration consists of discussions on a particular requirements problem generated from a selected version and evaluation of possible changes to the version to solve the problem. When a version is selected for discussion, for example the version MAW7.2 in the previous section, a temporary version is created as a candidate new version. The selected version is fixed while the candidate version is subject to intensive evolution moving towards a relatively stable and satisfactory stage. Changes to the candidate version and the underlying deliberation are documented using IBIS, for example the three IBIS arguments in the previous section. When it comes to making a final decision on the problem, a post hoc review is conducted using QOC, for example the QOC argument presented in the previous section. Therefore, all possible alternative models are gathered and examined thoroughly. If the QOC review confirms the “original” selected version then there is no new version to be released, and the candidate version is removed. Arguments which support the appropriateness of the selected model are attached to the Documentation of the model. However, if the QOC review supports the candidate version, then the temporary candidate version is released as a newly created version. Arguments which support changes to the selected model are accumulated in the relevant Transition. In the example discussed in the previous section, a new version was created. The previous QOC argument was therefore attached to the Transition from MAW7.2 to the new version MAW 8.0. Two issues generated from this process model are whether our classification of arguments is appropriate and whether all temporary states of requirements models should be recorded. This classification of arguments into the two types of Documentation and Transition, proposed during the previous research, is questioned for two reasons. First, one may argue for another classification of arguments that all arguments support either the currently selected model or a new model, if there is one. For example, if an argument supports a transition then it can
be attached to the newly created version. This classification is possible. However, we would prefer the classification of arguments into Transition and Documentation for it expresses a meaningful dynamic process of a requirements evolution, not merely the static states. V1
V2
V3
IBIS ... Vtem
IBIS ...
a. A model is selected. A temporary model is created. IBIS Arguments are documented. V1
V2
V3
IBIS ... V4
IBIS ... QOC
b. A QOC analysis is structured. A new model is released. Arguments are grouped in Transition/Documentation V1
V2
V3
IBIS ... QOC c. A QOC analysis is structured. The selected model is confirmed. Arguments are grouped in Documentation
Figure 8. Process of documenting evolution of requirements specifications and the rationale for the evolution Second, there are many open Issues. They are Issues without any Position at the time the Issues are recorded, or with only Positions for further consideration (?P) and/or rejected Positions (-P) but without any selected Positions (*P). Issues without any Positions or Arguments indicate to specifiers that they may not have completed their task. These issues may be returned to and resolved at a later time. There is a question about whether to keep and how to classify these issues. At present, these issues are attached to the Documentation of the Model under discussion since they do not result in any changes. Issue A is an example of an unsolved issue. These issues require further attention. Therefore, all design explanation arguments are attached to either a model they support or a transition they argue for. Recording all temporal specifications of all FOOM components as a FOOM specification evolves is too
expensive and complex. There is a complicated problem of granularity ranging from a more or less complete FOOM specification, a diagram or a “full” Object-Z specification and to a FOOM node or link (Object/Class) in the diagram or each Object-Z class. Furthermore, evidence shows that the explanation information about a particular (most often a current) FOOM specification is more often desirable than the reconstruction of all its temporary states. In addition, if design explanation is well documented and organised, then temporal specifications can always be reconstructed when needed. With regard to the reflection of QOC and IBIS notations, the evidence: • exhibits a limitation of the QOC notation in weighting Criteria, • confirms the usefulness of the added element Context and generates a current problem with using it, • encourages a flexible use of the IBIS notation. The QOC notation gives the specifier and other participants an expressive and explicit representation of a logical space around a specification with possible options assessed by a number of Criteria. The assessment of Options against a common set of Criteria promotes a comprehensive evaluation of different specification options. However, this leads to a problem of weighting Criteria. A Question is not solved merely because of the number of supporting Criteria to Options. Sometimes a Criterion which is the most important may override all the other Criteria. At present, we simply organise Criteria in the order of their importance/priority, the most important is written first and less important Criteria are written below (see figure 7). This issue was addressed in a more sophisticated way by [20] using Criteria Trees to represent the relationship between Criteria ranging from general criteria to more specific criteria in a design context. This issue needs further attention. The evidence also shows that the IBIS notation and the newly added element Context are very useful to capture on-the-fly FOOM modelling discussions. The IBIS examples in the appendix C demonstrate how Context explains why an Issue is initiated. Context is also highly desirable for the assessment of how an Argument supports or objects to a Position, especially when an Argument may addresses a particular Position in different ways in different Contexts. However, the relationships between the added element Context to other IBIS elements still needs further investigation. We have also experienced a rather flexible use of IBIS. This is consistent with the findings of [18]. Issues may be recorded incompletely without Positions or
Positions may be recorded without Arguments if Positions/Arguments are not available. Some Positions can be rejected or selected quickly without recording an Argument which is too obvious. According to IBIS, only an Issue may generate or specialise a sub-issue, however, sometimes it is better to record that a Position may lead to a sub-issue (Issue). The issue D in the appendix C illustrates this case. It is evident that the subissue is meaningful and represents an actual problem only within the Position P1. This suggests an extended link between Position and Issue for the current IBIS notation. The evidence also brings forward a comparison between IBIS and QOC information created within one project. This is not a new debate on the differences between the two design explanation approaches. However, this is the first time that the two notations are used in a complementary way in one requirements engineering process. Table 1 represents our comparison of IBIS and QOC arguments recorded during this research. Table 1. Differences between IBIS and QOC collected data A p p r o a ch
E lem e n ts O utco m e
IB IS a d h o c : IB IS is u s e d to r e c o r d o n -g o in g in c iden ts a n d a c tivities (F O O M p r o c ess)
QOC p o st h o c : Q O C is u s e d to b uild a lo g i c a l m a p to th e e va lu a tio n o f F O O M s p e c i c a t i o n s , n o t t h e h isto r y o f F O O M p r o cess I s s u e , C o n t e x t , P o sitio n a n d A r g u - Q uestio n , O p tio n , C r iterio n , a n d m en t A ssessm en t A r g u m en t A n o n -the-fly r e c o r d o f t h e m o d ellin g A r e fec t io n a t a h ig h e r level o f a b p r o c e ss s t r a c t i o n b y t h e s p ecier
Strengths G o o d fo r q u ic k n o te-ta k in g d u r in g d isc u s s io n s a n d p la n n in g m eetings (w itho u t e va lua tio n c r iteria ) C a n b e u s e d n o n -int r u s iv ely to F O O M m o d e llin g
H a s a set o f c o m m o n a n d e x p lic it c r iter ia fo r e va lu a tio n o f d ifferent O p tio n s E n c o u r a g es th e g e n e r a t io n o f e v a lua tin g C r iteria
P r o v id e s a c h r o n o lo g ic a l a n d d e t a iled r e c o r d o f a c tiv itie s
P r o v id e s a b r o a d er v iew a n d a lo g i c a l m a p to a r e q u ir e m e n ts s p e c ific a tio n
P r o v id e s a u s e f u l r e s o u r c e fo r c o n s t r u c t in g Q O C
C a n b e u s e d a s a r e v iew o f IB IS in fo r m a tio n
Im p lic it e v a lua tio n
D o c u m en ta tio n Q O C r e q u ir e s a d d itio n a l s e p a r a t e tim e fr o m design a c tivities
W ea knesses
L o c a lity o f issu e s
Both IBIS and QOC information was very useful to the FOOM process including elicitation, modelling and evaluation of requirements (see figure 4). During modelling, IBIS is non-intrusively used by the specifier to record her own reasoning and discussions with other participants. The IBIS records serve as more than just well structured ad hoc design memoirs, they also encourage accurate requirements modelling. Therefore, we believe that using IBIS improves the quality of the FOOM process.
Nevertheless, while IBIS provides an excellent mechanism to capture on-the-fly process of FOOM specification, it is hard to construct a broad logical analysis and evaluate alternative models (or parts of a model) documented in different local IBIS issues. QOC arguments were very useful in restructuring and reexamining IBIS arguments using a post hoc problem solving approach. The next paragraph explains how QOC arguments can be constructed from related IBIS arguments. Each Question represents a common requirements problem among the related issues. Each Option represents a modelling option, taken from Positions in IBIS arguments. Each Option often represents an existing or a candidate version. Some Positions may be expressed in different Issues but they are represented as one Option if they describe the same specification for the requirements problem (Question). In contrast to Positions in IBIS, Options are assessed against a set of common and more explicit evaluation Criteria. The QOC notation encouraged us to generate Criteria. Criteria can be generated from a global view of clients' requirements. Criteria can also be found from IBIS Arguments but are generally more abstract than concrete Arguments. While (IBIS) Arguments often are expressed differently, for example “this information could be deduced from… ”, “this is derivable information”, “there must be an explicit link”, they all can be expressed by one Criterion--expressiveness of the model. IBIS Arguments can serve as the Assessment Argument in an extended QOC [16] (see figure 7 for illustration). The question of automation converting from a number of IBIS arguments to a QOC argument is under current investigation. Finally, is the use of both design explanation approaches redundant as there may be overlap information between the two types of information? At the present, the answer is that there is overlap between the two types of information. However, as demonstrated in the examples discussed above, QOC arguments do not provide a full documentation of the modelling process while IBIS arguments document one issue after another as they occur. In particular, QOC arguments provide logical maps around FOOM specifications and explain how different IBIS arguments hold together.
5. Conclusion In conclusion, it is clear that the supplementation of the ad hoc IBIS arguments with post hoc rationale constructed using QOC offered the FOOM specifier many advantages. The application of both notations demonstrated that both notations were useful and
desirable for FOOM specifiers. The study also resulted in a deeper understanding of the process of documenting the evolution of the requirements specifications and the rationale for the evolution. In addition, the findings lead to a few suggestions about how to use both the QOC and IBIS notations effectively. Future research will consolidate findings from this study and will further examine and, most importantly, systematise the approach to using both design explanation notations within the requirements engineering method FOOM.
References [1] Bellotti, V.M.E., MacLean, A. and Moran, T, Generating good design questions, Technical Report EPC-1991-136, Rank Xerox Research Centre Cambridge Laboratory, Cambridge, 1991. [2] Boyle, S., An investigation of heuristics used in the construction of conceptual models from root definition in the Soft Systems Methodology, Master’s thesis, Swinburne University of Technology, Melbourne, Australia, 1995. [3] Checkland, P., From framework through experience to learning: the essential nature of action research, in H.-E. Nissen, H. Klein and R. Hirschheim (eds), Contemporary Approaches and Emergent Traditions, IFOP, Elsevier Science Publishers B.V., North-Holland, 1991. pp. 397-403. [4] Checkland, P. and Scholes, Soft Systems Methodology in Action, John Willey and Sons Ltd, 1990. [5] Conklin, E. J. and Yakemovic, K.B., A process-oriented approach to design rationale, Human Computer-Interaction 6:357-394, 1991. [6] Dix, A., Finlay, J., Abowd, G. and Beale, R., HumanComputer Interaction, Prentice Hall International (UK) Ltd, Cambridge, 1993, chapter 5, pp.180-190. [7] Duke, R. and Rose, G., Formal Object-Oriented Specification and Design Using Object-Z, Software Verification Research Centre, Department of Computer Science, University of Queensland, 1995. [8] Fischer, G., Lemke, A.C., McCall and Morch, A.I. Making argumentation serve design, Human-Computer Interaction 6: 393-419, 1991. [9] Fowler, D.C. Formal Methods in a Commercial Information Systems Setting: the FOOM Method, PhD Thesis, Centre for Information Systems Research, Swinburne University of technology, Melbourne, Australia, 1996. [10] Fowler, D.C. and Swatman, P.A., FOOM: Structure and Process, Proceedings of the Second Australian Requirements Engineering Workshop, Sydney, Australia, 1997, pp. 3-14. [11] Fowler, D.C., Swatman P.A. and Wafula, E.N., Formal methods in the IS domain: Introducing a notation for presenting Object-Z specifications, Object-Oriented Systems 2(2): 99-127, 1995. [12] Henderson-Sellers, B. and Edwards, J., BOOK TWO of Object-Oriented Knowledge: The Working Object, Prentice Hall International Ltd, 1994. [13] Johnson, C. Literate specification: Using design rationale to support formal methods in the development of humancomputer interfaces, Human-Computer Interaction Journal, 1996, 11(4): 291-320.
[14] Khushalani, A.J., Modelling and supporting opportunistic design problem solving, PhD thesis, School of Computer Science and Software Engineering, Swinburne University of Technology, Melbourne, Australia, 1997. [15] MacLean, A., Bellotti, V.M.E. and Shum, S.B., Developing the design space with design space analysis, in P. J. Barnard and J. May (eds), Design Issues, research and methods for integrated services, North-Holland Series in Telecommunication, Elsevier Publisher, Amsterdam, 1993, chapter 2.4, pp.197-219. [16] MacLean, A., Young, R.M., Bellotti, V.M.E. and Moran, T., Question, Option, and Criteria: Elements of design space analysis, Human-Computer Interaction, 1991, 6: 201-250. [17] Nguyen, L., Swatman, P.A. and Shanks, G., Using design explanation within Formal Object-Oriented Method, Technical report, Centre for Information Systems Research, Swinburne University of Technology, Melbourne, Australia, 1997. [18] Potts, C. and Takahashi, K., An Active Hypertext Model for Systems Requirements, IEEE Software, 1993, pp. 62-68. [19] Ramesh, B. and Dhar, V., Supporting systems development by capturing deliberations during requirements engineering, IEEE Transactions on Software Engineering, 1992, 18(6): 498-510. [20] Shum, S.B., QOC design rationale retrieval: A cognitive task analysis and design implication, Technical Report EPC1993-105, Rank Xerox Research Centre, Cambridge Laboratory, Cambridge, 1993. [21] Shum, S.B., Design rationale as as argumentation: What, Why, How, When and Who?, Design Rationale Workshop, CHI’94, 1994. [22] Shum, S.B., MacLean, A., Bellotti, V.M.E. and Hammond, N.V., Graphical argumentation and design cognition, Human-Computer Interaction, 1997. [23] Swatman, P.A., Formal Object-Oriented Method--FOOM, in H. Kilov and W. Harvey (eds), Object-Oriented Behavioural Specification, Kluwer Academic Publishers, 1996. [24] Wafula, E.N. and Swatman, P.A., FOOM: A diagrammatic illustration of Object-Z specifications, ObjectOriented Systems, 1996, 3(4): 215-242. [25] Yakemovic, K.C.B. and Conklin, E.J., Report on a development project use of an Issue-Based Information Systems, Proceedings of CSCW 90, 1990, pp. 105-118.
Appendix A. An introduction to Formal Object-Oriented Method As mentioned above FOOM is based on three theoretical pillars: • a modified object-oriented notation, MOSES [12] which provides the benefits of object-orientation, such as modularity, abstraction, classification, encapsulation and polymorphism, • the mathematical specification language Object-Z which provides the precision, and therefore may offer deep insights into the problem being specified, also supports the object-oriented structure of specifications,
• a socio-organisational theory based on Checkland and Scholes’s Soft Systems Methodology [4] which promotes open different subjective interpretations by various people involved in the development of a specification and addresses the social context within which the system will be used. The FOOM approach also includes a number of structure and process components, techniques and recommended activities [10]. According to Fowler and Swatman [10], the FOOM process model is described as cyclic. Figure 9 shows the FOOM modelling process. Each cycle is composed from information analysis, modelling and validation. During the information analysis process, the clients' requirements are interpreted, analysed and structured. During the modelling process, in which requirements models are constructed, the informal, semi-formal and formal modelling techniques are used in parallel. During the validation process, the results from the modelling are evaluated and conflicts identified from the formal modelling are solved by user-validators.
Figure 9. FOOM modelling process [10] The FOOM deliverable are a number of specification documents in three tiers: • an executive summary, • Event Chain diagrams with some Object-Z fragments to facilitate the validation of system behaviours, • modified MOSES Object/Class diagrams, InterObject Communication diagrams with complete Object-Z and textual documents. The first document, an executive summary, is prepared for managers in the form of a very high level overview of the functionality of the system in textual form, supplemented by informal diagrams. The second document, Event Chain diagrams linked to fragments of
the formal specification represent the behaviour of the system. This section of the specification is used to explain the behaviour that is defined by the specification to the users responsible for accepting it (validators). The third document, a complete Object-Z specification is provided with explanatory material, including static and dynamic object-oriented notations adapted from MOSES. The last section of the document is aimed at designers and implementors. The detailed description of these diagrams and the semantic mapping between them can be found in [9, 11, 24]. A more comprehensive overview of the method can be found in [10].
B. Design explanation and IBIS This appendix is aimed at giving some background information about design explanation, a description of IBIS and reasons why it was chosen in earlier research into the ad hoc design explanation.
design of other products [6]. The disadvantage is the time spent doing separate documentation. B.2. IBIS (Issue-Based Information System) is a method for representing the design and the structure of planning dialogues, developed first by Rittel in the 1970s. It takes the form of design questions or problems (Issues), possible answers (Positions) and Arguments for and against the answers (see figure 10). The central activity is deliberation and a trade-off analysis between Positions within an Issue. The decision to select or discard a Position is the resolution of the Issue. An Issue may question, generalise or specialise another Issue, and that may lead to other sub-issues with Positions and Arguments. On one hand, IBIS is preferred for its intuitiveness; on the other hand, it is restricted by the local context and lack of explicit goals [19] and outcomes of argumentation. objects to Position
B.1. Design explanation approaches Design explanation is information that represents the reasoning behind the design of an artefact [21]. Several semiformal design explanation notations and techniques have been developed to capture and structure the deliberation discussions and reasoning behind an artefact. They are classified into two main approaches, process-oriented and structure-oriented. In this section we will represent IBIS and QOC as two representatives of the two approaches and justify why they were chosen for our project. • Process-oriented approach These techniques (e.g. IBIS [5] and its descendants) concentrate on the historical record of design decisions and are very much suited for use during the actual design meeting. This approach focuses on the rationale behind the design of a specific product and provides a dimension of narrative about the evolution of the design product. • Structure-oriented approach or Design Space Analysis. In contrast to the above strategies, the design space analysis approach (e.g. QOC [16]) concentrates on the logical space of all design alternatives, which can be restructured by post hoc consideration of the design space. In other words, they emphasise the logical structure while the process-oriented techniques emphasise the history of the design process. The advantage of the second approach is that it can abstract away from the particulars and therefore represents the design knowledge in such a way that it can be of use in the
Argument
responds to Issue responds to Position
supports Argument
generates Sub-Issue
Figure 10. The IBIS notation [6] The itIBIS notation, a textual form of IBIS, is used in our research. Similar to IBIS, an itIBIS argument consists of three types of components, Issue, Position and Arguments. Across related Issues, the indentation represents the hierarchy of these Issues. Within an Issue, the indentation denotes the hierarchy of its components. I denotes issue, ?P, *P, -P denote positions being considered, selected and rejected respectively. AS and AO denote, respectively, supporting and objecting arguments for the according Position. In addition to itIBIS, Positions are enumerated within each Issue for easy communication. To describe the issue fully, we have added a description of the “context”. Therefore, each IBIS fragment describes a detailed situation (Issue: Context) and why (Argument) a decision (Position) is taken to address a problem (Issue: Specific).
• AS To express explicitly, for example, how arguments support each Specification and the transition from one Specification to another explicitly. • AO Information about design decisions and Specifications could be derived from the relationships Documentation-Model, Transition-Model and Specification-Model. • AS To attach design decisions to a Specification and transition between them directly. • AS To record evolution of the FOOM specification of a whole (in Specification). • AO It is derivable information, could be derived by an operation. • AO Faster than if derived by operation during the run-time.
C. IBIS examples---Extract from the research diary Specific Issue A. What is the definition of a FOOM specification version? • Context The Specifier and Participant 2 discussed about version control for different forms of a FOOM specification, text, structure, communication and Event Chains diagrams and Object-Z specification. Previously, in the specification for NGA, each version is modelled as a Model. Each Model is a node-link diagram. However, as the application becomes a FOOM CASE tool, this definition of a FOOM model as a version is questioned. • *P 1 There is the need for a class to co-ordinate all kinds of different diagrams for a FOOM specification version---Specification • AS: Different diagrams evolve differently. Structure diagrams are more stable than others. Object-Z documents evolve most, probably because there are operations involved in the specification of a class. Event Chain diagrams evolve more than structure diagrams because there are many scenarios possible for a structure diagram. • Sub-Issue In this case, do we want to record the evolution and the rationale (for the evolution) of both a complete specification and its component models? • *P Yes, we want to document the evolution and the rationale of changes to requirements at both levels of a complete specification and component models. Specific Issue B Do we need a relationship between Specification and Decision? • Context According to the outcome of Issues A2-01-201196 (Issue A) and A2-I2-071296 about the FOOM specification version, each FOOM project is seen as an acyclic graph of FOOM version---Specification, each Specification is an aggregation (though the type of this relationship is still examined) of different FOOM documents---Model. Since each design decision consists of a number of arguments about a Model or a transition between two Models, the relationship between Argument and Specification has become very difficult to understand. • ?P1 There must be an additional explicit relationship between subclasses of Decision and Specification. See figure 6(a).
D o c u m e n ta tio n D ec isio n m o del : M o del fo r S p ec : S p ec ific a tio n
.
8 a rg : a r gs a r g :c o n c e r n s
m o d e l :c o n te n t
C o m p o n e n t s r e c o r d e d in a n a r g u m e n t a b o u t a m o d e l m u s t b e fr o m t h a t m o d e l. m o d e l :c o n t e n t is a s e t o f a ll N o d e s a n d L in k s ( c o m p o n e n t s ) o f the m o del m o d e l 2 fo r S p ec :m o d e ls m o d e l m u s t b e lo n g to t h e s p e c ic a t io n
fo r S p ec
::::
T r a n sitio n D ec isio n fr o m T o : M o d e l x M o d e l fo r S p ec s : S p e c ific a tio n x S p ec ific a tio n fr o m T o (fir s t ) 6= fr o m T o (s ec o n d ) 8 a rg : a r gs a r g :c o n c e r n s fr o m T o ( fir st):c o n ten t [ fr o m T o ( s ec o n d ):c o n ten t
.
C o m p o n e n t s r e c o r d e d in a n a r g u m e n t a b o u t a t r a n s itio n m u s t b e fr o m t h e s e le c t e d o r t h e n e w ly c r e a t e d m o d e ls in v o lv e d in t h e t r a n sitio n fr o m T o (fir s t ) 2 fo r S p ec s ( fir st):m o d e ls ^ fr o m T o (s ec o n d ) 2 fo r S p ec s (s ec o n d ):m o d e ls T h e s e le c t e d m o d e l a n d t h e n e w ly c r e a t e d m o d e l o f a t r a n s itio n m u s t b e fr o m t h e s e le c t e d a n d n e w ly c r e a t e d v e r s io n s , r e s p e c t iv e l y ::::
• ?P 2 Redefine the class Documentation, Transition and Arguments. See figure 6(b). • AS Design decisions are sets of arguments and are attached to different specification versions and transitions between them. Each argument in a design decision, Documentation or Transition, must argue about models of a specification or transition between them respectively.
• AO Transitions between implicit. A rgum ent M o del
are
D e c isio n a r gs : P A r g u m e n t
c o n cerns :P # N o de m o d e ls : P M o d e l co n cerns
Models
[ f m : m o d e ls
:::
. m :c o n ten tg
m :c o n ten t is a s e t o f a ll N o d e s a n d L in k s ( c o m p o n e n t s ) o f t h e m o d e l m :::: D o c u m e n ta tio n D e c isio n
T r a n sitio n D e c isio n
x S p e c ic ific a tio n
fo r S p ec : S p ec ic ific a tio n
fr o m T o : S p e c ic ific a tio n
8 a r g : a r gs a r g :m o d e ls
fr o m T o ( fir s t) 6= fr o m T o (s ec o n d )
.
fo r S p ec :m o d e ls
fo r S p e c :m o d e ls is a s e t o f a ll F O O M m o d e ls o f t h e s p e c ic a tio n fo r S p e c . A ll a r g u m e n t s in a D o c u m e n t a t io n ( o f a v e r s io n ) m u s t b e a b o u t m o d e ls o f t h e sa m e v e r s io n . ::::
b e c a u s e t h e e v o lutio n g r a p h is a c y c lic . 8 a r g : a r gs a r g :m o d e ls fr o m T o (fir st):m o d e ls [ fr o m T o (s ec o n d ):m o d e ls
.
A ll a r g u m e n t s in a T r a n s itio n m u s t b e a b o u t m o d e ls in v o l v e d in t h e sa m e t r a n s itio n . :::
• -P3 Keep Documentation-Model and Transition-Model and add Object-Z operations to retrieve information of how subclasses of Decision support specifications and their transitions. See figure 6(c). • AO Slow run-time • AO The relationship between design decisions and different versions is implicit in the structure diagrams. Specific Issue C How to equate the previous definition of Model as a version to the newly defined more or less complete FOOM Specification? • Context As mentioned in the context of this issue, each version was previously a requirements model. After the argument on versioning FOOM specifications, each FOOM version---Specification becomes a set of FOOM documents/models, either in the text form, diagrams or in Object-Z---Models. This discussion is about the position of Specification and Model with other classes in the current specification MAW8.0. • *P1 All relationships remain the same. Only Specification is inserted between Project and Model. • AS Decisions and Arguments explain changes to each specific Model directly rather than a set of Models as a consequence. Different forms of FOOM
models are semantically linked and can be checked mechanically. • AS There is a need to keep track of transition between each form of the FOOM specification. • -P2 All previous relationships between Model and other classes concerning the definition of Model (as a graph of node and links and as the superclass of Argument) remain unchanged. However, the relationship between Documentation/Transition-Model is replaced by the relationships Documentation/TransitionSpecification, respectively. • AS There is a need to keep track of transitions between specifications as a whole. • AO Then how to trace back to which FOOM component/model involved in a particular change? This information is lost because there is nowhere to record the model(s) under discussion. Specific Issue D Should we allow create or document arguments? • Context Creating an argument means doing rationale, i.e. creating Issue, looking for Positions, adding Arguments, only then deciding which Position to select. The resolution of the argument will decide whether Documentation and Transition will keep the Argument. On the other hand, documenting an argument means recording a more or less complete Argument. This issue was created first by the specifier when specifying the operation CreateArgument of the classes Documentation and Transition. The issue was then discussed with Participant 2. • ?P1 Allow both to create arguments as well as to document complete arguments. • Sub Issue ID.1 Must Arguments be grouped in a Documentation or Transition? • [*P] Yes. • AS I do not think we should permit any Argument not being grouped in any Documentation or Transition. (Participant 2) … • ?P 2 Allow to document arguments only. …
