© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press
FULL PAPER The semiotics of control room situation awareness Jacques Hugo PBMR (Pty) Ltd 1267 Gordon Hood Avenue, Centurion 0046, South Africa
[email protected]
Abstract The term “situation awareness” has become almost synonymous with a concern for human safety and operational effectiveness in industrial ergonomics and human factors engineering. However, literature on situation awareness makes only very vague reference to the elements of meaning in user interfaces and task representations, and the way that meaning is structured as part of the user’s mental model. Situation awareness research has also included only superficial investigation of the process of visual communication, which can be regarded as a key trigger in human-system interaction. This gap is incongruous, since situation awareness is clearly affected by system complexity induced by, for example, the number of items in the interface, the degree of interaction or automation, system dynamics, and the predictability of changes. This complexity is in turn affected by operational complexity (either of the automation system or the operator’s task). In this brief introduction to the concept of “Control Room Semiotics”, it is suggested that situation awareness research may be enhanced by an understanding of how information perceived by the operator acquires meaning in specific situations, and why optimal information representation and visual communication are key attributes of effective task performance. Keywords: Semiotics, Situation Awareness, Human Error, Human-System Interface, Human Factors Engineering, Visual Communication
1.
Introduction
1.1 What is semiotics? Ferdinand de Saussure and Charles Sanders Peirce are regarded as the co-founders of semiotics, but modern semiotics is more commonly associated with the work of Peirce. The theories of both Saussure and Peirce are useful in explaining the nature and role of signs in the user interface. Their individual theories are not discussed specifically, but elements from both are used to explain the relevance of the concept to the Human-System Interface (HSI) and situation awareness. Although there are a number of more formal definitions, semiotics can be called the study of signs and meaning. The concept is particularly relevant to electronic space, because it is so rich in different forms of representation.
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press Peirce formulated the notion of the semiotic sign that serves as the theoretical basis for any perceptive phenomenon, be it internal or external. The visual nature of any user interface and its diversity and complexity of signs immediately implies that it requires more than just a superficial investigation of the role and meaning of signs. According to Peirce’s approach, a sign can be seen as having three functions. These functions are expressed in the classical Peircean triad (Chandler, 1994, p. 34, Peirce, 1931 - 58) which consists of the representation (the form that the sign takes, called the representamen), its referent (that to which the sign refers, called the object) and its meaning (the sense made of the sign in the operator’s mental model, called the interpretant). These three are interrelated through three functions of the sign: representation, communication, and signification (Nadin and Küpper, 2003): Re p re se ntame n (representation)
io n
es en t pr
at
si g ni fica tion
n ic
re
mu
at i on
m co
Obje ct (referent)
SIGN
Inte rpre ta nt (meaning)
Figure 1: One sign, three functions (Adapted from Nadin and Küpper, 2003) Peirce further explains that the relation of the representamen to its object can be either iconic (some kind of similarity), indexical (some kind of causal relation), or symbol (some kind of convention or rule) (Chandler, 1994, p. 36 – 42). The Saussurean model defines the sign as a dyadic relationship between the signifier and the signified. The signifier is the form that the sign takes, something that can be perceived by the senses, for example, an icon in the interface that represents a pump. The signified is the concept to which it refers, which can be a mental construct (for example, “start the sequence to drain the tank”), or something concrete in the world, such as a physical pump, to which the icon refers. Although this model seems easier to understand, it is limited in the sense that it does not address the dialogical nature of the process through which a sign acquires meaning.
Figure 2: Signifier = visual form, Signified = “pump”
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press
Figure 3: Icons for “process heating” Peirce refers to the process of interaction between the representamen, the object and the interpretant (that is, the perceptive-cognitive process of converting visual representations (“signs”) into meaning), as “semiosis”, which is, in short, the process of “making meaning”. The interaction between the elements of the sign triad creates a new entity, the interpretant, in the mind of a person. As shown in Figure 1, the physical carrier of the sign (for example a representation of temperature gauge on the user interface) is the representamen. This is interpreted recursively by another sign (which thus becomes its interpretant) as referring to an object (e.g. “temperature”). It is recursive, because the meaning of a representation becomes another representation in the mind. This new entity is the “meaning” of the sign, but in semiotic terms, it is in fact a new sign that can be further interpreted, and so the process continues. Whichever model is adopted, it seems that analysing situation awareness without considering this interaction means that one would focus mainly on observable operator behaviour and address only the object and the representamen of signs in the HSI. It further means that designing the HSI without understanding the combined effect of signification, representation and communication in the HSI on operator performance, may lead to cognitive complexity, information overload, high workload, poor usability or operator error. My key argument is that an analysis of the nature, role and composition of signs in the HSI is a necessary element in the analysis of situation awareness. The motivation for this lies in the very nature of the processes of representation, communication and interpretation of operational information in control rooms. In fact, the “semantic content” of artefacts in the control room is so high that it can be called a “semiotic space” characterized by a complex structure of “semiotic morphisms”. This will be explained later, and motivated through a discussion of the relationship of semiotic morphisms to the HSI architecture. 1.2 What is situation awareness? This discussion of situation awareness is superficial and is merely intended as the background against which the semiotic approach is investigated. There is general agreement that situation awareness refers to the state of the operator’s understanding of the process and relevant aspects of a dynamic environment (for example, in a nuclear power plant control room or an aircraft cockpit) with which a person is interacting (Sheridan, 2002). Optimum situation awareness requires knowledge of, for example, current process parameters and the normal value of those
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press parameters, the difference between current values and normal values, the past state of the process, and the predicted future state of the process. Situation awareness is maintained by integration of this information, and is thus critical when the operator is confronted with a complex and changing situation. It is directly related to operator performance, and is especially important during abnormal conditions when the operator is required to make correct diagnoses of faults, and to identify situations and problems not covered by normal operating procedures. There have been several proposals on how to analyse situation awareness, and how to design HSIs to optimize situation awareness so that the probability of human error is reduced. Research has shown that situation awareness is affected by many cognitive factors, by motivation and by workload (Endsley, et al. 2003). In particular, it has been suggested that the way in which information about the dynamic environment is represented in the operator’s mental model, plays a significant role in anticipation of certain events, and thus also affects a conscious attention and search for information. There is also common agreement that the work situation in complex industrial control environments is characterized by high information content, which, if not managed properly, may contribute to excessive workload and hence operator error. The Situation Awareness Global Assessment Technique has generally been accepted as a valid measure of situation awareness (Endsley 1993). According to this approach, operators need to know more about the process than is optimal or even useful (Seobok and Kaarstad, 1997). More situation awareness is not necessarily better for optimal performance, because too much information can cause “cognitive clutter” and may interfere with effective mitigation. This method is also inadequate to determine the effectiveness of visual communication of signals in the control room. We know that the schemata that make up the operator’s mental model are constructed through perception, attention, pattern matching, analysis, synthesis and metacognitive processes. These are all directly associated with the process of semiosis, but situation awareness analysis techniques have so far not included this perspective. To address this gap in situation awareness, we need to understand how information perceived by the operator acquires meaning in specific situations. More specifically, we need to ask how signs in the HSI are distinguished from the totality, that is, how does the operator decide where to focus his attention, whether regarding the external world (the plant) or regarding his own interior world (mental model)? We also need to know what contributes to the perceptual salience of the information. How does the sign modify the operator’s internal mental organization, that is, the mode of interpretation of the sign? (Nöth, W., ed. 1994.) These questions fall squarely in the domain of semiotics. Situation awareness measurements need to include an analysis of the actual information that the operator deals with: location, type, duration (transience), frequency (repetition), structure, format, accuracy, origin, etc. Development work to date on the HSI for South Africa’s Pebble Bed Modular Reactor has indicated that a study of situation awareness (and thus also the HSI design effort itself) may be augmented, in terms of overall value and validity, by a semiotic perspective. Understanding how information perceived by the operator acquires meaning in specific situations inevitably requires a closer look at the structure of signs in the HSI, and the formulation of a pragmatic approach to such a semiotic perspective. In “designing for situation awareness” (Endsley, et al., 2003), the actual signs in the HSI
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press therefore need to be analysed in terms of location, type, duration (i.e. transience), frequency (repetition), structure, format, accuracy and origin. Again, this cannot be done without a semiotic perspective.
Control room semiotics The notion of “Control Room Semiotics” is defined in this paper as “a study of signs and meaning in industrial human-system interfaces”. The obvious underlying requirement is the integration of principles from both semiotics and situation awareness into a subset that focuses specifically on task performance through optimal information representation and communication. Communication (and therefore interaction in the HSI) is mediated by signs (e.g. icons, symbols, text, and alarm sounds), which in turn are representations of other signs. In terms of this concept, the control room (which is part of the HSI) can be regarded as a semiotic space characterized by various forms of representation (iconic, indexical, or symbolic, ranging from abstract to concrete) and structured in four different sign forms: lexical, semantic, syntactic and pragmatic. A semiotic approach would consider the reasons why some representations in the HSI are better than others. It is possible to map these translations between sign systems (for example, from plant to mimic diagram, system to icon, event to alarm sound). These maps are called semiotic morphisms (Goguen, 1996), and they can be qualitatively analysed for consistency, coherence and complexity. In a later paragraph, the relationship between semiotic morphisms and some of Saussure and Peirce’s concepts described above, will be demonstrated. 1.3 The control room as semiotic space There are several rather abstract definitions of the term “semiotic space” (Goguen, 2000, and Lotman, 1990), but the concept is simplified here as “a virtual space for sign processes”. Jurij Lotman advocates “a holistic approach which considers the totality of all sign users, texts, and codes of a culture as one semiotic space: a ‘semiosphere’, which makes sign processes possible”. According to this approach, each semiosphere is unique in terms of its homogeneity, its opposition to the exterior, and irregularity in its internal structure. The border between the interior and the exterior of a semiosphere is maintained by the “mutual strangeness of sign users, texts, and codes and is partially overcome through processes of translation” (Lotman, 1990). This sounds like a reasonable description of most control rooms! The internal structure of a control room has a centre (the “interactive sphere”, consisting of the operator and the user interface) that is surrounded by areas (for example the various operating areas) which become increasingly amorphous in the direction of the periphery. This internal structure is responsible for the inner dynamics of the semiosphere. The centre contains the dominating sign systems which include sign users, symbols, icons, indexes and codes that are elaborately tuned to each other. In the periphery, there are also sign users (supervisors, engineers, managers, clients, other systems, etc.). The exchange that takes place between interior and exterior (that is, between the control room and the plant and the rest of the organization) and between the interactive sphere and periphery (the areas immediately outside the control room, such as supervisor’s office and other supporting facilities) leads to the emergence of new codes, the production of new types of signs,
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press and changes in the sign users, which make them receptive to new meaning, which is essential to ensure overall operability of the HSI.
Figure 4: Control room example (photo courtesy ABB) 1.4 Semiotic morphisms The control room can be regarded as a complex system of non-verbal, visual signs. This system allows the operator to have a basic orientation in physical as well as information space. This orientation entails a feeling of a position, which is then transformed into a sign system of semiotic morphisms and non-verbalized meaning. The system of semiotic morphisms is a multi-layered, recursive structure that possesses all the semiotic features of sign systems, and can thus be used to describe the structure and context of the control room semiotic space. One of the basic semiotic morphisms is spatial orientation of systems on the control room displays and controls. An example of the distortion of the spatial orientation semiotic morphism can be seen in the case of controls and displays that are not mapped to the physical layout of systems in the plant. Operator performance during plant operations is influenced by the layout presented in the HSI, rather than the actual layout of the plant. In order to control the plant properly in relation to the spatial layout, the operator has to correct for this erroneous semiotic morphism that is a mismatch between abstract and structural mental models. The semiotic morphism reflects the interpretation of spatial positions at a non-verbalized visual level. Operators have to overcome conflicts that result from the erroneous semiotic morphism through conscious effort and intellectual evaluation, often combined with verbalization (“the left button is for the right-hand pump train…”).
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press The user interfaces in a control room are composed of many interrelated (and often interdependent) components arranged spatially and temporally in many layers. Such layers could be distinguished in terms of system or process complexity, information detail, interaction scope, automation level, and operator response time requirements. Process representations in most industries have become fairly abstract, partly because computer displays now make it possible to represent processes in ways impossible on analogue instruments, and partly because increased automation means that the operator no longer interacts with discrete components, but rather with the process. This naturally requires different types of visualization of the process.
Figure 5: Operator interface example (image courtesy ABB) It is obvious that the design of displays and instruments for digital control rooms requires a different approach than with conventional hard controls and discrete analogue instruments. The assessment of situation awareness in digital control rooms thus also needs to take the effect of varying representational and interaction modalities into account. In such analyses, one needs to pay particular attention to the semantic and syntactic properties and constraints of the HSI that determine the pragmatics of different instruments. For detailed analyses, the individual instrument itself can be decomposed into various semiotic elements (or “graphemes”) such as scale, pointer, label, box, frame, line, etc. (For a discussion of instrument semiotics, see May, 2000.)
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press 1.5 The paradigmatic-syntagmatic framework as an HSI semiotic morphism The control room is a combination of physical and virtual (electronic) constructs. It is governed and defined by its own rules and laws of existence, and it can thus be seen as an expression of a subset of known semiotic systems. In semiotic terms, the HSI (which includes the control room) is simultaneously an index and an icon. When we delve deeper into its structure, it may be defined as a high order, complex sign consisting of a web of many other complex signs. In this semiotic space, all signs are coordinated to work together to express a particular model of the world. Quite literally, nothing is without meaning or without an intended message. In this way, the semiotic space of the control room is constructed from the synthesis of all syntactic and paradigmatic elements. This construct can be regarded as a semiotic morphism called the paradigmatic-syntagmatic framework (PSF). A semiotic approach supports the cognitive-semantic aspects of sign composition and is thus well suited to the analysis and design of HSIs in complex task domains. A coherent taxonomy or framework of structured representations is offered by the paradigmaticsyntagmatic approach, and it is a practical way to ensure consistency and coherence in the HSI architecture. Such a coherent framework will also help to structure the semantic architecture of the HSI. It should thus be possible to ascertain with a greater degree of accuracy and confidence why, how and when certain display and instrument configurations promote and others inhibit situation awareness. An analysis of the semantic architecture of the HSI within the PSF would also help to assess the “semiotic adequacy” of the HSI, in other words, why a particular instantiation of a display is better to promote situation awareness than another. As Saussure indicated, meaning arises from the differences between signifiers (Saussure, cited in Chandler, 1994, p. 29). These differences are either paradigmatic (i.e. dealing with substitution) or syntagmatic (i.e. dealing with positioning). Syntagms in the HSI are identified in five groups or relationships that describe the HSI from various perspectives and levels of detail: Physical Interaction System (PS), Subject Matter (SM), Spatial Organization (SO), Figurative Screen Image (FI), and Discrete Signs (DS) (see Table 1). The paradigm is the selection of alternative elements to create functional contrasts (e.g. analogue instrument or digital instrument or trend graph), for example a paradigm of Spatial Organization would be expressed as SO(A|B|C), which reads: “system information presentation will employ two-dimensional spatial layouts or threedimensional component images or dynamic trends graphs”. A syntagm is an orderly combination of interacting signifiers which form a meaningful whole in an interface, which means it is a combination of selected elements from all possible paradigms. Together, syntagms and paradigms thus create a framework that provides a coherent, structural context within which signs make sense. For example, a syntagm for an individual interface would be expressed as X(PS+SM+SO+FI+DS) as in the following syntagmatic sentence:
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press [SO] “The LCD monitor (displays)” + [SM] “controls (for the) main cooling cycle” + [SO] “(as a) flowchart” + [FI] “at 1280 x 1024 resolution” + [DS] “(using) abstract and concrete representations (of) plant equipment”.
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press Table 1: Example of a PSF MACRO LEVEL
Vertical Paradigms
Horizontal syntagms
MESO LEVEL
MICRO LEVEL
Physical Interaction System (PS)
Subject Matter (SM)
Spatial Organisation (SO)
Figurative screen image (FI)
Discrete Signs (DS)
A. Computer Hardware: LCD monitor, wide panel displays, keyboards, mouse, printers, buttons (shutdown controls, etc.)
A. Category: Control of the Nuclear Power Station
A. two-dimensional field: structural representation of plant and equipment
A. Size, brightness, resolution: operator displays are 19 inch LCD panels at 1280x1024 pixels. Wide display panels are 150 cm plasma displays at 1024x768 pixels. Brightness and contrast are adjustable by operator.
A. Shapes and forms: • Mode/State and performance diagrams • Process diagrams and flowcharts • Alarm and event lists • Display shortcuts (icons) • Faceplates (control panels) • Task Support and Help forms • Text and notes • Graphs, • Toolbars, scroll bars, status bars, buttons, etc.
B. Software: Operating system, User Interface Management System, Operational Control System
B. Level of complexity: High
B. three-dimensional field: physical appearance and spatial relationships of equipment
B. Colour: Colour is used generally to convey specific information and to categorise graphic elements. Within the limitation of maximum 4 colours per display (excluding shades of simulated 3D objects), standardised colour is also used to indicate interface backgrounds, functional areas and objects.
B. Icons: Function shortcuts, display shortcuts
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press C. Environment: control room structure, consoles, desks, communication equipment, storage, seating, lighting, etc.
C. Combination of graphic, symbolic, textual
C. 4-Dimensional Field (Temporal dimension): change of graph lines over time, valve position change, simulated flow in pipes, animation of machine parts, changing sounds, etc.
D. Concrete: physical layout and characteristics of equipment (reactor vessel, turbines, pipes, pumps, etc.) Abstract: system modes and states, process flow, system performance
C. Level of realism/abstraction: Top levels of the HSI employ abstract metaphors (e.g. mode/state diagrams) and lower levels employ more concrete metaphors with realistic or stylised objects to represent plant and equipment.
C. Symbols: electrical, mechanical, hydraulic and pneumatic equipment symbols - motors, pumps, valves, circuit breakers, vessels, cooling towers, reactors, tanks, bins, pipes, transformers, resistors, gearboxes, filters, heat exchangers, compressors, blowers, turbines, etc.
D. Composition, layout, image complexity: Composition and layout follow the operator's natural task flow. Images are kept as simple as possible to avoid ambiguity and visual noise.
D. Indexes: alarm lights and sounds, trend graphs, gauge pointers, values
E. Auditory signifiers: Alarm sounds, machine sounds, environmental sounds, voice. Signifier level
Spatiality: objects and signs are not arranged on the display to correspond to the spatial layout of the plant, but to reflect the logical flow of the process. Contrast: active objects are highlighted to achieve the highest possible contrast with the display background. Inactive objects are displayed with lower contrast so that operator can focus on active objects. Shape: Objects are either represented as stylised, recognisable forms (vessels, turbines, motors, etc.) or as abstract symbols (circuit breakers, valves, etc.) Colour: Equipment and indications are colour coded and subtle shades are used for 3D objects. Colours used on any display generally limited to 4. Backgrounds are consistently dark grey. Texture: This is generally avoided in the HSI since it introduces too much visual noise. Metaphor: Plant physical and abstract flows and other attributes are used as metaphor. This is implemented as state or flow diagrams. Abstract metaphors based on the thermohydraulic characteristics of the process are also used and represented as abstract, dynamic diagrams (e.g. efficiency diagrams plotting entropy against temperature).
Signification
Connotative and Denotative content:
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press level
Arbitrary Signs: - abstract symbols for processes and objects - graphs to indicate temperature, pressure, flow, etc. - text, diagrams, graphs to represent plant or system performance and status - digital instruments (numeric values and labels) Figurative representations (object schemata): - realistic or semi-realistic images - Reactor vessel, turbines, generator, pumps, valves, etc. to represent corresponding equipment - analogue instruments - dial faces, pointers, frames, etc. to represent process parameter (pressure, temperature, reactivity, etc.) Logical, cognitive or denotative content:Images on screen are recognised as flow diagrams, vessels, turbines, pipes, graphs, etc. Connotative or associative meaning: For example, a specific value on a specific temperature gauge in a specific flow diagram on a specific process display is associated with bearing temperature on the power turbine generator, the outline and form of a symbol indicates a specific type of valve, etc. Stylistic meaning (layout, context, etc.): For example, the spatial relationship between representations of valves on a process display indicates direction of flow, lines between symbols indicate dependency or isolation, relative size of symbols indicate importance of physical size, etc.
Syntagms:
Physical Interaction System is expressed by the paradigms of computer hardware, software and physical environment within which the operator performs his tasks. Subject Matter is expressed by the paradigm of nuclear power station control with a high complexity, employing a combination of textual, graphical and symbolic information representation, ranging from concrete to abstract. Spatial organisation is expressed by the paradigm of the two-dimensional layout of the user interface screens, the representation of threedimensional objects and the indication of dynamic changes of time during the execution of a process. The figurative screen image is constructed from the paradigms of size, brightness, resolution, colour, level of realism or abstraction, and composition and layout of objects. Discrete signs in the interface are constructed from the paradigms of shape and form, icons, symbols and indexes.
Individual syntagm:
Meaning is thus represented at three levels: The "frame message" by the vertical paradigms that form the 5 syntagms. Denotative meaning at the figurative level is represented by the signifiers of spatiality, contrast, shape, colour, texture and metaphor. Connotative meaning at the interpretive level is represented by the signification functions of context and association
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press We can now see that a paradigmatic connotation results when the meaning of a specific interface is derived from the knowledge that it is a choice from among other representations, e.g. the same system or process. A syntagmatic connotation results when meaning is not derived from a single element on the interface from among others, but based upon a comparison with other signs on the same display, or with preceding and succeeding displays. The difference between actual (current) and “potential” displays determines the connotation. Since this has a direct bearing on the operator’s ability to predict the future state of the system, it also has a direct influence on situation awareness. At the signification level of the PSF, connotative meaning is influenced by a large number of factors, including what is often referred to as “conventions”. It is vital that the signification level of the HSI, and especially the origin of connotative meaning, be included in the investigation of situation awareness. The obvious reason is because connotative meaning based on false conventions may result in inaccurate mental models.
2. Conclusion It is true that it is very difficult to “measure” mental states such as workload, stress or semiosis. However, all existing methods of determining the nature of operator behaviour and the parameters of task performance attempt to explain the reasons for deficient performance and human error. None of these methods explains the process whereby the operator observes and interprets the signs in the user interface and decides to perform a specific action. They fail to acknowledge that what really happens between an operator and an HSI is not simply “interaction”. It should rather be seen as a process of communication where the accuracy of information, and thus the correctness of the operator’s performance, is directly influenced by decisions made by designers of the HSI (de Souza, 2004). By understanding the semiotic nature of the HSI, designers should be able to make less arbitrary decisions about the overall architecture (that is, from macro to micro level, as indicated in the paradigmatic-syntagmatic framework). This would include decisions about choosing among different instrument types, and the allocation of specific tasks to specific instruments and displays. It is important to understand that operator performance cannot be divorced from the environment. Performance is triggered by perceptual or cognitive stimuli in the environment, and tasks are executed within the environment by interacting with various artefacts, which in turn produces other stimuli, giving rise to an endless process. Intimately interwoven with this is the process of semiosis, which has an inescapable effect on performance. In other words, by understanding the relationship between a sign, its meaning and the context, we will have a clearer understanding of the reasons why different representational modalities are better than others to convey operational information in specific contexts, thereby improving the chances of achieving the required performance. This introduction has barely scratched the surface of this potential synergy between the three scientific fields of semiotics, situation awareness, and human-system interaction, and many questions still need further investigation. For example, what are the design requirements for semiotic adequacy? What contributes to the perceptual salience of the
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press information? How does the sign modify the operator's internal mental organization? At a detail level of signification, much more research is necessary to extend Andersen & May’s work on instrument semiotics to the control room as a whole. Human factors engineering has always been regarded as a multidisciplinary field, but there is little doubt that the field can benefit from the synergy offered by findings in semiotics research. The concept of Control Room Semiotics seems to have obvious merits, and its practical implementation in the PBMR HSI seems promising, but it clearly needs further research before it can be incorporated into formal theories and techniques of task analysis, situation awareness and human-computer interaction. Some of these research results will come from verification and validation still to be done at PBMR, but it is hoped that this brief introduction will stimulate a new look at what are assumed to be tried and tested methods of human factors analysis and design.
References Andersen, P.B. & May, M. (2000). Tearing up interfaces. Retrieved January 14, 2005 from http://imv.au.dk/~pba/Preprints/Tearingup.pdf Bedny, G. & Karwowski, W. (2004) "Meaning and sense in activity theory and their role in the study of human performance". International Journal of Ergonomics and Human Factors. (26:2, 121-140.) Chandler, D.L (1994). Semiotics for Beginners. London: Routledge. Bedny, G. & Karwowski, W. (2004). Meaning and Sense in activity theory and their role in the study of human performance. In Ergonomia IJE&HF, 2004, Vol. 26, No.2, 121-140. Berlin: Mouton de Gruyter. Retrieved January 14, 2005 from http://www.percepp.demon.co.uk/semiosis.htm Dekker, S. & Hollnagel, E. (2004) Human factors and folk models. Cognition, Technology and Work, 6:2, 79-86. De Souza, C.S. (2004). The semiotic engineering of human-computer interaction. New York: The MIT Press. Eco, U. (1976) A theory of semiotics. Bloomington, IN: Indiana University/London: Macmillan. Endsley, M.R., Bolté, B. & Jones, D.G. (2003). Designing for Situation Awareness – An approach to User-centred design. New York: Taylor and Francis. Goguen, J.A. (2000). Semiotic Morphisms. Retrieved January 14, 2005 from http://www-cse.ucsd.edu/users/goguen/papers/sm/node5.html Goguen, J.A. & Harrell, D.F. (2003) Information Visualization and Semiotic Morphisms. Retrieved March 15, 2005 from www-cse.ucsd.edu/users/goguen/papers/sm/vzln.html Troy Innocent (2003) Exploring the nature of electronic space through semiotic morphism. Retrieved March 15, 2005 from http://www.fineartforum.org/Backissues/Vol_17/faf_v17_n08/reviews/innocent.html Lotman, J.M. 1990. The Semiosphere. Zeitschrift für Semiotik. The Research Centre for Semiotics, 1990, 12:4. Retrieved March 15, 2005 from http://ling.kgw.tu-berlin.de/semiotik/english/ZFS/Zfs90_4_e.htm#1
© Thatcher, A., James, J & Todd, A. (2005). Proceedings of CybErg 2005. The Fourth International Cyberspace Conference on Ergonomics. Johannesburg: International Ergonomics Association Press May, M. & Andersen, P.B. (2000) Instrument Semiotics Retrieved January 14, 2005 from www.cs.auc.dk/~pba/Preprints/InstrumentSemiotics.pdf Nadin, M. (ed). (1985). The Meaning of the Visual: On Defining the Field. In Semiotica 52:3/4. Amsterdam: Mouton. Nadin, M. & Küpper, A. (2003). Applied Semiotics: HCI as the result of semiotic processes. On-line lectures on semiotics. Retrieved March 30, 2004 from http://www.code.uni-wuppertal.de/uk/hci/Applied_Semiotics/applied.htm Nöth, W. (Ed.) (1994). Language and the origin of semiosis (extract from Sign Evolution in Nature and Culture, Part III Glottogenesis: Phylogeny, ontogeny, and actogeny. 255-268. Parasuraman, R. & Mouloua, M. (Ed.) (1996) Automation and Human Performance. Theories and Applications. Mahwah, NJ: Lawrence Erlbaum. Peirce, C.S. (1931-1958): Collected Writings. (Ed. Charles Hartshorne, Paul Weiss & Arthur W Burks). Cambridge, MA: Harvard University Press. Sebok, A. & Kaarstad, M. (1997). Crew Situation Awareness, Diagnoses and Performance in Simulated Nuclear Power Plant Process Disturbances. CNRA/CSNI Specialist’s Meeting on Human Performance in Operational Transients. Chattanooga, TN, USA. 13 – 17 October, 1997. Sheridan, T.B. (2002). Humans and Automation: System Design and Research Issues. Santa Monica, CA: Wiley.
