A COMPARISON OF THREE VISUAL REPRESENTATIONS OF COMPLEX MULTIDIMENSIONAL ACCOUNTING INFORMATION
Richard B. Dull Assistant Professor of Accounting Indiana University Kelley School of Business 801 West Michigan Street, BS 4026 Indianapolis, IN 46202-5151 (317) 274-0867
[email protected]
David P. Tegarden Assistant Professor of Accounting Information Systems Virginia Polytechnic Institute and State University Pamplin College of Business Department of Accounting and Information Systems (0101) 3007 Pamplin Hall Blacksburg, VA 24061 (540) 231-6099
[email protected]
Acknowledgements: We appreciate everyone that contributed to the completion of this project and paper, including Traci Hess, Jason Lockhart, the three anonymous reviewers and the journal editors.
A COMPARISON OF THREE VISUAL REPRESENTATIONS OF COMPLEX MULTIDIMENSIONAL ACCOUNTING INFORMATION ABSTRACT This study investigates the relationship between three visual representations (twodimensional, three-dimensional fixed, and three-dimensional rotatable) of multidimensional data, and the subjects’ability to make predictions based on the data. Output of a momentum accounting system was simulated and graphics were rendered based on that information. An interactive computer program was developed and used to administer the laboratory experiment and collect results. Subjects made prediction decisions based on the graphics produced for four companies. Each subject made predictions for one type of graphics representation for each of the four companies. Subjects using three-dimensional data that could be rotated provided the most accurate predictions. This finding is significant in a systems environment where visualizations and graphics are steadily increasing. The results should be considered when developing systems to provide accounting system users with information for making decisions, especially when the information to be presented is multidimensional in nature. Key Words: Accounting information systems, information visualization, cognitive fit, proximity compatibility principle, momentum accounting. Data Availability: Please contact the authors.
Information, once rare and cherished like caviar, is now plentiful and taken for granted like potatoes. The First Law of Data Smog, David Shenk, 1997 I. INTRODUCTION Today's powerful accounting information systems provide decision-makers with an abundance of accounting information that previously has not been cost-effective to produce. However, despite significant technological advances in the accumulation of accounting data, the formats used for presentation of the information produced by accounting information systems have not kept pace with the information technology available. As such, advanced accounting information systems, with their increasing volume of information and increasing complexity, can simply overwhelm the decision-maker. This increase in the quantity of information is compounded by the generally accepted trend towards enterprise-wide information systems that integrate all aspects of the business. There is an increasing trend towards supporting decision-making with multidimensional data, e.g., on-line analytical processing (OLAP), knowledge discovery, and data mining (Adriaans and Zantinge, 1996; Fayyad, et al., 1996; Thomsen, 1997). Additionally, there is a trend of using multidimensional visual representations to support the decision-maker using multidimensional data (Adriaans and Zantinge, 1996; Card et al., 1999; Parsaye and Chignell, 1993; Thomsen, 1997). One area of accounting research that provides complex multidimensional accounting information is momentum accounting.1 One of the main criticisms of momentum accounting is its higher level of complexity, due chiefly to its multidimensionality, when compared to traditional double-entry accounting (Fraser 1993). Recent advances in visualization technologies provide one possible solution to this information "glut." Specifically, data/information visualization may allow both novice and expert
decision-makers to use their visual/spatial abilities in the process of making complex decisions using accounting information. Data/information visualization is the use of visualization technology to transform non-spatial, business, or behavioral data into multidimensional visual images that represent an analogy or metaphor of the problem space (Schroeder, et al. 1998). Visualization technologies have been successfully deployed in finance, litigation, marketing, manufacturing, training, and organizational modeling (Brown, et al. 1995; Choras and Steinmann 1995; Grantham 1993; Markham 1998; Schroeder, et al. 1998; Thierauf 1995).2 Although these technologies have been deployed, there are few scientific studies within business and accounting to analyze the usefulness of this technology. The existing studies are generally limited to the traditional “graph versus table” literature and do not address multidimensional data/information visualization. The research reported on in this paper addresses the following general question: Can accounting-based decision making performance be improved by using data/information visualization technologies? Specifically, we investigate whether multidimensional visualization technology can increase the usefulness of momentum accounting information when compared to using more traditional twodimensional graphics. The remainder of the paper is organized as follows. The next section provides the literature background. Section III provides the theoretical basis and states the hypothesis. The research method is described in the section IV. Section V presents the data analysis and results, and the final section contains the conclusions and limitations of the study.
2
II. PRIOR LITERATURE Momentum Accounting Ijiri (1982, 1986, 1990) proposed an extension of double-entry accounting that considers a third dimension of accounting data -- the time effect of accounting entries. This extension, referred to as momentum accounting, would include information relating to the wealth of an organization and the corresponding rates of change in the wealth of an organization3. Momentum accounting increases the complexity of an accounting system by expanding on the singledimension stock (assets and liabilities) and two-dimensional flow (revenues and expenses) concepts, by adding a third dimension (time) and, therefore, a corresponding third component of each entry. This study centers on making decisions using momentum accounting information rather than on the specifics of a momentum accounting system. As such, we focus on the output of a momentum accounting system rather than the details of journal entries and the recording of transactions. Momentum accounting systems should provide a decision-maker with not only current period information such as income, but also with the rate of change of income – information useful when making decisions regarding future periods. If accounting information associated with momentum accounting is presented suitably, the efficiency and effectiveness of decision-making using momentum accounting information should be improved. The most suitable presentation of momentum accounting information involves the use of multidimensional visualization technology to match the dimensionality of the momentum accounting information, thereby enhancing the understanding of that information (Cooper, 1990; Vessey, 1991). Blommaert and Olders (1995) used financial statements4 based on momentum accounting concepts to investigate managers’use of the additional information provided in their decision
3
making process when predicting earnings for a future year. Their findings indicate that subjects using momentum accounting based financial statements were capable of making more accurate predictions of earnings than subjects using traditional financial statements. It may be possible to reduce the complexity of using momentum accounting information via information visualization technologies, hence, addressing one of the major criticisms of momentum accounting. In the current study, we use momentum accounting information to test the viability of using visualization technologies in accounting. Visualization Research From a cognitive science perspective, visualizations can enlarge problem-solving capabilities. Miller (1956) describes a set of results that imply that a human’s input channel capacity can be increased by using visual abilities. The results suggest that different parameters in the visual channel can be exploited to increase the amount of data that decision-makers process without suffering information overload. Furthermore, visual imagery research suggests that visual recall is better than verbal recall, i.e., a picture really is worth a thousand words (Kosslyn 1980; Shepard and Cooper 1982). However, imagery-based recall is highly associated with how the objects were learned (visual images on visual perception, auditory images on auditory perceptions, etc.). Schkade and Kleinmuntz (1994) found that information format had a significant effect on the decision process when used during information acquisition. Larkin and Simon (1987) found that diagrams were superior representations to written ones and suggested three reasons for these results. First, diagrams group related information together. Second, diagrams use location to aid in information search. Finally, diagrams aid in many perceptual inferences. They concluded that if
4
the user is capable of using diagrams to acquire information, diagrams seem to support more efficient computational processes than their written counterparts. Jones and Schkade (1995) found that if the information representation did not match problem representation, decision-makers sometimes “translate” an information format into one with which they are more familiar and base their decision on the new format. Cooper (1990) suggests that when confronted with two-dimensional representations of structural objects, subjects mentally create a three-dimensional representation of the object. Providing a twodimensional representation of a three-dimensional object may slow the decision-making performance of a subject. Wickens, Merwin and Lin (1994) indicate that for questions requiring an integrative knowledge of information, extraction and retention of data is superior for 3D images over 2D images. In addition, although statistical differences in responses were not reported, they concluded that rotation was used more by subjects when answering questions that were relatively more integrative. Furthermore, Pani (1993) reports that the angle of rotation affects the comprehension of an object that is presented within a 3D representation. The above studies suggest that multidimensional visualizations can be useful in decisionmaking. They also suggest that rotation of the multidimensional visualizations facilitates comprehension. Therefore, if the visualization fits the problem and the decision making process, the speed and accuracy of problem solving should benefit. Accounting-Based Visualization Research The majority of research in accounting visualization relates to report format and presentation. The formats for most research projects differ significantly, although one theme continues throughout the literature; the presentation format of information affects the decisions
5
that are made (MacKay and Villarreal 1987; Moriarity 1979; Stock and Watson 1984). Additionally, visual representations have been investigated from several accounting perspectives including cash flow predictions, financial forecasting, and time series analysis (Bouwman, et al. 1995; Carbone and Gorr 1985; DeSanctis and Jarvenpaa 1989; Goldwater and Fogarty 1995; Moriarity 1979). However, most of the past research addresses only two-dimensional graphs. Generally, most of the multidimensional visualization research was performed using Chernoff Faces.5 Although most results indicate that using Chernoff Faces improves decisions (Moriarity 1979; Stock and Watson 1984), MacKay and Villarreal (1987) indicate that using faces may reduce the content validity of a visualization and, as such, limit its usefulness as a decision aid. Additionally, Amer (1991) reports that using Chernoff Faces and radar charts (Harris, 1996) in an integrative decision-making task provided no differences when compared with bar charts and tables. As such, much of the multidimensional visualization research in accounting provides only mixed results. With today’s technological advances, most desktop computers can represent multidimensional data by generating complex multidimensional graphics rather than limited formats such as the faces and radar charts. The current study moves beyond the use of twodimensional graphics, Chernoff Faces, and radar charts to using a multidimensional information visualization to fit an integrative accounting problem-solving task. Novice Decision-Making and Visualization Technology Using multidimensional visualizations to aid novices in decision-making is not a new concept. Visual technologies are frequently used to help medical students understand how the human body functions, by using three-dimensional graphics to represent areas that are difficult to understand (Hallgren and Gorbis, 1999). Multidimensional visualizations have also been used to
6
help novices understand abstract accounting concepts such as absorption costing. Park (1989) suggests a three-dimensional graphic representation to aid in teaching accounting students the impact of inventory changes on absorption and direct costing incomes. Duffy (1990) suggests that using graphical analysis can increase student conceptual understanding of interest capitalization. Schadewald and Limberg (1992) found that students using pictorial and textual representations outperformed students using only textual representations of routine problem solving tasks. However, they found that there were no significant differences between the two groups when tackling novel problems. Within the realm of accounting decisions, Moriarity’s (1979) investigation of multidimensional representations found that experienced practitioners and persons of limited accounting knowledge were able to more accurately make predictions using his representations. Holland, Lorek and Bathke (1992) found no significant differences among majors, for students (including MBA students with experience) when using graphics for extrapolative forecasting. Research has shown that while making decisions, novices and experts may benefit from the use of graphics within accounting and other disciplines. Additionally, in many instances multidimensional graphics help novices understand complex concepts. As a first step in visualization research in accounting, it is important to investigate whether novices are aided by multidimensional graphics.
III. THEORETICAL DEVELOPMENT The Cognitive Fit Model and Proximity Compatibility Principle are described in this section, as is their relationship to the current study. Using these concepts in conjunction with the prior literature discussion, the hypothesis and research question are developed.
7
Cognitive Fit Model The Cognitive Fit Model (CFM) suggests that the effectiveness of the problem-solving process is a function of the relationship between the problem-solving task and the problem representation (Umanath and Vessey 1995; Vessey 1991, 1994; Vessey and Galletta 1991). Using the CFM, Vessey (1991) provides an explanation for the mixed results in the "tables versus graphs" literature. When the problem representation "fits" the problem-solving task, a preferable mental representation of the problem will be realized and the solving speed and accuracy of the problem solving process will be improved. Vessey (1991, p. 223) concluded that “matching the problem representation to the type of task to be solved results in improved decision-making performance.” Therefore, it should be emphasized that when mismatches do occur between the problem solving task and the problem representation, decision-making performance suffers in terms of speed, accuracy, or both. Proximity Compatibility Principle The Proximity Compatibility Principle (PCP) suggests a set of guidelines on which to base a visual display design (Wickens and Carswell 1995; Wickens, et al. 1994). In a similar manner to the CFM, PCP ties together both the problem solving task and the problem representation. In PCP, the task is described as the processing, or task, proximity and the representation is described as the perceptual, or display, proximity. Task proximity has three levels of processing: integrative processing (high task proximity), non-integrative processing of similar tasks (lower proximity), and non-integrative processing of dissimilar sources (independent processing). In this study, we address integrative processing.6 Integrative processing requires combining multiple pieces of information through some form of mathematical manipulation. There are several different ways in which display proximity can be manipulated: spatial proximity, connections, source similarity,
8
code homogeneity, object integration, and configuration. In this study, we address spatial integration and object integration.7 Spatial integration requires that related variables be displayed in close proximity. In this manner, the user can see all semantically related information in one location. Object integration requires multiple variables to be displayed in such a manner that the variables appear to be part of a single object. In general, PCP suggests that the higher the level of required integration, the higher the level of proximity the display should possess, and vice versa. The lower the "fit" between the task and display proximity, the greater the "information access cost" will be. Hypothesis Development Visualizations provide a method of understanding complex information by accumulating, grouping, and displaying data in a manner that may be understood more effectively than by viewing the details of the data. For decision makers performing a prediction task using momentum accounting information, visualizations can be used as a tool to aid their understanding of the underlying functional relationships8. Based on CFM and PCP, we expect that there should be a difference in decision-making performance (prediction accuracy and time) based on the visual representation used. Furthermore, we expect that the better the "fit" of the visual representation to the dimensionality of the momentum accounting information, the more accurate the prediction. In this case, accuracy is defined in terms of how close the subject’s prediction is to the momentum accounting model prediction. In regards to the effect that rotation will have on decision-making performance, the previous results have been mixed (Wickens, et al. 1994; Pani 1993). However, due to issues related to occlusion, we believe that rotation may be capable of improving the accuracy of the prediction.9 As such, our null hypothesis is:
9
H1:
Increasing the dimensionality of the visualization does not increase the accuracy of predictions of novices based on momentum accounting information.
For the purpose of this research, the dimensionality relationships are operationally defined as 2D < 3D < 3D rotatable. Based on these relationships, the null hypothesis implies decisions will not differ as dimensionality increases from 2D to 3D to 3D rotatable. In addition to prediction accuracy, the time to make a decision is important to the decision process. Vessey (1991, 228) suggests that there are only two objective performance variables for decisions: accuracy and decision time. Moriarity’s (1979) research with the multidimensional Chernoff faces found significant decision time differences between experimental groups. Benbasat and Dexter (1986) suggested that when using multiple forms of information representation to solve problems, differences in time to solve problems might exist. These findings are in agreement with the expectations set out by the CFM and PCP. However, there have been no studies that have investigated the effects of 3D and rotatable 3D images on decision time. Although from a CFM and PCP perspective, 3D and rotatable 3D images represent a more natural "fit" for multidimensional information, due to issues related to occlusion and the mechanics of rotating the 3D images, a decision-maker could take more time when using a 3D image than when using a 2D image. Moreover, this increase in decision time when using 3D images could be even more pronounced for novices, relative to experienced decision makers. As such, we also investigate the following research question: RQ1: How does the dimensionality of the visualization affect the time to make a prediction based on momentum accounting information? IV. METHOD A laboratory experiment was conducted to investigate the hypothesis and research question. This section describes methods used to develop the components of the experiment, including the
10
experimental visualizations, task and reward system; the techniques used to select subjects and companies are also described. Additionally, the dependent variables are identified and operationalized. Experimental Visualizations Three visual representations were developed for this study (referred to as R1, R2, and R3). Since the experimental task was an integrative prediction task (described below), it was expected that the two-dimensional representation would be associated with the lowest subject performance in the experiment. Each subject was exposed to only one of the three visual representations. The first representation, R1, a two-dimensional line graph, demonstrated the functions of wealth, momentum and impulse independently— three functions within momentum accounting (see Figure 1). Wealth was operationally defined as the market value of a company at a specific point in time; momentum as the rate of wealth change (first derivative of the wealth function); and impulse as the rate of momentum change (second derivative of the wealth function). Each graph contained three lines, with each line representing a function for the wealth, momentum or impulse of a company. This graph was a combination of the typical representations based on the physics concepts that underlie momentum accounting (Brown, et al. 1995, p. 57; Halliday, et al. 1993, p. 21). Furthermore, since the experimental task (described below) was a prediction task, a line graph seems appropriate. Users of the graph should be able to incorporate the rate of wealth change (momentum) and rate of momentum change (impulse) when making their predictions of future wealth. The data displayed in this graph were semantically related and the task was integrative. Therefore, a spatial proximity manipulation was used to make this graph as integrative as possible. The graph displayed the relationships between the data in close proximity. To put the three lines
11
into close proximity required multiple y-axes and three different y-axes scales because of the unit of measurement differences between the three functions. The first y-axis used wealth, the second momentum, and the third used impulse. The momentum and impulse scales were multiplied by a constant to allow similar relative values so that they were capable of being graphed in close proximity with wealth. The x-axes, measured in months, ranged from one through 120 for each of the representations. The lines were displayed to show the functions for months seven through 100 for a specific company. Each line on the visualization was displayed in a unique color. The y-axes range, measured in dollars, varied based on the individual companies used for data. (Insert Figure 1 here) The second visualization, R2, used an object integration manipulation to maximize the level of integration supported by this graph. It combined the three 2-dimensional lines from the first representation into a modified trajectory line graph (see Figure 2). A trajectory line graph is a line graph that curves through three separate dimensions (Harris 1996). This representation was chosen to minimize differences between the representations; both are forms of line graphs. In our modified trajectory line graph, time (month) was maintained as the x-axis, wealth as the y-axis, and momentum was added as the z-axis. As a fourth dimension, we used the color of the line at any point on the line to represent the impulse for a company at that point in time.10 The line was plotted based on the values for the functions for the same time range used in the 2-dimensional representation. This representation was displayed from a position of 30 degrees above the xz plane and 35 degrees to the left of the yz plane.11 (Insert Figure 2 here) The third representation, R3, looked identical to the R2 representation (see Figure 2). The only difference was the ability of the subjects to rotate the image to view the data relationships
12
from different perspectives. By allowing subjects to rotate the image, issues related to occlusion could be mitigated. The vertical rotation was allowed for 360 degrees (completely around the yaxis) and the horizontal rotation was allowed for 180 degrees (the top half of the sphere divided by the plane xz).12 Function Determination/Company Selection The wealth function for a specific company was generated based on historical data for the company’s monthly market equity (month-end stock price times the number of shares outstanding at the end of the month). The derivative of the generated wealth function was taken (with respect to time) yielding the momentum curve. A second derivative was taken to produce the function of impulses. These three functions were used to generate the visualizations and were used for predictive purposes. The company data used in this study was selected from Compustat. Only those companies that had monthly market values for November 1980 through May 1995 were selected. This criterion was necessary to ensure that the companies were in business during the entire period for which the wealth functions were to be generated. From these criteria, 188 companies were selected that did not pay dividends during the period.13 These companies were ranked according to total market value at the end of 1995; the group was then divided into thirds. Two companies were selected from each of the high and low thirds, based on their relative rate of wealth growth during the period, and total wealth. These companies are identified as large company/high growth (LH), large company/low growth (LL), small company/high growth (SH), and small company/low growth (SL). Two other companies were identified as having “interesting” patterns14 and were used as training cases for the experiment.
13
The market values for the six experimental companies were used to generate an “nth” degree polynomial to plot the data for each company. The polynomial was considered to be an acceptable fit of the function when the graph displayed the major turns of the function and the Rvalues for the polynomial generally were in the range of .7 to .9. After the generation of the model functions and their first and second derivatives, 133 months of data were generated for wealth, momentum, and impulse values for the each of the companies. Months 1-6 and 121-133 were eliminated, because of statistical issues that may exist in rounding endpoints when generating equations from data points. Since months 101 - 120 were included the prediction period of the task, they were not used. Months 7 - 100 were used to develop the visualizations used in the experiment. Measures The data collected from the subjects included numerical values of wealth predicted for a future period by each subject. For each company, the subjects predicted wealth for one, ten and twenty month(s) beyond the values contained in the visualization. The dependent variable, predicted wealth accuracy (WA), was measured for each subject, for each prediction period, by subtracting the subject’s predicted wealth (S) from the model's predicted wealth (M). WA = M - S
(1)
The model predicted values were obtained from the wealth functions describe above for the months of 101, 110 and 120. For data analysis purposes, we converted this measure to a standardized measure that expressed the wealth accuracy relative (WAR) to the model's prediction. WAR = (M - S) / M
(2)
14
A second dependent variable examined consists of the time it takes a subject to process a visualization and to make a prediction. This variable, prediction time, was operationalized as the number of seconds expended since the previous response. This calculation was performed using the internal clock of the PC. There was no fixed time limit imposed on the subjects although the reward system, described below, was structured to provide incentives for speed if it was not obtained at the expense of accuracy. Because of the method of collecting times, each prediction generated an occurrence of the time variable. Subject Selection Subjects for this research were senior business school students at a major eastern state university. These students were used because they are close to entering the job market in positions that should require basic decision-making. They were a relatively homogeneous group with basic training in financial decision-making. Although they had exposure to financial statement representations, they were not biased for or against the “new” concepts presented by momentum accounting. The usable sample consisted of 124 subjects allowing for the assignment of approximately forty students to each of the three case groups. Subjects were recruited from two courses at the university. One course was a senior-level auditing course with class members being exclusively accounting majors. The second course was an upper-level managerial accounting course with members being primarily of senior status and business school, non-accounting majors. Each subject was randomly assigned and exposed to one of the three visual representation groups.15
15
Experimental Task The experimental task was completed using personal computers, configured in a laboratory environment. Each of the laboratory booths contained identical personal computers. Each machine had a two-button mouse and a 15-inch color VGA monitor. Upon entering an assigned booth, each subject read on-screen instructions and completed practice predictions for one of the three visual representations.16 After the practice problems, all participants were presented with visualizations for four companies and asked to make predictions for three future periods for wealth. The predictions made by subjects required them to recognize trends of wealth, based on the visualizations (R1-R3) presented. The predictive results and additional information were collected on a diskette along with the times each subject took to answer each question in the training session and actual case. The case was similar to the case in DeSanctis and Jarvenpaa (1989). Reward System A reward system was developed to attract and compensate the participants. The compensation package included guaranteed cash and extra credit in their course. The guaranteed cash consisted of six dollars per hour, paid in one-tenth hour increments and the extra credit received was a one percent bonus of the total possible points available in the course. Since no subjects were members of both classes, the grade compensation was equal for all subjects. In addition to the guaranteed segments of the reward system, each subject was entered into a “bonus pool” with the chance of winning additional cash. The higher the performance of the subject, the greater the likelihood of winning the additional cash. Three “pools” were established, one for each type of representation (R1, R2 and R3), to eliminate compensation bias should one representation be more effective than the others.
16
V. RESULTS AND DISCUSSION The experiment yielded twelve wealth predictions (three for each of the four companies) and the associated prediction times for 124 subjects. The difference between the subjectpredicted value and the model-predicted value was used as the measure of the accuracy of the prediction. Since the model-predicted value represents the “best” information that was available to the subjects when making their decisions, it was used in this calculation. The model-predicted value was determined by substituting the desired prediction period into the formula used to generate the visualization. The data required for the research question were collected based on the time required to make a prediction. For each of the four case companies, each subject made three predictions for time periods extending beyond the information provided. One would expect, as the times increased (the predictions reaching further into the future) the accuracy of predictions would decline. These expected differences suggest that the within-company predictions should be examined individually, rather than in aggregate. In addition to the expected variability within company predictions, a wide variation was expected between companies due to the characteristics of the companies used in the experiment.17 Due to these expectations, it was determined that the accuracy of the predictions for each company should be examined independently. In order to evaluate the data independently with respect to company and prediction time “distance," the accuracy data was segmented by observation made by the subjects. Each subject made exactly one prediction for each time period for each company. A randomized block18 design was used where the representations were blocked by subject observations. The mean for each block was calculated providing the model with one observation per cell. These data, reported in table 1, in conjunction with Friedman’s Rank Sums Test19, were used to test the hypothesis.20
17
Friedman’s test (S= 7.17), using the representations as treatments, resulted in a p-value of .028. This provides strong evidence that there were differences in the accuracy of predictions, based on the visual representation used to make those predictions. (Insert Table 1 here) Although this evidence supports rejection of H1, a major issue in this research concerns not merely the existence of treatment differences, but the direction of those differences. As described earlier, it was expected that R3 would provide the highest accuracy of the treatments, R2 the midrange of accuracy, and R1 the lowest accuracy. With this expectation, Page’s (1963) test21 was used to test for the direction of the relationships. Page's test (L= 155) resulted in a range of pvalues from 0.01 to 0.05. 22 This result suggests strong support for the order of accuracy of representations where at least one of the inequalities was not a strict equality. In this instance, this implies that as the dimensionality of the representation was increased, i.e., 2D to 3D to 3D rotatable, the accuracy of the decision was improved. Evidence from Page’s test, in conjunction with the result from Friedman's test, supports rejection of the null hypothesis (H1) and is consistent with the alternative that proposed increased accuracy of predictions using multidimensional representations of multidimensional data. Results from the tests regarding H1 are consistent with several streams of research. First, presentation format importance was confirmed. MacKay and Villarreal (1987) and Stock and Watson (1984) all found that differences in decisions occur based on the format that the data were presented to the decision-maker. Furthermore, Wickens, Merwin and Lin’s (1994) finding that 3D images are superior to 2D images when subjects are required to extract data was corroborated by the current study. Within the accounting field, the current study findings were consistent with
18
the preferable decision outcomes when using multidimensional representations (MacKay and Villarreal 1987; Moriarity 1979; Stock and Watson 1984.) The research question suggests that there could be differences for time required to make a prediction using the different visual representations. Again, Friedman’s test was used to examine the research question. The means and ranks used to investigate the research question (R1 R2 R3) are included in Table 2. The results of the Friedman’s test (S= 15.5) indicate strong support (p-value = 0.001) for differences between the three representations with respect to prediction times between all three visual representations. We also investigated the three binary comparisons between the three representations: R1 R2, Friedman’s test (S= 1.33) indicate no support for differences (p-value = .248) R1 R3, Friedman’s test (S= 12.00) indicate strong support for differences (p-value = .001) R2 R3, Friedman’s test (S= 8.33) indicate strong support for differences (p-value = .004) These results were consistent with prior research that suggests that time differences exist when using multiple forms of information to solve problems (Benbasat and Dexter 1985). (Insert Table 2 here) Based on these results, we investigated the following relationships. Again, Page's test was used to test the significance of the relationships. R1 ≤R3, Page’s test (L= 60) indicate strong support for differences (p-value = 0.000) R2 ≤R3, Page’s test (L= 59) indicate strong support for differences (p-value = 0.000) R1 ≤R2 ≤R3, Page’s test (L= 159) indicate strong support for differences (p-values range from 0.001 to 0.01) R2 ≤R1 ≤R3, Page’s test (L= 162) indicate strong support for differences (p-values range from 0.000 to 0.001)
19
From a practical perspective, these results imply that static visual representations (e.g., R1 and R2) may take less time to support decision making than dynamic visual representations (e.g., R3). In this instance, going from a 2D visual representation (R1) to a 3D visual representation (R2) did not affect the amount of time to make the decision. Furthermore, the combined time and accuracy results suggest that the static 3D (R2) visual representation may be more effective than the static 2D (R1) or 3D rotatable (R3) visual representations. The R2 representation was more accurate than R1 and took less time than R3. Furthermore, R2 seemed to take about the same amount of time as R1. However, additional research is required to more fully explore these relationships. Although there was evidence that prediction time differences result from the different treatments, there are several factors to consider in interpreting these results. One factor that may have affected the time differences is the process by which individuals make decisions. Tan and Benbasat’s (1993) study examines decision-making using 2-dimensional graphs and describes the process used to make a decision. The process includes anchoring, disembedding, and projecting, data points and values. It is also possible that a deeper knowledge of each of the visualizations could impact the time differences noted in Research Question 1. Future research should include additional training to mitigate any novice/expert effect of the representations. The relationship between time and accuracy should also be considered. Furthermore, it is possible that R3 may take less time than either R1 or R2 with a more difficult task. Additional research of the 3D decision-making process considering these factors may provide insight into the intricacies of this issue.
20
VI. CONCLUSION AND LIMITATIONS The results of this study indicate that the form of the representation of data affects the accuracy of the predictions novices make based on that data. Additionally, one can conclude that multidimensional visual representation of complex multidimensional data results in greater decision making accuracy because it facilitates the direct examination of the complex relationships in the data. This conclusion has implications within the realm of accounting with respect to decision-making when multiple variables are involved. As variables increase in complexity (defined as dimensionality), there should be representations that show the interaction among the variables to help enhance decision-making accuracy. Furthermore, with the trend towards supporting decision-making with multidimensional data using on-line analytical processing (OLAP), knowledge discovery, and data mining tools, this conclusion implies that in the future, designers of advanced accounting information systems should be cognizant of the need to “fit” the dimensionality of the data to the dimensionality of the visual representation. Otherwise, the effectiveness of the decision maker may be compromised. Although decision accuracy was improved by using the 3-D formats, the results do not indicate that in this instance decision-makers use less time in forming their conclusions when using these higher-dimension representations. The results do indicate that differences exist in decision time, but these differences may be due to the physical limitations of rotating the graphic, lack of familiarity with the visualization tool, or other factors not investigated here. Increased complexity may add time to the decision-maker’s interpretation time, which is a possible explanation of the data differing from the expectations given the minimal training that subjects received. It also is possible that higher-dimension representations may only decrease decision making time with a task more difficult than the one used in this study. Finally, it is possible that if the subjects are not
21
comfortable with the complexity of the task, they will take more time to investigate and understand the visualizations. A principal assumption of this study is that the visualization employed is appropriate for momentum accounting information. In addition, other limitations may be identified with the experiment. First, it is questionable whether the results reported in this paper can be generalized beyond momentum accounting. Second, the results may have been partially confounded due to the subject's lack of familiarity with momentum accounting and the multidimensional visualization tool. Future research should conduct additional training to level the knowledge of the tools, prior to the experimental trials. Expert users of the tools should also be considered in addition to the inexperienced users examined in the current research. Finally, the issue of visual impairments and color-blindness were not considered. Basic sight problems such as these may affect an individual’s ability to use the presentations to make decisions. Although research in accounting visualizations primarily has focused on simple graphics, research in other fields has demonstrated benefits using data/information visualization technologies. Further research into the application of these technologies to accounting problems may yield similar results. Finally, visualization technologies may enable multidimensional approaches to accounting problems, such as momentum accounting, to finally be comprehensible and useful.
22
REFERENCES Adriaans, P. and D. Zantinge. 1996. Data Mining. Harlow, England: Addison-Wesley. Amer, T.S. 1991. An experimental investigation of multi-cue financial information display and decision making. Journal of Information Systems 5(Fall):18-34. Benbasat, I. and A.S. Dexter. 1985. An experimental evaluation of graphical and color-enhanced information presentation. Management Science 31(11): 1348-64. -------. 1986. An investigation of the effectiveness of color and graphical information presentation under varying time constraints. MIS Quarterly (March): 58-81. Bertin, J. 1983. Semiology of Graphics. Madison, WI: The University of Wisconsin Press. Blommaert, A.M.M and E.A.M. Olders. 1995. Renewing accounting systems. Triple entry and momentum accounting: an exploratory study. Proceedings of the First Asian Pacific Interdisciplinary Research in Accounting Conference, University of South Wales, Sidney, Australia. Bouwman, M.J., P. Frishkoff, and P.A. Frishkoff. 1995. The relevance of GAAP-based information: a case study exploring some uses and limitations. Accounting Horizons 9 (4): 22-47. Brown, J.R., R. Earnshaw, M. Jern, and J. Vince. 1995. Visualization: using computer graphics to explore data and present information. New York: John Wiley & Sons. Carbone, R. and W.L. Gorr. 1985. Accuracy of judgmental forecasting of time series. Decision Sciences 16:153-60. Card, S.K., J.D. Mackinlay, and B. Shneiderman. 1999. Readings in Information Visualization: Using Vision to Think. San Francisco, CA: Morgan Kaufmann.
23
Chernoff, H. 1973. The use of faces to represent points in k-dimensional space graphically. Journal of the American Statistical Association 68 (June): 361-68. Choras, D. N. and H. Steinmann. 1995. Virtual Reality: Practical Applications in Business and Industry. Englewood Cliffs, NJ: Prentice Hall. Cleveland, W.S. 1993. Visualizing Data. Summit, NJ: Hobart Press. Cleveland, W.S. 1994. The Elements of Graphing Data, Revised Ed. Summit, NJ: Hobart Press. Cooper, L.A. 1990. Mental representation of three-dimensional objects in visual problem solving and recognition. Journal of Experimental Psychology 16 (6): 1097-106. DeSanctis, G. and S. Jarvenpaa. 1989. Graphical presentation of accounting data for financial forecasting: an experimental investigation. Accounting, Organizations, and Society 14 (5,6): 509-25. Duffy, W.A. 1990. A Graphical Analysis of Interest Capitalization. Journal of Accounting Education 8:271-284. Fayyad, U.M., G. Piatetsky-Shapiro, P. Smyth, and R. Uthurusamy (Eds.) 1996. Advances in Knowledge Discovery and Data Mining. Menlo Park, CA: AAAI Press / MIT Press. Fraser, I.A.M. 1993. Triple-entry booking: a critique. Accounting and Business Research 23 (90): 151-8. Goldwater, P.M., and T.J. Fogarty. 1995. Cash flow decision making and financial accounting presentation: a computerized experiment. Journal of Applied Business Research 11 (3): 16-29. Grantham, C. 1993. Visualization of Information Flows: Virtual Reality as an Organizational Modeling Technique. In Virtual Reality: Applications and Explorations, Wexelblat, A. (ed.). Cambridge, MA: Academic Press Professional.
24
Hallgren, R.C. and S. Gorbis 1999. Visualization technology in medical education. T H E Journal 26 (6): 66-68. Halliday, D., R. Resnick, and J. Walker. 1993. Fundamentals of Physics: Extended. 4th ed. New York: John Wiley & Sons. Harris, R.L. 1996. Information Graphics: A Comprehensive Illustrated Reference. Atlanta, GA: Management Graphics. Holland, R.G., K.S. Lorek and A.W. Bathke, Jr. 1992. A comparative analysis of extrapolative and judgmental forecasts. Advances in Accounting 10: 279-303. Hollander, M. and D.A. Wolfe. 1973. Nonparametric Statistical Methods. New York: John Wiley & Sons. Ijiri, Y. 1982. Triple-Entry Bookkeeping and Income Momentum: Studies in Accounting Research No. 10. Sarasota, FL: American Accounting Association. -------. 1986. A framework for triple-entry bookkeeping. The Accounting Review 61 (4): 74559. -------. 1990. The evolution of bookkeeping to triple-entry systems: multimedia authoring experience. Working paper. Carnegie Mellon University. Jones, C.V. 1996. Visualization and Optimization. Boston, MA: Kluwer Academic Publishers. Jones, D.R. and D.A. Schkade. 1995. Choosing and translating between problem representations. Organizational Behavior and Human Decision Processes 61 (2): 214-23. Kosslyn, S.M. 1980. Images and Mind. Cambridge, Mass: Harvard University Press. Kosslyn, S.M. 1994. Elements of Graph Design. New York, NY: W.H. Freeman and Co. Larkin, J.H. and Simon, H.A. 1987. "Why a diagram is (sometimes) worth ten thousand words." Cognitive Science (11): 65-99.
25
Lockhart, J. 1997. Personal communication. MacKay, D.B. and A. Villarreal. 1987. Performance differences in the use of graphic and tabular displays of multivariate data. Decision Sciences 18: 535-46. Markham, S.E. 1998. "The scientific visualization of organizations: A rationale for a new approach to organizational modeling." Decision Sciences 29(1): 1-23. Miller, G.A. 1956. "The magical number seven, plus or minus two: some limits on our capacity for processing information." Psychological Review (63): 81-97. Moriarity, S. 1979. Communicating financial information through multidimensional graphics. Journal of Accounting Research 17 (1): 205-24. Page, E.B. 1963. Ordered hypotheses for multiple treatments: a significance test for linear ranks. Journal of the American Statistical Association 58: 216-30. Pani, J.R. 1993. Limits on the comprehension of rotational motion: mental imagery of rotations with oblique components. Perception 22: 785-808. Park, H.G. 1989. A Three-Dimensional Graphic Display of the Impact of Inventory Changes on Absorption and Direct Costing Incomes. Journal of Accounting Education 7: 279-292. Parsaye, K. and M. Chignell. 1993. Intelligent DataBase Tools & Applications: Hyperinformation Access, Data Quality, Visualization, Automatic Discovery. New York, NY: John Wiley & Sons. Schadewald, M. and S. Limberg. 1992. Using Pictorial Models to Teach Complex Tax Rules: An Experimental Investigation. Journal of Accounting Research 10: 133-149. Schkade, D.A. and D.N. Kleinmuntz. 1994. Information displays and choice processes: differential effects of organization, form and sequence. Organizational Behavior and Human Decision Processes 57: 319-37.
26
Shenk, D. 1997. Data Smog: Surviving the Information Glut, Revised and Updated Ed. San Francisco, CA: HarperCollins. Shepard, R. N. and L. A. Cooper. 1982. Mental Images and Their Transformations. Cambridge, Mass: The MIT Press. Schroeder, W., K. Martin and B. Lorensen. 1998. The Visualization Toolkit: An ObjectOriented Approach to 3D Graphics, 2nd Edition. Upper Saddle River, NJ: Prentice-Hall. SPSS, Inc. 1998. SPSS Base 8.0 Applications Guide. Chicago, IL: SPSS. Stock, D. and C.J. Watson. 1984. Human judgment accuracy, multidimensional graphics, and humans versus models. Journal of Accounting Research 22 (1): 192-206. Tan, J.K.H. and I. Benbasat. 1993. The effectiveness of graphical presentation for information extraction: a cumulative experimental approach. Decision Sciences 24: 167-191. Thierauf, R.J. 1995. Virtual Reality Systems for Business. Westport, CT: Quorum Books. Thomsen, E. 1997. OLAP Solutions: Building Multidimensional Information Systems. New York, NY: John Wiley & Sons. Tufte, E.R. 1983. The Visual Display of Quantitative Information. Chesire, CT: Graphics Press. Tufte, E.R. 1990. Envisioning Information. Chesire, CT: Graphics Press. Tufte, E.R. 1997. Visual Explanations. Chesire, CT: Graphics Press. Umanath, N.S. and I. Vessey. 1995. Multiattribute data presentation and human judgment: a cognitive fit perspective. Decision Sciences 25 (5,6): 795-824. Vessey, I. 1991. Cognitive fit: a theory-based analysis of the graphs versus tables literature. Decision Sciences, 22(2): 219-41. Vessey, I. 1994. The effect of information presentation on decision making: A cost-benefit analysis. Information & Management, 27: 103-119.
27
Vessey, I. and D. Galletta. 1991. Cognitive Fit: An empirical study of information acquisition. Information Systems Research, 2(1): 63-84. Wickens, C.D., and C.M. Carswell. 1995. The proximity compatibility principle: its psychological foundation and relevance to display design. Human Factors 37(3): 473- 94. Wickens, C.D., D.H. Merwin and E.L. Lin. 1994. Implications of graphics enhancements for the visualization of scientific data: dimensional integrity, stereopsis, motion and mesh. Human Factors 36(1): 44-61.
28
FIGURE 1 R1 – 2 Dimensional Representation of Accounting Wealth, Momentum and Impulse
29
FIGURE 2 R2/R3 – 3 Dimensional Representation of Accounting Wealth, Momentum and Impulse
Impulse Scale
x-axis – Month y-axis – Momentum z-axis – Wealth
30
TABLE 1 Standardized Wealth Accuracy Means and (ranks) used with Friedman’s and Page’s tests for H1 Block\Representation Company 1: Obs 1 Company 1: Obs 2 Company 1: Obs 3 Company 2: Obs 1 Company 2: Obs 2 Company 2: Obs 3 Company 3: Obs 1 Company 3: Obs 2 Company 3: Obs 3 Company 4: Obs 1 Company 4: Obs 2 Company 4: Obs 3
9.842 18.155 21.625 1.962 1.060 -0.020 -0.098 -0.689 -0.912 -0.977 -0.980 -0.983
R1 (2) (3) (3) (2) (1) (1) (1) (3) (3) (3) (3) (3)
31
9.844 17.694 19.754 1.979 1.725 0.141 1.048 0.010 -0.657 -0.945 -0.961 -0.973
R2 (3) (2) (1) (3) (3) (3) (3) (1) (1) (2) (1) (2)
9.068 17.173 21.329 1.862 1.154 0.091 0.138 -0.256 -0.775 -0.933 -0.962 -0.971
R3 (1) (1) (2) (1) (2) (2) (2) (2) (2) (1) (2) (1)
TABLE 2 Question/Observation Response Time Means and (ranks) used with Friedman’s and Page’s tests for RQ1 Block \Representation Company 1: Obs 1 Company 1: Obs 2 Company 1: Obs 3 Company 2: Obs 1 Company 2: Obs 2 Company 2: Obs 3 Company 3: Obs 1 Company 3: Obs 2 Company 3: Obs 3 Company 4: Obs 1 Company 4: Obs 2 Company 4: Obs 3
45.4 25.9 19.3 43.4 22.0 20.3 39.3 18.9 14.6 30.3 22.5 15.2
R1 (1) (2) (2) (2) (2) (2) (1) (2) (1) (1) (2) (2)
32
50.3 22.6 15.4 42.1 21.9 15.6 47.7 17.6 17.0 44.9 18.8 13.4
R2 (2) (1) (1) (1) (1) (1) (2) (1) (2) (3) (1) (1)
52.6 35.1 29.4 52.9 32.4 22.6 53.5 41.2 21.4 42.3 25.0 16.8
R3 (3) (3) (3) (3) (3) (3) (3) (3) (3) (2) (3) (3)
ENDNOTES 1
Our reasoning for using momentum accounting (MA) rather than traditional concepts, was
twofold. First, given the functionality and derivatives associated with MA, it represents a “natural occurrence” of multidimensional data within accounting. , MA thus appeared to be an ideal choice to investigate the potential advantages of visualization technologies for presenting accounting information. Second, for experimental purposes, it was deemed necessary to minimize subjects’ prior knowledge of the topic. Since student subjects used in the experiment had no exposure to MA, there was little risk of them being biased either for or against the technique. 2
For a more complete description of visualization examples and issues, see Bertin (1983), Card,
et al. (1999), Cleveland (1993, 1994), Jones (1996), Kosslyn (1994), and Tufte (1983, 1990, 1997). 3
See Blommaert and Olders (1995) and Ijiri (1982, 1986, 1990) for a more complete description
of momentum accounting. 4
In addition to a traditional balance sheet, income statement and notes, the subjects received a
“Combined Momentum/Impulse/Action Statement” to convey the momentum accounting information (Blommaert and Olders 1995, 17) 5
Chernoff used facial characteristics as variable representations for multivariate analysis
(Chernoff, 1973). 6
The experimental task used in this study is an integrative task.
7
The experimental visualizations used in this study were manipulated using spatial (2D
visualization) and object (3D visualizations) integration. 8
The experimental task used was similar to the task in DeSanctis and Jarvenpaa (1989)
33
9
In 3D graphics, if an object is occluded, then the observer cannot see it since it is covered by
objects that are "closer" to the observer. In this instance, it is possible for the trajectory line graph to cover part of itself. As such, the only way the observer could see the entire graph is to rotate it. 10
The use of a color overlay to add information to a graph is common when creating
visualizations. See Chapter 5 of Brown et al. (1995), Chapter 7 of Kosslyn (1994), and Chapter 5 of Tufte (1990). 11
This angle was selected through observations of other static 3-dimensional representations and
discussions with several pilot subjects. These individuals were allowed to rotate a graph and asked what angle they would choose to display the graph for decision making, if the image could not be rotated. With no exceptions, the individuals chose the approximate angle that was used in this research. 12
A graphics expert suggested that subjects would receive no additional benefit from the bottom
half of the sphere (Lockhart 1997). 13
Companies that did not pay dividends were selected because dividends may have caused
discontinuities in the wealth functions used in the experiment. 14
Interesting as defined by a variety of directional moves relative to wealth, momentum and
impulse. 15
The random assignments were tested by statistically evaluating results based on gender and
college major. No differences were noted between groups. 16
Based on the results in DeSanctis and Jarvenpaa (1989), multiple practice predictions were
performed using the "interesting" (see footnote 14) firms. Prior to the experiment, each subject
34
was trained using an interactive automated routine. For each of two trial companies, three practice predictions were made for wealth, three for momentum, and one for impulse. Each practice prediction was followed by feedback that provided the model-predicted value for that prediction interval. Each subject was exposed to only one type (dimension) representation for the training and experiment. 17
One large, high growth company; one large, low growth company; one small, high growth
company; and one small, low growth company 18
This type of design is also known as a within-subject or repeated measures design.
19
Friedman’s Test is the nonparametric alternative to a repeated analysis of variance (SPSS, Inc.
1998). It is a distribution free test requiring data with one observation per block (Hollander and Wolfe 1973.) The distribution free characteristic is important considering the unknown, nonnormal distributions of the samples collected in this research. To test the hypothesis of treatment effect equality, Friedman’s Test ranks the observations within a block, and evaluates the rankings for a specific treatment (Hollander and Wolfe 1973.) 20
The researchers analyzed the initial data, and various transformations (log, square root, and
inverse) of the data to determine if they met the normality of distribution and homogeneity of variance assumptions for a classical Analysis of Variance test. The procedures used for this analysis included the Kolmogorov-Smirnov Goodness-of-Fit Test, Levene’s Test and Bartlett’s Test. After determining the data (and transformations) did not meet these assumptions, nonparametric procedures were used to report results. 21
Page’s Test is a distribution-free test, based on the Friedman rank sums test. While Friedman’s
Test is used to look for the existence of differences between treatments, Page’s Test is used to
35
evaluate the order of those differences; the order should be identified prior to performing the test. Each block is ranked by the relative position among the treatments. 22
P-values obtained from Hollander and Wolfe (1973, p. 372).
36
