AbstractâUnderstanding and improving software development and maintenance ..... We used advanced visualization concepts to display up to ..... 2, pp. 61â82, 2005. [6] J. Seo and B. Shneiderman, âA rank-by-feature framework for interactive.
1
CodeVizard: Combining Abstraction and Detail for Reasoning About Software Evolution Christopher Ackermann and Nico Zazworka University of Maryland {chrisack, nico}@cs.umd.edu
Abstract—Understanding and improving software development and maintenance activities are the main objectives of software engineering. Experimental studies have been used extensively to gather data and gain insight into these key points. Often times a large amount of data is collected but there is no strategy in place to analyze this information and extract valuable observations. At the same time the area of information visualization has been steadily growing as analysts start recognizing the power of appropriate visualization. We propose a solution to visualize software evolutionary data in a way that allows analysts to quickly capture interesting patterns and use interactive features to gain further insight to ultimately derive valuable information. The work described in this paper was motivated by concrete issues, researchers of the High Productivity Computing Systems (HPCS) project were facing when analyzing source code files captured during class experiment with the goal to develop insight into the programming behavior of programmers new to the domain of High Performance Computing (HPC). We developed a visualization tool called CodeVizard, which visualizes metrics that describe important characteristics of the captured source code versions in one panel (metric view) and displays all versions of the source code lined up with transitions describing the kind of maintenance activity each source code line experienced in another panel (code view). The combination of metric view and code view should allow analysts to quickly recognize interesting patterns in the data and to then use detailed information to support or refuse hypotheses. Index Terms—Software Visualization, Software Evolution, High Performance Computing.
I. I NTRODUCTION The goal of software engineering is to increase the efficiency of software development and at the same time achieve high product quality. One key interest is to understand the process of software evolution, i.e. how the software changes over time. Researchers and pracitioners have been struggling to comprehend the characteristics of software evolution due to several reasons. First, every software product is unique and it is difficult to establish generalizable rules that pertain to all software systems [1]. Secondly, it is not clear what measure can be used to evaluate a software procuct. There has been extensive research in finding software metrics that can be used to describe interesting properties software properties, such as size measures and design complexity measures. These metrics have shown some success in describing software products. The challenge many face when coping with software metrics is that it is difficult to draw conclusions solely based on a number value. Two approaches can support the software evaluation with metrics: analyzing the interplay of
different measures and the considering data points in relation to other data points. By using multiple measures at one time the analyst can see how one metric value behaved in relation to another measure. The meaning of one data point also becomes more clear when put into relation to other data points, such as values recorded earlier or later in the software development process. It is also difficult to reason about software solely based on metrics as there are numerous factors that need to be taken into account. It is therefore necessary to include low level information into the analysis process that can help to understand the metric values at a certain point and also to support or refute a claim made based on an evaluation of metrics. We propose an appproach to combine high level data with more detailed information to allow for analyzing a large amount of data on any level of detail. We implemented this approach in a visualization tool called CodeVizard. It uses a special kind of bar graph to illustrate up to 4 dimensions of meta information (i.e. software metrics) at any time and a panel called the code view to show the source code of all versions of a single source code file lined up with transitions that reveal maintenance activties. Combining these two views allows analysts to quickly detect interesting patterns in the data and then conduct a more detailed analysis to support or refute hypotheses. The paper is organized as follows: In section 1 we will motivate the work by explaining the analysis tasks and issues that led to the demand of a tool like CodeVizard. Section 2 will then first talk about the basic ideas of CodeVizard and subsequently elaborate on the two main components: the metric view and the code view. The section will be concluded with two scenarios that illustrate in a concrete manner how CodeVizard can be used for software evolutionary analysis. Case studies to evaluate the usefulness of CodeVizard are described in section 3 along with results and suggestions for improvements. Section 4 concludes the paper and elaborates on future work. II. O BJECTIVE While the general goal of this research was to develop a technique that can help gain understanding in the process of software evolution, it was motivated by concrete needs of researchers facing the issues many encounter when attempting to analyze software development data.
2
In an effort to gather knowledge about the software development practices in parallel programming environments, the researcher conducted a number of class studies in which student subjects where asked to implement a small scale program. The progress of every assignments was captured using various tools that monitored the programming activities. That information was then stored in data bases as well as in XML files. A. High Productivity Computing Systems Project The issues we are addressing emerged during work done by researchers of the Development Time Working Group as part of the High Productivity Computing Systems (HPCS) project. Aiming to analyze the productivity of programmers that are new to High Performance Computing (HPC), the researchers captured data from over 20 HPC classes at several universities in the United States [2]. Various measurements such as effort data, defect rates, background information and work flow data was recorded to be able to evaluate how student programmers work and learn. The results should allow the researchers to give advice to vendors, create tools to improve productivity and to establish new learning concepts for HPC. The issues of software evolutionary analysis are not limited to the field of HPC but also occur in other areas, such as architectural and behavioral analysis. We therefore believe that further insight into the concepts of HPC are not necessary to understand the idea behind CodeVizard. B. Analysis Goals In order to approach this challenge, the researchers created a list of more concrete analysis targets: • Domain specific obstacles: Since High Performance Computing distinguishes itself from other programming approaches in several ways, the researchers were particularly interested in the obstacles the subjects experiences that were specific to HPC. • Defect Types: As there are domain specific obstacles in HPC, the type of defects found in systems developed with this approach are specific as well. The goal is to identify the types of defects introduced due to domain specific obstacles and the circumstances that led to them. • Debugging: Closely related to the previous point, this addresses the question as to how the subjects went about fixing the defects. • Development phases: One of the main objectives were the identification of so called development phases in the recorded student programs. The goal is to establish knowledge about what kind of development phases there are and the characteristics thereof. One approach divides the development process into phases of work and rework. Work being the implementation of new features and rework structural of functional modifications to existing features. C. Analysis issues The data analysis was conducted in collaboration of several researchers over an extended period of time. This was done
with use of basic information retrieval and analysis tools. Despite the high level of experience with the domain, the researchers reportedly struggled with the analysis. This was due to the amount and detail of information that was captured and also due to general issues of software evolutionary analysis stated above. In particular, three main issues were disclosed: Analyzing progress over time. In order to detect interesting characteristics of a software evolutionary process, it is necessary not to merely analyze a single version (e.g. the final version) of a file but to consider a number of code versions over time. The researchers could not find appropriate tool support for this type of analysis task. Information overload. Due to the fact that detailed data from many studied were collected the researchers had a large amount of data available. This prevented the researchers from getting an overview of the information, which they expressed to be a necessity. Information delocalized. The different pieces of information that were gathered during the class studies were not accessible at one location but stored at different places and formats. Implicit Information. Another challenge in understanding the development process is the high amount of implicit data. For instance, the total number of lines of code or the amount of code added for a version are important properties. However, they are not directly accessible as they are only implictly contained in the source code. III. C ODE V IZARD The goal of this research was to build a tool to support the analysis activities mentioned above and resolve the issues the HPCS researchers were struggling with. We used techniques from the information visualization domain to approach this challenge and developed a tool called CodeVizard. We will first explain the underlying idea of CodeVizard and will then elaborate on its different components and in particular the graphical user interface. A. Background Since the analysis done by the researchers of the Time Development Working Group focused on single files from individual subjects, we decided to set the same scope for CodeVizard. Abstraction. Only considering the entire development process over time makes it possible to detect patterns in the programming behavior of a subject. Thus, one of the main criteria was to illustrate this time series data, i.e. the change progress of the entire development process. An abstract view could allow analysts to get an overview of the information of interest. Detail. While abstract information is useful to derive a high level picture, it fails to take into account details that are crucial to be able to draw conclusions pertaining the development behavior. For instance, when only considering the amount of lines added to system, one could argue that major code additions illustrate the addition of new features. However, the lines could simply be comments. We see that to
3
be able to draw definite conclusions, one must support high level observations with more detailed information. A second main goal was therefore to also provide detailed information about the recorded software evolutionary process. Combining abstract representation with low level detail is the main goal of CodeVizard. Both the term of abstraction and detail are vague. We will clarify the different level of details visualized by CodeVizard when elaborating on the graphical user interface in the follow section. B. Components CodeVizard is composed of three main parts, the data reader, the computation component, and the graphical interface. Data reader. Since the tool was especially aimed on researchers of the HPCS project, we implemented a database reader that can access the information that was collected during the HPCS class studies. To also account for future needs and make the tool usable to a broader audience we added a data reader for Subversion repositories (SVN). The input to CodeVizard are all versions of a single source code file and possibly additional meta information. While the source code versions are needed to be able to show detailed change information, the meta information can be used to describe characteristics of the different versions and the entire development process, respectively. What kind of meta information can be read and displayed in CodeVizard will be explained in the section about the graphical user interface. Computation. The computation component is responsible for automatically computing meta information in form of software metrics that can be used to support the analysis process. The metrics that are currently computed are for each code version, how many lines of code were added, deleted, and modified. Additionally it computes a measure we call change amount, which is the sum of the previous three measures. How these metrics are used for the analysis process will be explained at a later point in this paper. In the following sections we will first introduce the main part of every information visualization tool, the graphical interface, and elaborate on the kind of information that is being displayed as well as possible user interaction to adjust the representation of the data displayed. C. Graphical Interface The graphical interface consists of two main parts: the metric view and the code view. The information illustrated in the metric view is abstract and completed by the information displayed in the code view, which illustrates more detailed information. More specifically, the metric view displays the values of a number of software metrics and the code view shows the actual source code and what modifications the code experienced. We will now elaborate on both views and then explain how these views interact and complement each other. D. Metric View The analysis of software is often guided by software metrics that describe different characteristics, such as change proneness and coupling. The purpose of the metric view is to visualize these code metrics.
Fig. 1. An overview of CodeVizard. The top part is called metric view and the bottom part is the code view.
Software Metrics. Software metrics are measures of properties of pieces of software or its specifications. They provide a quantitative basis for the development and validation of models of the software development process [3]. Researchers have used software metrics to analyze software evolution in various ways and from different angles. A thorough analysis requires considering not only one but multiple metrics at once. We used advanced visualization concepts to display up to 4 different metrics using a special kind of bar graph where each bar represents one code version that was captured during software development. The following 4 bar attributes are used to display the different dimensions: Width. The x-axis of the metric view represents a time line of the software development process. The width of the bar indicates how much time passed between capturing two versions. The corresponding code version for a bar is the one that was captured at the end of the bar. For instance, if the bar expresses that a high number of lines were added over 20 min, that means that the version that was captured after this 20 min had a large number of additional source code lines. This will become clearer when describing different analysis scenarios in the succeeding chapter. Position on the y-axis. The position of each bar on the yaxis corresponds to a metric value. The bars that are aligned on the bottom of the graph intuitively have low values and the bars on the top have high values. Thickness. Another dimension is expressed by the thickness of the bars. A thin bar indicates that the version has a low value for the respective metric. A thick bar signals high values. Color. The use of color can guide analysts in their work by highlighting important information [4], which are often times extreme values. In order to express a fourth dimension we used color coding. Low values are represented a light green color and high values are indicated by a dark red color. The closer a metric value is to the medium the more subtle the color is. 1) Discretization: When displaying multiple dimensions in a graph it is important to avoid an information overload. If too much data is shown on a high level of detail the analyst can not capture and comprehend the information but rather looses focuses since the display is too ”crowded”. A way to increase the amount of information that can be captured by reducing
4
the number of distinguished values, i.e. discretization. In this process an input space that consists of an infinite many values can be reduced to only a few values. This is done while retaining enough information value for the analyst to make qualitative judgments. Before converting the space of continuous values into a discrete value space, the inputs are normalized. While this leads to loosing information about the offset, it allows us to use the entire spectrum of discrete values for the given range of input values. In the second step, the normalized value space C is evenly divided into α sections: C1 , C2 , . . . , Cα with C = normalized input space Cn = n-th slice Every slice Cn has an upper bound Un and a value representing this slice Vn : Un = α ∗ δ Vn = (Un − Un−1 )/2 with U0 = 0 and δ = Cmax − Cmin where Cmin is the lowest value of the input space and Cmax is the highest value. To convert a single real value ϕ into a discrete value µ it is first determined in which slice Cn the real value ϕ is located, i.e. the Cn for which Un−1 < ϕ ≤ Un The discrete value µ is then the average value of this slice Vn . By adjusting the number of discrete values that are distinguished it is possible to adjust the level of granularity in which the values are shown. [6] [5] Using novel visualization concepts, the metric view can display up to 4 different software metrics at any time and yet does not overwhelm the analyst with information. The normalization and discretization approach was motivated by the rank-by-feature framework [5] implemented in the Hierarchical Clustering Explorer (HCE) and VizTree [6]. With the rank-by-feature framework users can easily perceive the most interesting projections, and grasp the overall ranking score distribution. Both these characteristics are also important when using software metrics for analysis since metric values are only meaningful when considered in context with other values. VizTree introduced the idea discretization as a means of illustrating only major differences in values and highlighting extreme values. Like CodeVizard it allows the user to adjust the number of discrete values and hence the amount of detail displayed. While the discretization makes the screen look less ”crowded”, it preserves enough detail to make qualitative judgements and find interesting patterns. E. Code View In addition to the metric view, analysts can consult the code view to analyze the data in more detail. The code view is
Fig. 2.
Dark blue lines indicate modified lines.
Fig. 3.
Red closing triangles signal deleted lines.
located at the bottom part of the window and displays the source code of all versions of a single file lined up horizontally. Color coding is used to indicate the maintenance activities (lines added, modified, deleted) that occurred between two versions. By zooming in and out of the view, the analyst can adjust the display to show a high level image of all versions, the actual source code of a few versions, or intermediate representations. The visual zoom in combined with a semantic view. When zooming out, the source code details are hidden to allow for better zooming performance and to provide the analyst only with the information that can be captured on that level of abstraction. Likewise, zooming into the code view reveals more details up to a point where the actual source code is displayed. Transitions The space in between two versions is used to illustrate which lines in the two versions correspond to each other. Lines that remained the same are connected by a light blue line. Objects of different shape and color are used to illustrate the maintenance activities that occurred between two versions. A dark blue line indicates that the source code line was modified in between the two versions 2. For lines that only exist in the predecessor code - in that case lines were deleted - a red triangle is drawn 3. Lines just existent in the successor - newly added lines - are represented by green triangles 4. As the zoom is one of the most powerful features of CodeVizard we will now describe the three different levels of abstraction that are distinguished. That is, the three views with different amount of semantic information. Please note, however, that the zooming is not discrete but is done continuously. CodeVizard automatically adjusts the level of semantic detail depending on the visual zoom factor. Code Block View At the lowest zooming level the code
5
F. Analysis Scenarios
Fig. 4.
Green opening triangles show where code is added.
versions are visualized as blocks (solidly colored rectangles), where the height of the block represents the length of the source code, i.e. the total number of lines of code. This view helps the analyst to judge on how much code was changed over a series of versions. With some further insight this helps to judge if a subject was working on adding new features or reworking on code. Also anomalies like deleting or modifying a larger set of lines get visible immediately. Also patterns like changing one specific line for testing purposes (e.g. a loop variable) can be discovered. Even more specific questions like how stable is a newly added set of code in the next versions can be answered quickly. Code Representative View By zooming further into the view the code blocks turn into a code representative view in which the outline of single source code lines are shown. Dark lines indicate the outline of the source code, which reveals details such as the line indention and length. Also, the shapes that indicate the maintenance activities show single lines. Presumed that code was formatted by the programmer (e.g. code intend, empty lines between logical modules) the analyst can get more insight into which part of the code was affected by the maintenance activities (e.g. functions, loops). Code Text View To ultimately judge about the idea behind a certain change the analyst can zoom in further to read the code. By cutting off the code after a number of characters we avoid that long code lines mess up the width of the representation of a code version. The cutoff value can be adjusted through a slider at the bottom. By pushing it to the left code versions are hidden completely and therefore the series of modification become more visible. Algorithm To calculate the transitions between two versions we implemented and modified the Levenshtein Distance Algorithm [7]. The basic algorithm can be applied on strings to count the minimal number of insertions, deletions, and modifications needed to convert a word A with letters (a1 , . . . , an ) into word B with letters (b1 , . . . , bm ). An optimized version of the Levenshtein algorithm has complexity O(n · m) and requires O(m) space in memory. Since we were only interested in differences of whole lines from one code version to the next we converted each line into one letter by applying a hashing function. Our algorithm has been extended by the functionality to retrieve the position of insertions, deletions and modifications, which decreased its space complexity to O(n · m).
CodeVizard was developed with certain analysis scenarios in mind, i.e. an idea of how the tool could be used to conduct the analysis tasks. For the following scenarios we will use a dataset that was recorded during a student’s implementation of ”Game of life”, which is a small scale program with a total of X number of lines of code. A version of the source code was captured at every compile. The dataset consisted of 160 versions that were developed over a total of 36 hours. The metrics we will use for scenarios are entirely computed by CodeVizard and include for each version: • Total LOC. The total number of source code lines. • LOC added / modified / deleted. The number of source code lines that were added, modified, and deleted to reach the version. • Change Amount. A variable describing the total maintenance effort that was applied to reach the current version. It is computed as follows: Change Amount = LOC added + LOC modified + LOC deleted. When the data is initially loaded into CodeVizard, the three dimensions y-position, color, and thickness that are encoded in the bars are set to the same metric: total LOC. 1) Scenario I: Another question asked in the case study was to identify the phases of work and rework. We loosely defined work as addition of new features and rework as structural or functional changes. More specifically, we asked the subjects to roughly divide the entire development process into these two phases. CodeVizard offers two possibilities to conduct such an analysis, either by using the metric view or the code view. We will now explain both approaches. Metric view. The basic idea of identifying the implementation of new features is to identify the versions in the code that experienced a lot of additions. Modifications and deletions will be considered as rework activities. Of course, these conventions are not exact but sufficient to illustrate the concepts. A more accurate and detailed analysis can be conducted by using both, the metric and the code view. In order to identify the versions with the ”work” characteristics, we highlighted the version with many code additions by simply setting the y-axis to represent the LOC added. The color and the height of the bars are set to represent the LOC modified and LOC deleted, respectively. The work phases are now illustrated by the bars that are positioned higher up on the y-axis. The light color bars indicate low change amount and the thinner bar represents few LOC deleted. Figure 8 shows the setup of the metric view that reveals the information described. Code view. The code view illustrates the maintenance activities (LOC added, modified, and deleted) through color coding as mentioned above. To identify work phases, one can simply look for versions whose transitions leading up to them have ”‘a lot of green”’, i.e. major code additions. Blue and red marks illustrate rework phases as they express code modifications and deletions. 2) Scenario II: As the motto of information visualization is ”‘Discovering the unexpected”’ it is not surprising that we found several interesting patterns and anomalies in the datasets we analyzed. We first used the metric view to look
6
Fig. 5.
The most abstract image of the code view reveals the progress of code length and illustrates the maintenance activities.
Fig. 6.
The code representative view reveal details about the code structure, which allows the analysts to recognize method blocks etc.
Fig. 7.
The code text view displays the actual source code. This level of detail is often necessary to conduct a more thorough analysis.
7
Fig. 8. Phases of work and rework identified using the metric view. The bars that are positioned higher up on the y-axis show versions that experienced a large amount of code additions and are therefore marked as work. The unmarked versions are rework phases.
Fig. 9. Phases of work and rework identified using the code view. The transitions that are predominantly green signal a high amount of change and can be identified as work phases. The transitions with blue and red lines represent code modification and deletions, respectively. They show rework phases.
for version which had extreme change amount values, i.e. exceptionally low or high. We encountered that the subject had applied an exceptionally high amount of change to a number of consecutive versions (see Figure 10). After taking a closer look on the source code we discovered that the added code section remained unchanged for the rest of the development process. This is especially interesting since the programmer seemed to have struggled when adding rather small functionality at the beginning of the program development. This is just one example of the patterns we found when applying CodeVizard to the given dataset but it already shows what interesting patterns a quick analysis can reveal. While CodeVizard does not provide final answers, it provides strong support in giving clues as basis of reasoning and further sense making process.
user guide, a list of tasks and a debriefing questionnaire. The user guide commented on the information that is displayed in CodeVizard and explained the features that the user was supposed to use during the study. 3) Tasks: The main part of the study was a list of analysis tasks which the participant should conduct using CodeVizard. The task list consisted of two sections: ”‘Using CodeVizard”’ and ”‘Detecting Complex Patterns”’. The tasks in the first section asked the participants to identify source code section with certain properties and more detailed information about software changes. Examples include
IV. C ASE S TUDY
The second section asked the user to retrieve more complex pattern in from the dataset. The tasks in this section closely reassembled the questions HPCS researchers were trying to answer in their previous analysis without CodeVizard. The purpose was to confirm that the tool indeed supports the HPCS analysis activities. Tasks of the second section included:
In order to evaluate the efficiency of CodeVizard we conducted several case studies. The general goal of the studies were to determine whether CodeVizard can support analysts in finding the patterns of interest and quickly retrieve valuable information from the available data. The case studies were conducted with 6 participants, 3 of which were senior scientists and 3 were experienced graduate students. None of the participants took part in the tool development or was familiar with CodeVizard. The studies were conducted individually in the presence of the authors of this paper with the goal to give advice if needed and to observe how the participant uses the tool. Some of the participant also chose to speak out loud as they did the study, which gave additional insight. Each case study was conducted as follows. At the beginning of the study, the participant received a document with a brief
• •
•
•
What was the number of the version that took the longest to implement? The source code of which method(s) was/were changed?
In the previous task you identified the version(s) that had the most change leading up to it. Did the code added remain stable (based of the amount of rework that followed)? Identify these two main activity types: Work (Adding new features) and Rework (Modifying existing code in order to apply structural or functional changes).
4) Questionnaire: To conclude the study, the participants were asked to fill out a debriefing questionnaire that should capture the participants impression of the tool regarding the
8
Fig. 10. Scenario II - The metric view clearly shows the two versions that experiences major additions. Analyzing the code on a high level of abstraction reveals that the added code sections were added it remained unchanged.
handling, color scheme etc. It also contained questions pertaining the usefulness for conducting the type of analysis tasks as done for HPCS. Furthermore, we asked the participants to point out their most favorite features and functions the tool was lacking, i.e. tool improvements. A. Conducting the Study No time restrictions were imposed on the participants as to provide enough time to get familiar with the tool and produce valid results. In order to make the participant familiar with CodeVizard we presented its basic idea and its features at the very beginning. The user guide that was part of the documentation could be used as a reference during the study. B. Results In average it took the participants 37 minutes to complete the study, not including the introduction. All participants were able to finish all tasks. For the first task section ”‘Using CodeVizard”’ all participants answered the questions correctly. The results of the second task section ”‘Detecting Complex Patterns”’, however, varied. This was to be expected as the especially one of the questions required domain knowledge and interpretation. The challenge was to decide how ”‘work”’ and ”‘rework”’ phases are characterized. Most participants decided to define the versions as ”‘work”’ when there was a large number of code lines added. The way the participants detected the versions with this characteristic was by looking for ”‘a lot of green”’ in the code view. The participant that was most familiar with the code as he had analyzed it before took a slightly different approach. He worked on a more detailed level and used variable names and method names to determine when new features were added and when existing features were modified. During this more detailed analysis the participant also detected other interesting patters, such as
changes in programming strategies. Many of the sections were classified identically by all participants. The sections for which the classification differed were the ones that required a more detailed analysis. Using the code view in CodeVizard such an analysis was easily possible. The debriefing questionnaire was used to capture the participant’s impressions about the tool and document its strengths and weaknesses. The following results are derived from the answers to the questions in the questionnaire and also from post study conversations. The overall impression of CodeVizard was very positive. Among the features that the participants liked most were the ability of the tool to combine high level metrics with source code details and the easy handling. Some of the participants were particularly impressed that it was possible to view the entire development process with all its source code changes. The handling of the tool was perceived as comfortable and in average rated 4.5 on a scale from 1 to 5 with 1 being the least and 5 being the most comfortable. A weakness that was pointed out to us was that it was very easy to loose the focus while zooming the code view. This was in part due to the fact that zooming is done by right clicking on the part of interest and then dragging the mouse to the right. Also, there was not yet a minimum and maximum zoom factor that would prevent the image from disappearing. This issue is fixed in the most recent release. The color schemes used in CodeVizard were in average rated 4 on a scale from 1 to 5 with 1 being the least and 5 being the most helpful. This question addressed the coloring of different change types (lines added, modified, deleted) in the code view and the coloring used to visualize one of the metrics in the metric view. While the coloring in the code view remained the same for all participants we changed the color scheme of the metric view in order to determine which color scheme is perceived as most useful. The coloring of the bars in the metric view has the purpose of distinguishing the discrete
9
values and highlight extreme values. The two color schemes that were used were shadings of blue and a combination of green and red as shown in Figure X.
Fig. 11. The color schemes used for the case studies. The blue color scheme on the left and the green-red color scheme on the right.
The sequential blue color scheme was perceived by most participants as useful but some mentioned that the different shading of blue are difficult to distinguish. participants that used the green-red color scheme noted that they had difficulties to recognize the meaning of the faded colors, which represented the values toward the medium value. The critiques of the green-red color scheme were significantly stronger. We therefore conclude that the blue color scheme is more appropriate to illustrate the metric values. When asked for missing features that could improve the usefulness of the tool the participant listed a number of extensions that could be implemented in the future: • Couple metric view and code view more tightly. The code view did not focus on the version the user marked in the metric view. It was therefore confusing to find the highlighted version. Also, the participants suggested it would be useful to display a window in the metric view that shows the versions that are currently displayed in the code view. This is because the metric view was often used as a high level view for the entire development process while the code was used to conduct more detailed analysis. • Comparison feature. A frequent task in the HPCS analysis was the comparison of two files. The participants asked for a feature that can support such a comparison. • Annotation. Some comments suggested an annotation feature which could enable the analyst to capture information about the discoveries made while analyzing a code version with CodeVizard. • Export feature. One way to compare two files could be to simply export the data displayed in CodeVizard. • Handling multiple files. CodeVizard is currently limited to a single file. In order to analyze entire software systems it must support the analysis of multiple files. The case studies led to interesting findings pertaining the tool performance and usefulness. It also illuminated what extensions could be most beneficial for analyzing source code of the HPCS and other projects. It was also pointed out that the tool could be used in a more general context, e.g. to conduct change evolution analysis with slightly different goals as the ones of HPCS. V. R ELATED W ORK Software visualization is not an entirely new field and there has been some work in visualizing several aspects of software
[8]. Few approaches, however, address the issue of visualizing software evolution. Gevol is a system that visualizes the evolution of software using a certain graph drawing technique for visualization of large graphs with temporal component [9]. Gevol extracts information about a Java program stored within a CVS version control system and displays it using a temporal graph visualizer. While Gevol’s goal is mainly to visualize a large amount of data, it does not provide any low level information that is necessary for a thorough analysis. Xing et al. present a method for studying the evolution of object-oriented software at system/subsystem level and analyzing the underlying factors that drive its unfolding over time [10]. Their method relies on analyzing changes between two subsequent software versions represented as change trees. The main drawback of their work is that it is limited to only two version, which is often not sufficient when attempting to establish knowledge about the evolution of a system or a source code file. Seesoft is a comercial visualization tool that provides an abstract and high level view of the source code by mapping each line of code into a thin row [11]. The color of each row indicates a code characteristic. It also provides a separate window that illustrates the actual source code. While combining an abstract with an detailed code view provides some notion of semantic zooming, the software is limited to display only a few metrics. It is also not desirable to visualize all metrics in code lines as some measures can only apply to entire files, such as coupling and such. Seesoft does not provide a higher level metric view that allows through a quick glimpse to detect interesting pattern but rather requires to study the coloring of the source code lines more carefully. VI. C ONCLUSION AND F UTURE W ORK Many approaches have been developed to support the analysis of software evolution, yet there are still numerous challenges researchers face when attempting to evaluate how a software system evolved over time. At the same time the field of information visualization has attracted more and more attention as researchers and practitioners recognize the power of visualizing data in new ways. The work described in this paper aimed to tackle some of the concrete issues researchers encountered when analyzing software evolutionary data by using techniques from the field of information visualization as a means of presenting the right amount of data, at the right amount of detail at a right point in the analysis process. We developed a tool called CodeVizard, which uses the principles of abstraction and detail to support the analysis of the change evolutionary process for a single source code file. Software metrics are used to provide high level information and different representations of source code allows to evaluate the software versions on a more detailed level. CodeVizard consists of two main parts: the metric view and the code view. The metric view shows a bar graph that illustrates several software metrics, which can be used to illuminate various characteristics of the development process.
10
The combination of 4 dimensions in bar graph allows to visualize multiple metrics at a time and show how the metrics relate to each other. The code view illustrates the source code of all code version lined up with transitions describing the maintenance activities that led to a version. It highlights the code lines that were added, modified, and deleted. Visual combined with semantic zooming allows the analyst to plot a high level view that shows all code version on the screen and also an intermediate and a low level view that shows the actual source code. This allows for a detailed analysis. The interaction of these two views make CodeVizard a powerful tool for the analysis of software evolution. We evaluated CodeVizard by conducting a series of case studies in which we asked colleagues to conduct a number of first simple and then more complex analysis tasks. We observed them while executing these tasks in order to document how analysts would use CodeVizard in these situations. Furthermore, we handed our colleagues a list of questions to record their perception of the tool and determine how CodeVizard could be improved. The results show that the tool was well perceived and that especially the participants that conducted the same kind of analysis without CodeVizard appreciated its concepts and handling. The case study also revealed some features CodeVizard was lacking in its current version but which could be useful for analyzing software evolution in the HPCS setting. Requested functionality include the ability to handle multiple files rather than a single one, a comparison functionality for multiple code files, and an annotation feature that could be used to capture findings and mark detected patterns. Our goal for future work is to continue improving CodeVizard based on these points. ACKNOWLEDGMENT The authors would like to thank Forrest Shull, Mikael Lindvall, Patricia Costa, Taiga Nakamura, Rola Alameh, and Nabi Zamani for their time and effort during the case studies and and we would like to express our appreciation for their thoughtful insight, which will help us to improve on CodeVizard. R EFERENCES [1] J. C. Munson and S. G. Elbaum, “Code churn: A measure for estimating the impact of code change,” in ICSM ’98: Proceedings of the International Conference on Software Maintenance. Washington, DC, USA: IEEE Computer Society, 1998, p. 24. [Online]. Available: citeseer.ist.psu.edu/346505.html [2] T. Nakamura, “Challenges in measuring hpcs learner productivity in an age of ubiquitous computing: The hpcs program,” in Proceedings of the first International Workshop on Software Engineering for High Performance Computing System Applications, 2004, pp. 27–31. [Online]. Available: http://care.cs.umd.edu:8080/hpcs/resources/papers/spaa06.pdf/view [3] E. E. Mills, “Software metrics,” Seattle University, Tech. Rep., 1988. [4] L. W. MacDonald, “Tutorial: Using color effectively in computer graphics,” IEEE Comput. Graph. Appl., vol. 19, no. 4, pp. 20–35, 1999. [5] J. Lin, E. Keogh, and S. Lonardi, “Visualizing and discovering nontrivial patterns in large time series databases,” Information Visualization, vol. 4, no. 2, pp. 61–82, 2005. [6] J. Seo and B. Shneiderman, “A rank-by-feature framework for interactive exploration of multidimensional data,” Information Visualization, vol. 4, no. 2, pp. 96–113, 2005. [7] V. I. Levenshtein, “Binary codes capable of correcting deletions, insertions, and reversals,” Tech. Rep. 8, 1966.
[8] J. T. Stasko and J. B. Domingue, Software Visualization. Cambridge, Massachusetts: The MIT Press, 1998. [9] J. N. J. P. Christian Collberg, Stephen Kobourov and K. Wampler, “A system for graph-based visualization of the evolution of software,” in SoftVis ’03: Proceedings of the 2003 ACM symposium on Software visualization. New York, NY, USA: ACM Press, 2003, pp. 77–ff. [10] Z. Xing and E. Stroulia, “Understanding phases and styles of objectoriented systems’ evolution,” in ICSM ’04: Proceedings of the 20th IEEE International Conference on Software Maintenance. Washington, DC, USA: IEEE Computer Society, 2004, pp. 242–251. [11] S. G. Eick, J. L. Steffen, and J. Eric E. Sumner, “Seesoft-a tool for visualizing line oriented software statistics,” IEEE Trans. Softw. Eng., vol. 18, no. 11, pp. 957–968, 1992.
