An Interactive VBA Tool for Teaching Statistical Process Control (SPC ...

10 downloads 0 Views 2MB Size Report
VBA Tool for teaching SPC and process management issues. Students can experiment with the tool to inter- actively examine the various issues that affect SPC ...
Vol. 5, No. 3, May 2005, pp. 19–32 issn 1532-0545  05  0503  0019

informs

®

doi 10.1287/ited.5.3.19 © 2005 INFORMS

I N F O R M S Transactions on Education

An Interactive VBA Tool for Teaching Statistical Process Control (SPC) and Process Management Issues Jaydeep Balakrishnan, Sherry L. Oh

Operations Management Area, Haskayne School of Business, University of Calgary, Calgary, Alberta T2N 1N4, Canada {[email protected], [email protected]}

W

ith a global emphasis on improved quality, Statistical Process Control (SPC) is an important process management tool with renewed significance. In order to address this issue, we have developed an interactive VBA Tool for teaching SPC and process management issues. Students can experiment with the tool to interactively examine the various issues that affect SPC and gain insight into the important issues in managing a process. The graphical nature of the interface should allow students to visually see the effect of changes in process parameters. A detailed Instructor’s Manual and a Student Lab Manual accompany the software.

1.

Introduction

2.

Rationale

importance. Many current business philosophies make specific use of SPC. For example, Just-In-Time (JIT) concepts have become pervasive in business worldwide. JIT espouses SPC as a form of defect prevention. Further, SPC as well as process capability are crucial to the concept of “Six Sigma” quality programs at global corporations such as GE and Motorola. SPC is also seen as integral to the Total Quality Management (TQM) philosophy and the ISO quality standards. Increasingly, managers must think in terms of processes rather than functions, and a thorough understanding of SPC will be valuable for future managers of processes. At the same time, in our experience, SPC is also one of the more difficult concepts for students to comprehend. The theory behind SPC is based on probability and statistics, topics many business students are not known to excel in, or have great interest in. Thus, concepts such as process control limits and process capability, often taught back-to-back, are frequently confusing for many students. Similarly, explaining the effect of different z values on errors and the interpretation of sample statistics is difficult without simulating actual process measures. While the issues can be discussed with classical visual aids such as a board or overhead, an interactive computer tool teaches the desired concepts more effectively and quickly. The SPC tool first evolved from an Excel-based demonstration of how to create statistical process control charts to a VBA-enabled spreadsheet model representing situations of Type I/II errors, and then to

With a renewed emphasis on managing processes in Operations Management (OM), discussion of Statistical Process Control (SPC) is often included as an integral part of the OM course. Also, what was once thought of as a statistical tool used mainly for production control in manufacturing has now achieved mainstream status in an increasing number of Fortune-500 companies (including service-based). To assist in the teaching of SPC and business process management, we have developed an Excel VBA tool that can be used in class. The following concepts are explored through guided use of the teaching model: 1. False out-of-control and false in-control indications 2. The role of reduced variability on improved process control 3. Process capability 4. The role of reduced variation in ensuring better process capability 5. Six Sigma 6. Understanding the role and differences between control and specification limits 7. Information from control charts As we discuss in the next section, this VBA tool can be more than an SPC teaching tool, it can also help demonstrate process management principles.

With a global emphasis on improved quality, SPC has become a process management tool with renewed 19

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues

20

INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

its current format demonstrating both process control and capability concepts. The choice of using VBA was natural as the mathematical calculations are handled easily and it allows modifications to be made quickly by anyone with access to Excel on their computer. The graphical demonstration could have been attained with a package such as Macromedia FLASH, but the interactive nature of the tool would have been more difficult to achieve as so much of the graphics are based on user inputs and subsequent calculations. Many statistical packages, such as Minitab, have the ability to produce a wide array of control charts quickly and easily. While the mechanics of creating a control chart are straight forward, it is important for students to recognize that implementing SPC requires the understanding of the theory behind control charts and the practical implications of that theory. We believe that an interactive visual tool will be much more effective in understanding this concept that relies on a knowledge of probability theory. This tool can also be used to link theory to managerial practice.

3.

Previous Work

While interactive exercises in quality control have existed for some time (see Heineke and Meile 1995, for examples), the use of computer software in teaching is becoming more popular as students’ access to software and hardware increases. Many OM textbooks now include a CD ROM with software that can be used to solve problems in a variety of topics and often with graphical and simulation capabilities. Excel-based software is one of the more popular formats because of the ease of use and the availability of Excel. Of interest to this paper are the previous interactive software programs that have appeared in the fields of statistics, decision analysis, and operations management as SPC and process improvement fall under these fields of study. The topic of SPC is generally dealt with in Operations Management, Decision Analysis, and Business Statistics textbooks. SPC is not usually a topic covered in general-purpose (not business-oriented) statistics textbooks. While textbooks in these three areas cover the basics of preparing and interpreting control charts, only a few (one example is Krajewski and Ritzman 2002) explain aspects such as the relationship of variance and control limit spread to Type I and Type II errors and other managerial issues. Operations Management textbooks also generally explain process capability through the index formula; however, they do not relate the notion of process capability back to managing the process through process control. In general, the software available with textbooks in the three categories is limited to basic plotting of process control charts.

Advanced quality management and control textbooks often deal with the more sophisticated concepts described above, but these textbooks tend not to include any interactive software. One text that did include a CD-ROM (Summers 2003) had a limited version of a commercial product with excellent graphics capabilities which allows students to generate various reports. However, it is not well-suited to in-class, hands-on demonstration. For example, it does not generate data to demonstrate in-control or outof control points (the data set has to be created), nor does it create two overlapping distributions in case of a mean shift to illustrate Type II error or the magnitude of the shift. Thus, the focus is not on the basic principles of SPC and its managerial implications but rather, on control chart preparation and interpretation. Another advanced quality management textbook by Evans and Lindsay (2002) includes only Excel based charts to plot control data. Statistics textbooks are also available in online or CD-ROM format. Visual Statistics 2.0 (Doane et al. 2001) is an example. Again, the focus in this text is on control chart plotting and interpretation rather than on the illustration of basic principles such the interrelationships between variation, control limit spread and errors, and process improvement. The student-run website, http://www.freequality. org/, has a variety of useful software programs and quality tools aimed at professionals looking for assistance in solving quality and process management problems. These programs would not be effective in the interactive demonstration of SPC and process management. A search of the Journal of Statistics Education did not reveal any article that described software that does what we propose to do in this article. Further, a survey paper by Mills (2002) in this journal categorized the computer simulation methods used to teach statistics. They included areas such as the central limit theorem, t-distribution, and confidence intervals but none on SPC. Pappas et al. (1982) describe a computer-based SPC teaching tool developed and used in 1980 with their 3rd year mechanical engineering students. They illustrate a teaching tool called PEPEVO that generates process attributes from a normal distribution as well as either gradual or sudden shifts in the process mean. The student can then apply different process chart “rules” that give an out-of-control indication, such as one point outside the limit or seven successive points on one side on the mean and so on. The user also has the ability to specify other parameters such as the sampling frequency and sample size. The objective is to find the testing procedure that minimizes the cost of the system, including those of sampling, stopping,

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

and resetting the machine and the undetected defectives, for the set process parameters. The focus of PEPEVO is on the design of the sampling system and its implications. Its output includes rudimentary graphics in the form of control charts, but it does not report nearly the same level of interaction as our software. PEPEVO’s design is modular and it requires students have the ability to modify the code in order to create new testing procedures and specify processes parameters before running the simulation to view outputs. The authors suggest that its optimal use requires approximately 12 hours of classroom time. The level of technical skill and time required to implement PEPEVO would preclude this tool from use in most introductory Operations Management courses. In summary, it appears that while software-based SPC teaching tools exist, the focus has generally been on plotting and interpreting control charts. Our focus is unique in that we aim to link control chart issues with their managerial implications. Through this tool, we hope students will attain a better understanding of issues such as: • “How does increasing the z-value impact the frequency of unnecessary process stoppages or the amount of time out-of-control processes remain undetected?” • “How does focusing on training employees and better equipment improve the control of the process?” • “What is the difference between a control limit and a specification limit?” We believe that the advantage of our software is that these and other SPC managerial issues can be investigated using a single, interactive teaching tool that is simple enough for students to use on their own, or as a guided classroom activity.

4.

Learning

An effective teaching tool will help transform business questions into a theoretical framework and then link that theory back into practice. For example, consider the question: Why does the Type II error decrease with greater shifts in the process mean? With the click of a button using this VBA tool, students can see that the overlap between the distributions decreases, thus understanding why there is a decreased probability that a sample reading from one distribution will be mistakenly assumed to be from the other. This is the theory, how do we link it to practice? Going back a step, one might ask, “why is discussing the value of the shift in the mean important?” Consider a call centre manager who wants to control the mean time that a customer is put on hold. For such a manager, small shifts in the mean and the associated high Type II error may be inconsequential. However,

21

a large shift in the mean implies that customer service has dropped considerably and thus, the magnitude of the Type II error becomes an important consideration when making process control decisions. Similarly, the tool can be used to emphasize the managerial tradeoffs that exist in process control and process capability. Graphically, is it is easy to see that increasing the sample size results in less distribution spread. Thus the overlap between the sampling distributions is decreased and detecting shifts in the process becomes easier. This strategy however would increase sampling time and costs, and would have no beneficial effect on process capability. Alternatively, a reduction in process variance will also increase the ability to detect shifts in the process while at the same time, improve process capability. Through manipulation of process variables, students will appreciate the positive managerial implications of reduced variance and recognize for themselves how it may be a less expensive long-term option than increased sampling. In practice, reduced variance implies more consistency in the process (process improvement) achieved through employee training, less system breakdowns, fewer mistakes that need to be fixed, advanced technology and the like. This links the theory to practice. Within the JIT context, you have reduced waste and variance. Consider our call centre manager who wants to ensure that no customer waits more than a certain time. With a graphical tool presenting the process distribution and the waiting time specification limit, it can easily be demonstrated that for a given mean, higher variance/process inconsistency increases the probability of violating this waiting time limit. To ensure that performance goals are met, the process manager could lower the mean waiting time by hiring extra staff, resulting in waste through lower staff utilization. The graphics can also be used to discuss a better option, namely the concept of Six Sigma and how a Six Sigma process is less wasteful and less vulnerable to small changes in process mean. Thus, we believe that in addition to being a simple quality control teaching tool, the interactive VBA tool also allows the instructor to discuss more general process management principles. Managing processes and their improvement straddles different functional areas such as human resources (employee training), information systems (technology), and accounting (audit). Given that this tool can foster a discussion of process management issues over a wide spectrum of functional areas, its greatest contribution may be made with a more mature audience. We see it as being most effective at the MBA level (it has been tried at the Executive MBA level with good results) or in an elective class on quality management. At the introductory undergraduate level, students may find this

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues

22

INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

a useful tool for them to grasp the more abstract concepts of process distribution versus distribution of sample means, Type I versus Type II error, and process capability. This tool can be used after the concepts of variation and control limits are introduced. An exercise to demonstrate the concept of variation (the coin-andtube exercise) can be found in Heineke and Meile (1995). The proposed hands-on activity also assumes a basic understanding of probability distributions and sampling. The interface of the VBA tool invites hands-on exploration of the concepts; its use would be best suited to a lab or classroom with enough computers so that the students can experiment along with the instructor.

5.

Using the Tool

Exhibit 1 is a screen print of what the user sees upon opening the VBA spreadsheet. The software allows the user to specify a situation requiring process control. The example shown is from a call centre where the average time a customer is put on hold is measured, but any process parameter may be used by changing the value in the process parameter text box. The screen allows the user to specify two sets of process parameters (both normally distributed for simplicity) through the  and . The software graphs the distributions automatically. The process, as it was designed and set up, is called the Planned Process. The second distribution, called the Current Process, represents the process as it is actually working. If the process is working as planned (in control), the Planned Process and the Current Process are shown to be identical (same  and ). The TAB key is used to move between controls as is the ENTER key. Exhibit 2 is a screen print of the Process Control Sheet and Exhibit 3 is used for Process Capability issues. The title in Exhibit 1 and Exhibit 2 will state “Distribution of Individual Observations” or “Distribution of Sample Means” depending on whether the sample size is 1 or greater. Exhibit 1

Process Parameter Sheet

Exhibit 4

Annotated Student Lab Manual

Set n = 1, z = 1. Click the Generate Sample button. This generates a sample and calculates the sample mean. Since the sample size is 1, the mean is just the individual value. This value is shown on the bottom right of the chart and it is represented as a green dot if within the UCL and LCL and as a red square if outside the UCL and LCL. (Note A sample size of one would usually not be used for SPC in practice, but required for the purpose of this demonstration). Click it a few more times until you get a red square. In practice, if you were managing this process what would you do when you get a red square? You would stop the process to investigate the cause of a sample mean outside the UCL/LCL. Does the call on hold violate company guidelines (defective)? Probably not    a red square is displayed if the generated value exceeds 11.5, while it would need to exceed 15 minutes in order to be “defective”

The tool actually consists of three parts: this document for the instructor, a Student Lab Manual for students to work through, and the software. The instructor document explains the software and the concepts that can be taught through its use. It is expected that the instructor would be projecting the software image on a screen in the classroom. The Student Lab Manual (Exhibit 4 shows a part of the annotated instructor version) is a detailed step-bystep series of exercises that the students can follow through with the instructor, with space for them to take notes about the results of each exercise and class discussion. The manual and the software may be posted on websites so that students can download and work through them outside the classroom, if necessary. The instructor version of Student Lab Manual has also been prepared where the space for student notes has been filled in with expected results and suggested discussion (shown in bold in Exhibit 4). Since all the documents accompanying the software are in MS Word, they can easily be modified should the instructor choose another example (rather than Exhibit 2

Process Control Sheet

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

Exhibit 3

Process Specification (Capability) Sheet

the call centre) to discuss the process management concepts. 5.1. Process Control Issues After clicking on the “control” tab, the “z” dropdown listbox in Exhibit 2 is used to set z to 1, 2, or 3. The Lower Control Limit (LCL) and Upper Control Limit (UCL) vertical lines are automatically generated and shown in orange. Sample size n may be specified in the given textbox and defaults to 1 if contents are non-numeric or non-positive. If the n is 1, the software uses “Distribution of Individual Observations” as the screen title whereas if n is greater that 1, the title “Distribution of Sample Means” is used. This serves to emphasize that SPC generally uses the sampling distribution. The “Generate Sample from Current Process” button is used to generate a sample mean from the current process. If the value generated (displayed on bottom right of chart) is within the LCL and UCL, a green dot is correctly placed along the horizontal axis, indicating an in-control reading. If the value generated is outside these limits, a red square is placed, indicating an out-of-control reading. The total number of values generated and the number of these values outside the limits are tallied in the “total” and “out” textboxes respectively. By clicking the “Generate Sample from Current Process” button and then keeping the Enter key pressed down, up to one hundred samples can be generated. This process can also be rapidly simulated by pressing the “Generate 100 Samples” button. The “Reset” button clears tallies to re-start the sampling process. 5.1.1. False Out-of-Control Indication (a or Type I Error). Assuming that the process is in control, a false out-of-control indication occurs when the sample mean falls outside of the control limits (indicated by the red square). It can be seen that this will occur about 33% of the time when z = 1. Changing z to 2 and then 3 will clearly show that we can reduce

23

the Type I error by increasing z. Students can be asked to repeat the process with various values of n to confirm that larger samples will not help to lower the false out-of-control indication. Thus, adjusting z is the only way to reduce the frequency of Type I errors. 5.1.2. False In-Control Indication (b or Type II Error). A false in-control indication occurs when the Current Process is no longer the same as the Planned Process, but the sample indicates that the process is still working as planned. To demonstrate this, set n = 10, z = 3 and make a minor change in  for the Current Process. With the process now out-of-control, the sampling would correctly indicate this with a red square, representing the mean, falling outside of the control limits. It can be seen that since the Planned and Current distributions overlap considerably, most readings will fall within the control limits, resulting in green dots (now a false in-control indicator). In fact, with z = 3, running a simulation of 100 samples will show that the error rate is almost 100%. The next step would be to discuss how this error may be controlled. The VBA teaching tool can demonstrate it in three ways: 1. Reduce z. This will lower the probability of false in-control. However this will also increase the probability of a false out-of-control (Type I error). 2. Increase sample size, n. Since increased n reduces sample variance, students will note the narrowing of the distributions and the resulting decrease in the overlap between the Current and Planned distributions. The control limits will also narrow as a result and the probability of a false in-control indication will decrease. Since n has no effect on false out-ofcontrol (Type I error), this is a good option. However, it comes at the cost of increased sampling and as mentioned previously, has no benefit with respect to process capability. 3. Only be concerned about larger shifts in the mean. By making the Current Process mean significantly differently from the Planned Process mean, it can be shown that the probability of a false in-control depends on how much out-of-control the process actually is. A combined strategy using the latter two approaches can also be presented—determine how much of a shift (given the particular example) is enough to cause concern and then choose a sufficiently large sample to reduce the probability of a false in-control reading to an acceptable level. What if the resulting n is too large? The next section answers this. 5.1.3. Reduced Process Variability. The role of reduced variability in improved process control can be demonstrated using the same settings described in the previous section. With the process out-ofcontrol, the  can be reduced for both the Planned and Current Process, preferably with an explanation

24

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues

about achieving lower  through better training, more advanced technology, better management, and the like. If one used the coin-and-tube game previously mentioned, one could relate back to this by asking the students to think about how variability could have been reduced, perhaps by using a narrower tube or placing the tube much closer to the target. Students will observe that the two resulting distributions will overlap less and as a result, the probability of the false in-control decreases. Thus, a reduction in variability has the same positive effect on Type II error without the ongoing cost of more sizeable sampling. It is probably during the discussion of the effects of z on the probability of a Type I or Type II error that the instructor can help students avoid a common misconception. We have often found that when students are asked during exams about the choice of z, they often erroneously state “use z = 3 instead of z = 2 because it implies better quality control since more sample means will fall within the limits.” It is our hope that through hands-on experimentation, most students will avoid this mistake. 5.2. Process Capability The issue of process capability can be explained by using the Specification sheet (Exhibit 3) where the control limits are not shown and n defaults to 1, because process capability is by definition based on individual units. From a customer’s perspective, it is the possibility of receiving a defective good or service. The , , and specification limits can be set by entering appropriate values on the process parameter and specification sheets. While generating units of the product or service, a green dot would indicate a good unit, whereas a red square would denote a defective unit. Initially, , the Upper Specification Limit (USL) and the Lower Specification Limit (LSL) can be set such that process capability is low, for example where 15% of the units are defective. By simulating 100 units or observations, students will observe a large proportion of the units will fall outside the USL/LSL. Such a process with high defectives is not generally viable in practice. Students will also be able to graphically see the negative effect of shifts in the process mean on process capability. As seen in the Process Specification sheet (Exhibit 3), the distribution displayed can be set to the Current Process using the “Planned Process/Current Process” toggle button. For a given USL/LSL, a shift in the Current Process mean results in the distribution graphically moving closer to the USL or LSL and more units falling outside one of the specification limits. One method of reducing the defect rate would be to move the LSL/USL farther away from the mean. Students will quickly point out that this does not

INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

really solve the problem as it implies lower quality standards. Assuming that the customer is not willing to accept lower quality, how can the situation be improved? 5.2.1. Reduced Process Variability, Revisited. The effect of reduced variability on the defect rate can be demonstrated by going to the process parameter sheet and reducing  so that the specification limits are close to 3 standard deviations away from the process mean. A few 100-unit simulations will show that less than 1% of units will be defective since almost the entire distribution falls within the USL/LSL. It should be noted that this solution is superior; specification limits have not been changed (implying the same level of quality). Also in many cases, the customer or regulatory bodies determine specifications limits and thus these cannot be allowed to deteriorate. By reducing the  further to make the USL/LSL ±6 away from the mean, the concept of Six Sigma can be explained by showing that the USL −  = 6. One could run a few 100-unit simulations to show that there is virtually no chance of a defective product (even with minor shifts in the mean). It might be useful to point out two aspects of Six Sigma as practiced by organizations. The term “Six Sigma” as originally coined by Motorola has a slightly different statistical interpretation than 6 as it allows for 15 shifts in the mean before calculating the probability of an error, which results in 3.4 defects per million. Secondly, it is important to emphasize that Six Sigma in organizations is a process improvement philosophy of which statistical methods are just one aspect (Chase et al. 2004 is an introductory Operations Management text has a detailed section on Six Sigma methodology). To quote former GE CEO Jack Welch (2001), “The big myth is that Six Sigma is about quality control and statistics. It is that—but its much more. Ultimately, it drives leadership to be better by providing tools to think through tough issues.” 5.3.

Differentiating Between USL/LSL and UCL/LCL This tool also helps to differentiate between the USL/LSL and UCL/LCL, which is often confusing to students. On the control limits screen, if the n is set to 1, the distribution shown is for individual observations and the option to show the USL/LSL (in blue) on the same graph becomes available. Setting n > 1 results in the removal of the specification limits, demonstrating that they are not relevant when sample averages are plotted. A discussion of the differences between the two types of limits (control versus specification) as well as the effect of customer requirements on USL/LSL would be appropriate at this time. To demonstrate the differences, start with a process that is in control (Current Process  and  to be the

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

same as that of the Planned Process) set z = 3, n = 1, and on the specification screen, adjust the specification limits so that they are approximately 1 from the mean. The instructor should point out that a sample size of one would NOT typically be used for SPC in practice, but is required for the purpose of this demonstration only. Returning to the control limits screen and selecting “show specification limits,” one can demonstrate how an individual measure can fall within control limits, but still not meet specifications (is defective) since the UCL is outside the USL. Naturally, this situation is not desirable in practice, as it will result in defective products left undetected. A highly desirable system is one in which the SPC mechanism detects an out of control process long before defectives, as designated by the specification limits, are produced. How can this achieved? This is a good place to reemphasize the attractiveness of Six Sigma. Through process improvement, assume that the  has been reduced enough such that the USL is 6 away from the target. When the “show specification limits” is selected in the Control sheet, the USL will fall outside the UCL, which is desirable. To show why this is desirable, shift the current process mean slightly away from the planned process mean. When samples n = 1 are generated, indications to stop and correct the process (red squares) should occur before actual defective products are observed, i.e., a warning system. Under a Six Sigma Quality system, it is uncommon for moderate shifts in the mean to result in defective product, and furthermore, these shifts should trigger the SPC mechanism to detect and initiate measures to correct the process. It can also be demonstrated that control limits are automatically adjusted when one or more of z, n, , or  are changed, while specification limits are externally set and thus are not affected by the process parameters. Students should be asked how the specification limits would be set—in our call centre example, it may be set by management beliefs based on customer preferences. Students will then be able to more clearly see differences between the two types of limits and how each are quite independent of one another. In summary, the UCL/LCL is used to manage the process while the USL/LSL is used to determine whether the product is defective or not—two separate issues.

6.

Conclusions

We have presented an Excel VBA tool that can be used to enhance students’ understanding of managerial issues in Statistical Process Control (SPC), process variability and Six Sigma. We believe that its ease of use, combined with its graphical interface, make it an effective classroom teaching tool. The development of the software and manual has been iterative over the past few years and has been

25

based on student and instructor feedback. Colleagues who have used all or part of the SPC tool have felt that it was effective in improving instruction. Acknowledgments

The authors wish to thank their colleague Janice Bodnarchuk Eliasson, Tom Grossman of the University of San Francisco, the anonymous reviewers, and the Associate Editor for their valuable comments on this article. Financial support was provided by the Natural Sciences and Engineering Research Council (NSERC) of Canada.

References Chase, R. B., F. R. Jacobs, N. J. Aquilano. 2004. Operations Management for Competitive Advantage. McGraw Hill, New York. Doane, D., K. Mathieson, R. Tracy. 2001. Visual Statistics 2.0. McGraw Hill, New York. Evans, J. R., W. M. Lindsay. 2002. The Management and Control of Quality. Southwestern, Cincinnati. Heineke, J. H., L. C. Meile. 1995. Games and Exercise for Operations Management. Prentice Hall, Englewood Cliffs, NJ. Krajewski, L. J., L. P. Ritzman. 2002. Operations Management: Strategy and Analysis. Prentice Hall, Upper Saddle River, NJ. Mills, J. D. 2002. Using computer simulation methods to teach statistics. J. Statist. Ed. 10(1). Pappas, I. A., K. Maniatopoulos, S. Protosigelos, A. Vakalapoulos. 1982. A tool for teaching Monte-Carlo simulation without really meaning it. Eur. J. Oper. Res. 11(2) 217–211. Summers, D. C. S. 2003. Quality. Prentice Hall, Upper Saddle River, NJ. Welch, J. 2001. Straight from the Gut. Warner Business Books, New York, 330.

LAB MANUAL http://archive.ite. journal.informs.org/Vol5No3/ BalakrishnanOh/lab_manual_nov05_ instr.doc An Interactive VBA Tool for Teaching Statistical Process Control (SPC) Issues

The following are the concepts covered in this exercise: 1. Process capability. 2. The role of reduced variation in ensuring better process capability. 3. Upper Specification Limit (USL) and Lower Specification Limit (LSL) and differentiating these from Upper Control Limit (UCL) and Lower Control Limit (LCL). 4. False out of control and false in control indications. 5. The role of z in the Type I and II error. 6. The role of sample size in Type I and II error. 7. The role of reduced variability on improved process control.

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues

26

INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

8. Six Sigma. 9. Managerial trade-offs. 10. Detecting process shifts. Note to Instructors: The lab manual contains the correct responses to various questions asked of the students as they follow the exercise. The responses are formatted using “hidden text” that can be displayed (or not) when printed by using the appropriate setting under: Tools ⇒ Options ⇒ Print ⇒ Include with document [x] Hidden Text. The hidden text can be viewed on the screen by: Tools ⇒ Options ⇒ View ⇒ Formatting marks [x] Hidden Text.

1.

Software Set-Up http://archive.ite. journal.informs.org/Vol5No3/ BalakrishnanOh/ControlLimitDemo _v1_nov05.xls

Exhibit 1 is a screen print of what the user sees upon opening the VBA spreadsheet. The example shown is from a call center where the average time a customer is put on hold is measured, though this can be changed to suit the situation in the Process sheet, (Exhibit 1). The Process sheet screen also allows one to specify two processes (both normally distributed for simplicity) through the  and . The software graphs the distributions automatically. The process, as it was designed and set up, is called the Planned Process. The second distribution, called the Current Process, represents the process as it is actually working. If the process is working as planned (in control) the Planned Process and the Current Process are shown to be identical (same  and ). The TAB key is used to move between buttons and as an ENTER key. Notice that you have There are two other sheets you can use: Control and Spec (Specification). Exhibit 2 shows the Control sheet where you can specify the SPC parameters such as the z value (1, 2, or 3), and Exhibit 1

VBA Sheet (Process)

Exhibit 2

Process Control Sheet

sample size n. The UCL/LCL will always be shown. If n = 1, the chart will plot “individual” observations as indicated in the title. If n > 1, then sample averages are plotted. The sample mean will be displayed at the bottom right of the screen, as shown in Exhibit 2. You can superimpose the USL/LSL in the Control sheet by selecting the Show Specification Limit button, but ONLY if the sample size n is set to 1. Exhibit 3 shows the Spec sheet. In the Spec sheet you can specify (by entering it directly or by using the increase/decrease buttons) the specification limit up to +/− 50%. This represents the maximum acceptable deviation from the planned process mean. You can also toggle the specification limits to be either one or two sided, as well as determine whether the actual current or planned process is displayed by clicking on the toggle buttons below the Simulation Stats display. In the situation of a call center, time on hold would generally have just a single-sided USL. In our example, the Generate Single Observation button generates a normally distributed “call on hold time” and a corresponding marker along the x-axis of the graph. If the marker is a green circle, it means that that the caller was on hold for a time less than the USL. If it Exhibit 3

Specification Sheet

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues

27

INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

is a red square, it indicates that the caller was put on hold for more than the USL, i.e. a violation of the call center guidelines;—a defective service! In the Spec sheet, the Simulation Stats display gives the number of total calls and the number of defective calls (in red). In the Control sheet, the Simulation Stat buttons gives the number of total samples taken and the number of sample means (in red) outside the LCL and UCL. On either sheet, you also have the option of generating 100 samples or observations to see the statistics displayed instantly. The title in Exhibit 1 and Exhibit 2 will state “Distribution of Individual Observations” or “Distribution of Sample Means” depending on whether the sample size is 1 or greater. Note: 1. Hit Tab or Enter after any changes in values to ensure that it is recognized 2. After finishing a sampling experiment, use the Simulation Stat Reset button before proceeding.

2.

Process Capability

Consider the following example. You are in charge of a call center for the Internet division of a major bank. Historically, the mean time ( that a caller has been placed on hold is 10 minutes and the standard deviation ( has been 5 minutes. Top management dictates that all calls have to be fielded within 15 minutes, (i.e., cannot be put on hold for longer that that). This is your USL. Your call center would be considered “capable” if there was very little chance of a call exceeding this time. Now consider whether the following cases are capable: Case I Is the call center currently capable? To answer this, in the Process sheet set the Planned Process  to 10 minutes and  to 5. Select the Process in Control button to ensure that the process is in control. Only one normal distribution will be seen. As the variability is fairly high relative to the mean, the distribution will be “truncated” on the left-hand side, and any simulated calls with negative call times will be counted as having a wait time equal to zero. In the Spec sheet, increase the specification limit to +/− 50%. For a  of 10, it means that the USL is 50% greater (15 minutes). Since management considers only long waits as defective, click the two-sided toggle button to switch to a one-sided test. This will make it correspond to the call center example. Now click the Generate Single Observation button to generate a normally distributed “call on hold time” along the x-axis of the graph. If the marker is a green dot, it means that that the caller was on hold for a time less than the USL of 15 minutes. If it is a red square, it indicates that the caller was put on hold for more

than the USL of 15 minutes, i.e. a violation of the call center guidelines—a defective service! Click Run 100 iterations to generate 100 calls. Based on the results displayed in the Simulation Stat box, how many calls took more than 15 minutes to answer? Approximately 16% This can also be calculated mathematically using the properties of the normal distribution. For a given  and , the z value corresponding to the probability that a value greater than X will be generated is given by Equation (1). X −  (1) z=  Therefore, if X = 15,  = 10 minutes, and  = 5 minutes, what is the value of z? z=

X −  15 − 10 = =1  5

z=1

Given this z, using a normal distribution table, what is the probability that X > 15? Pr(X > 1.0) = 1 − 08413 @ 16%. This value should be approximately the same as that obtained through simulating the 100 calls. Is this likely acceptable (i.e. is the call centre capable)? No, with 16% of calls exceeding the USL, probably not. The value of z indicates how many ’s the specification limit (maximum allowed wait) is from the process mean, . We refer to this as the  level of the process and it is shown near the USL on the chart displayed with the Spec sheet (Exhibit 3). The higher the  level, the less likely that we will get an observation outside the limit. What is the  level of this process? Case II—Reduced Variability Assume that the mean “on hold” time is still 10 minutes, but by installing more user friendly software and better employee training, the variation in the time required to help customers is reduced. Thus the on hold standard deviation  has also been reduced. The new  is 1.5 minutes. With these improvements, the process is now more consistent, but is the process now more capable? To answer this, set the  of the Planned Process in the Process sheet to 1.5 minutes and switching to the Spec sheet, observe the USL. Is there any change in the area of the curve that is above the specification limit compared to Case I? (You may need to switch back to  = 5 to verify) What does this imply for the likelihood of a defective product? When less of the process distribution curve is above the USL it implies that there is a lower probability of defectives.

28

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

Tables of the Normal Distribution

Click Run 100 iterations in the Spec sheet to generate 100 calls. Based on the results displayed in the Simulation Stat box, how many calls took more than 15 minutes to answer? Close to 0%. Calculate the probability of defects mathematically using Equation (1) and the Normal Distribution table given below. What is the  level? Is the process more capable?

Yes. There is a significant portion of the distribution curve above 15 minutes. Click Run 100 iterations to generate 100 calls from the Current process distribution. What is the defective rate? Close to 10%. Calculate the probability of defects mathematically using Equation (1). What is the  level? Is the process capable?

Case III—Shift in the Mean For some reason, the mean “on hold” time has recently risen to 13 minutes. The  is still 1.5 minutes. Is the process still capable? To answer this, in the Process sheet, unselect the Process in Control button and change  of Current Process to 13. This means that the process has shifted and people on average are put on hold for 13 minutes (though as a manager, you may not detect this until you take a sample). Observe in the Spec sheet the new distribution with  = 13. Make sure that you are viewing the “current” process by clicking on the process toggle button so that it displays the current processWould you expect more defects when compared to Case II? Why?

Case IV You have instituted process improvement measures such that the mean “on hold” time is back to 10 minutes and the  has been reduced to 0.8 minutes. Is the planned process capable? To answer this, in the Current Process in the Process sheet, change the  back to 10 and  to 0.8, and select the Process in Control button. This implies that the process is very consistent. Click Process in Control and return to the Spec Sheet to view the planned process distribution. Would you expect more or less defects when compared to Cases I, II, and III? Why? Fewer. There is hardly any portion of the distribution above 15 minutes.

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

Click Run 100 iterations to generate 100 calls from the Planned process distribution. What is the defective rate? Should be ∼0%. Calculate the probability of defects mathematically using Equation (1). What is the  level? Is the process capable? z=

X −  15 − 10 = = 625  08

 level = Pr(X > 625 < 9866 × 10−10 so YES, the process is very capable. What does 6 imply for process accuracy and capability? 6 implies that the process is very accurate and i.e, most people are put on hold for about 10 minutes with very little variation. Thus hardly, hardly, hardly anybody will be on hold for more that 15 minutes. Students can be asked to change the mean to 11 minutes and examine the defective rate. It will still be zero, thus showing that with a Six Sigma process, slight shifts in the mean will not cause defectives. Note: 6 is similar in concept but not the same as “Six Sigma.” The term Six Sigma as originally coined by Motorola has a slightly different statistical interpretation as it allows for 1.5 shifts in the mean before calculating the probability of an error which results in 3.4 defects per million. Secondly, it is important to emphasize that Six Sigma in organizations is a process improvement philosophy of which statistical methods are just one aspect. To quote former GE CEO Jack Welch (2001), “The big myth is that Six Sigma is about quality control and statistics. It is that—but its much more. Ultimately, it drives leadership to be better by providing tools to think through tough issues.”

3.

Difference Between USL/LSL and UCL/LCL

In the Process sheet, set  = 10 minutes and  = 15 minutes for the Planned Process and select the

29

Process in Control button. This means that process is working as it should. In the Spec sheet, ensure that USL/LSL is at 50%. In the Control sheet, set z to 1, sample size n = 1 and select the Show Specification Limits button. Note that the USL and LSL (in blue) are seen and are different from the UCL and LCL (in orange). The software is calculating the UCL/LCL from the SPC formulae for variables (shown below in Equations (2) and (3)) while the USL is company policy.    (2) UCL =  + z √ n    LCL =  − z √ (3) n Click z = 2. What happens to UCL/LCL in terms of spread? As z increases, the control limits widen. What happens to USL/LSL? Nothing, the specifications limits do not change. What is the significance? The UCL/LCL can be controlled by z while USL/LSL is externally set (company policy, government safety, or consumer regulations etc). Set z back to 1 and increase sample size n to 5. What happens to the spread of UCL/LCL as the sample size increases? The spread between LCL & UCL narrows. What happens to the specification USL/LSL? Why? The specification limits disappear and the “show specification limits” button becomes grey because process capability is based only on individual values. It is also externally set (company policy, government safety, or consumer regulations etc). So just as in the case of z, the UCL/LCL can be controlled by sample size. Set n = 1, z = 1. Click the Generate Sample button. This generates a sample and calculates the sample mean. It also plots this mean along the x-axis of the

30

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues

graph. It is represented as a green dot if within the UCL and LCL and as a red square if outside the UCL and LCL. The calculated mean value is shown on the bottom right of the chart (Exhibit 2). Since the sample size is 1, the mean is just the individual value. (Note A sample size of one would usually not be used for SPC in practice, but required for the purpose of this demonstration). Click it a few more times till you get a red square. In practice, if you were managing this process what would you do when you get a red square? You would stop the process to investigate the cause of a sample mean outside the UCL/LCL. Does the call on hold violate company guidelines if the red square was above the UCL? In other words is the generated on hold time a defect? Probably not    a red square is displayed if the generated value exceeds 11.5, while it would need to exceed 15 minutes in order to be “defective” Now, in the Control sheet, set z = 3, n = 1. In the Spec sheet, adjust the specification limits so that they are approximately 1 from the mean (corresponding to 16% in the Spec Limits). After returning to the Control sheet, click the Generate Sample button till you get a green dot that is above the USL but below the UCL. What does this indicate? Is this a desirable situation? An individual reading can fall within control limits, but still not meet specifications (is defective) since the UCL is outside the USL. Naturally, this is a situation that is NOT desirable in practice as it will result in defectives being undetected. On the other hand, it is desirable is to have a system where even when the process is out of control, defectives will not be produced before the SPC mechanism detects it. To illustrate the desired system, in the Process sheet, unselect the Process in Control button, set  of the Planned Process to 10 minutes and of the Current Process to 11 minutes. Set  of both processes to 0.6 minute. In the Spec sheet, change USL to 50%. In the Spec Sheet, what  level is this? 8.3 In the Control sheet, select the Show Specification Limits button. Click the Generate Sample button until you get a red square. You would naturally stop the process to investigate. How many clicks did you need to get a red square? Is sample defective? The UCL is 11.8, 3 away from the planned process mean. The current process mean is 11, only 1.33 away from the calculated UCL. This corresponds to the situation in case III, where approximately 9% of the samples indicate an “out of control” process and on average, 11 “clicks” will be required to generate a red square. In this case though, the sample is NOT defective as samples fall well below the USL.

INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

What is the probability that calls will be defective? Practically 0 as it is a 6 (similar to a Six Sigma) process. Why is this a desirable situation? The USL is much farther out than the UCL. When the process mean has shifted to 11 minutes, your SPC limits will detect it soon. Better yet, hardly any call violates the 15 minute limit. So the process will be fixed when needed and no defectives will be produced. This shows the attractiveness of 6 (similar to Six Sigma)—rarely producing a defect while helping the SPC mechanism detect shifts in the mean. How does this demonstrate differences between Control and Specification Limits?

4.

Managerial Issues in Process Control

The Effect of z on Type I /Type II Error A Type I error occurs when the process is in control but the sample indicates that the process is out of control. A Type II error occurs when the sample indicates that the process is still in control while it fact it is no longer in control. For this example, we want to start with a process that is operating as planned. In the Process sheet, set  = 10 minutes and  = 5 minutes for the Planned Process and select the Process in Control box. In practice, SPC uses the statistics from samples (n > 1) gathered to evaluated the process and in this example, we will assume a sample size of four calls. In the Control sheet, set z = 1, n = 4 and unselect the Show Specification Limits, if necessary. Remember that the process is working as it should and the correct managerial decision is to NOT stop the process. When samples are generated, green dots are the correct indicators while the red squares incorrectly indicate that the process is out of control. You can try this out by generating a few samples using the Generate Sample button. Now generate 100 samples. The number in red in the Simulation Stats box indicates the number of times the sample mean would have fallen outside the LCL/UCL. Each time this happens we would have stopped the process in ERROR (Type I error) What is the approximate error rate (Type I) and is it acceptable? 33%. Naturally, this would not be acceptable because you would frequently investigate problems that did not exist. The cost is wasted time and resources. Now set z = 2 and generate 100 samples. Note the Type I error compared to z = 1. Set z = 3 and generate another 100 samples. What is the effect of z on Type I error?

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues

31

INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

When z = 2, approximately 5% of the values will fall outside the control limits. When z = 3, no (or perhaps 1) Type I errors occur. Larger z values reduce the probability of a Type I error. Next we examine the Type II error. In the Process sheet, unselect the Process in Control button and change  of the Current Process to 11. Note that the process is now out of control and now you have two distributions: one is the planned process and the other the current, shifted process. Assume that we do not know it has shifted and will be relying on samples to decide whether it is in or out of control. This is the situation that process managers face when determining whether or not the process is in control. You can try this out by generating a few samples using the Generate Sample button. In the Control sheet, with z = 3 and n = 4, generate 100 samples. Note that now a red square is the correct indicator, i.e, we should be stopping the process. The green dot is actually a Type II error. How often (%) does a Type II error occur? Usually 100%. Repeat for z = 2 and z=1. Compare the Type II errors. Which is better? Why? The error rate is much higher for z = 3, so smaller z values are better from a Type II error perspective. Since the distributions overlap very much, z = 3 covers both distributions and all points fall within the LCL/UCL, thus incorrectly indicating that the process is in control. With z = 1, many points fall outside as it covers less of the distribution, correctly indicating a shift in the mean. What is the dilemma that you observe with respect to Type I and Type II error and the value of z? Increasing z to improve Type I error will result in a higher Type II error and vice versa. In other words z = 3 DOES NOT mean that our quality control is better than when z = 1. Given that many companies choose to use z=3, there are generally two different methods to reduce the possibility of a Type II error and ensure the accuracy of the monitoring: Method A: Use a larger sample size by changing n = 100. Set z = 3 and generate 100 samples. What happens to the sampling distribution? The distribution spread decreases. Has the error probability decreased? How do larger samples help? Yes. Larger samples will reduce the spread in the sampling distributions as shown in the Control sheet. As a result, when the process mean shifts, there is less overlap between the two distributions (planned vs. current). With less overlap, there is a higher probability that the samples from the shifted distribution will fall outside the limits based on the planned process, reducing the probability of a Type II error.

What is the disadvantage of larger samples? Larger samples will cost more money to implement, especially if destructive testing is required. So what is another approach? Method B: Set n back to 4. In the Process sheet, for both processes set  = 05 minute. This indicates less variability through better equipment, training of employees and so on. In the Control sheet, generate 100 samples. What is the Type II error probability? What is the overlap between distributions? Low error rate (about 13 to 20%) and low overlap between distributions How do both of these methods help to reduce Type II error? As with larger samples, lower process variance reduces the spread in the sampling distributions. When the process mean shifts, there is less overlap between the two distributions, resulting in a higher probability that the samples from the shifted distribution will fall outside the LCL/UCL. This CORRECTLY indicates an out of control process and results in management stopping the process and taking corrective action. Both of these methods to reduce Type II error will cost money—which is better? Although reducing variability costs money initially, over the long term it is better to have a more accurate process than to spend more money indefinitely with larger samples to achieve the same low error rate. No matter how accurate the process, there is always a trade-off between Type I and Type II error. The issue of how low the error rates should be is a managerial decision based on the costs of allowing the error to continue versus the costs to detect it. Other Information from Control Charts In the Process sheet, set  of the Planned Process to 10 minutes, set  of the Current Process to 11 minutes,  of both to 1.5 minutes. In the Control sheet, set n = 4, and z = 3. Reset the Simulation Statistics and generate twelve samples from the current process. After generating each sample, roughly plot the sample mean (as displayed in the bottom right corner of graph in Exhibit 2) in Exhibit 4 below. Are the plotted points randomly scattered above and below the planned process mean of 10? No, they are predominantly above the mean What does this imply for managers looking for an indication of an out of control process? Exhibit 4

UCL 10 LCL

Mean of planned process

32

Balakrishnan and Oh: An Interactive VBA Tool for Teaching SPC and Process Management Issues

It will be seen that most of the values are above the mean indicating non-randomness and that we should investigate. This implies that we need not wait for a red square (sample mean above UCL); non-random green dots are a sign that the process may not be in control. The Magnitude of the Shift Ensure that I i n the Process sheet, set  of the Planned Process to is 10 minutes, set  of the Current Process to is 11 minutes, and set  of both to processes is 1.5 minutes. In the Control sheet, ensure that set n = 4, and z = 3, and now generate 100 samples. Recall that since the process is out of control, a red square is the correct indicator. What is the resulting probability of a Type II error? High In the Process sheet change  of the Current Process to 15. In the Control sheet generate 100 samples. What is the Type II error compared to the previous case? Close to zero

INFORMS Transactions on Education 5(3), pp. 19–32, © 2005 INFORMS

What does this imply for managers that are concerned about type Type II error? For the same n and z, the larger the shift, the more likely you are to catch it. If a shift of the process mean from 10 to 11 is not important to you (customers may not notice), then you may not need to increase sample size or variance to reduce Type II error since the effort and cost may not be worth the benefit. You may decide that a shift of 5 is very important to detect (customers may start screaming!). In that case, you will want to spend money increasing the sample size or reducing variance so that you have a high probability of detecting the shift. Once again, it is a managerial decision as to whether or not resources need to be deployed to reduce the probability of a Type II error for small shifts in the process. It may be adequate to allocate resources only for large shifts in process mean. Managers will need to understand the cost of process control versus the cost of not detecting process shifts.