Session T2D TEACHING DESIGN OF EXPERIMENTS ... - CiteSeerX

Session T2D TEACHING DESIGN OF EXPERIMENTS AND STATISTICAL ANALYSIS OF DATA THROUGH LABORATORY EXPERIMENTS Stacy Gleixner1 , Greg Young 2 , Linda Vanasupa3 , Yasser Dessouky4 , Emily Allen5 and David Parent 6 Abstract  A new laboratory course at San Jose State University, Advanced Thin Film Processes, integrates fabrication of thin films with design of experiment and statistical analysis of data. In the laboratory section of this course, students work through six multi-week modules that increase in the complexity of design of experiment and statistical analysis of data. The six modules have been developed with a standard format that includes learning objectives, background on the specific thin film process, theory of design of experiment principles, instructor notes, dry lab exercises, experimental plan worksheets, and assessment tools. While the modules were developed for a semiconductor processing class, they can easily be implemented in other engineering classes. The modules have been developed with a robust framework that allows the instructor to teach design of experiments and statistical analysis of data along with the specific engineering principles related to their class. Index Terms  teaching design of experiment, semiconductor processing laboratory class, statistics experiments, DOE

OVERVIEW OF M ICROELECTRONICS PROCESS ENGINEERING PROGRAM San Jose State University has initiated an innovative undergraduate degree in microelectronics process engineering (µProE). The program evolved from conversations with industry partners about their demand for microelectronics process engineers. This demand was seen in a range of industries where the term microelectronics processing is meant to generalize anyone using manufacturing processes similar to those of the semiconductor industry. These include traditional semiconductor companies that manufacture computer chips as well as companies that package microelectronic products; semiconductor equipment manufacturers; data storage companies; flat panel and other display manufacturers; chemical and pharmaceutical companies using micro-fluidic

devices; and industries making micro-electrical mechanical devices such as sensors and actuators. Process engineers are those that maintain, monitor, evaluate, and optimize the manufacturing processes in these industries. The skill set needed to successfully perform these job functions requires a broad base of engineering fundamentals. Companies usually generate successful process engineers by hiring graduates from a range of engineering majors and training them in house for the needed process engineering skills not covered in their major. Our program was developed in order to give engineering students the fundamentals needed to be effective process engineers immediately upon graduation. The program combines existing classes in chemical, materials, electrical, and industrial and systems engineering along with three new laboratory classes in semiconductor processing. Figure 1 shows the engineering curriculum for the µProE degree. A detailed description of the development of the program and curriculum is published elsewhere [1].

MatE

E.O.M. Props

Materials Structures

EE

Materials Characterization

Device Physics

µProE

ISE

ChemE

Intro to IC Proc. & Design

Engr. Stat.

Thermo

Adv. Thin Film Process.

Micro. Mfgr. Methods

DOE

SPC

Transport

Reactor Kinetics

FIGURE. 1 CURRICULUM FOR A BACHELOR’ S DEGREE IN MICROELECTRONICSPROCESS ENGINEERING

1

Stacy Gleixner, Dept. of Chemical and Materials Engineering, San Jose State University, One Washington Square, San Jose, CA 95192-0082, [email protected] 2 Greg Young, Dept. of Chemical and Materials Engineering, San Jose State University, One Washington Square, San Jose, CA 95192-0082, [email protected] 3 Linda Vanasupa, Materials Engineering Department, California Polytechnic State University, San Luis Obispo, CA 93407, [email protected] 4 Yasser Dessouky, Industrial and Systems Engineering, San Jose State University, One Washington Square, San Jose, CA 95192-0085, [email protected] 5 Emily Allen, Dept. of Chemical and Materials Engineering, San Jose State University, One Washington Sq uare, San Jose, CA 95192-0082, [email protected] 6 David Parent, Electrical Engineering, San Jose State University, One Washington Square, San Jose, CA 95192-0084, [email protected]

0-7803-7444-4/02/$17.00 © 2002 IEEE November 6 - 9, 2002, Boston, MA 32 n d ASEE/IEEE Frontiers in Education Conference T2D-1

Session T2D Figure 1 emphasizes the interdisciplinary nature of the µProE engineering education. The faculty involved in the development of the curriculum and the microelectronics processing lab are also interdisciplinary: coming from the chemical, materials, electrical, and industrial and systems engineering departments. Part-time instructors from industry have also contributed to course development. The three µProE classes shown in Figure 1 were added to the engineering curriculum to specifically target the education gaps cited by our industry partners. They include extensive theory and laboratory work related to the chemistry and physics needed to understand the manufacturing of microelectronics. The classes also build skills in teamwork, communication, and the use of statistics in manufacturing. They are all hands-on laboratory classes that are typically team taught by faculty from different disciplines. In addition to µProE majors, the classes are taken as electives by electrical, chemical, materials, mechanical, industrial, and computer engineering majors. The first class in the series is Introduction to Integrated Circuits Processing and Design (MatE/ EE 129). This class operates as a fictitious company where students rotate between being on manufacturing and research teams [2]. In this class, students learn a basic overview of microelectronics processing through coursework and the hands-on manufacturing of NMOS transistors. Teamwork and communication skills are taught and extensively utilized in this class. Feedback from our industry partners emphasizes that these skills are critical to process engineers who work with a range of other engineers and technicians every day. Microelectronics Manufacturing Methods (MatE/EE 167) teaches the advanced microelectronics theory needed by process engineers including limitations on current materials and devices and the use of modeling in process optimization. There is a strong emphasis on the skills needed to monitor and optimize the manufacturing process, specifically statistical process control (SPC) [3]. Students in MatE/EE 167 have already taken MatE/EE 129. Like MatE/EE 129, students alternate between manufacturing and research teams in the lab. This class produces CMOS circuits and extensively utilizes SPC to track and optimize their manufacturing process. Students in MatE/ChE 166: Advanced Thin Film Processes have already taken MatE/EE 129. They have a basic understanding of the microelectronics process. This class goes into depth on the materials and chemical engineering principles needed to understand the deposition, patterning, and etching of thin films. Deposition and etching of thin films are the most utilized step in the overall manufacturing process and one of the most critical. The laboratory sections of the class place a strong emphasis on using design of experiments (DOE) and statistical analysis of data. The statistics and DOE laboratory modules developed for this course are discussed in detail in this paper.

STATISTICS AND DOE EXPERIMENTS Statistics and DOE were cited as skills our industry collaborators felt were weak in engineering graduates they were currently hiring. The employers also felt these skills were critical to a process engineer’s success. A report generated by the Semiconductor Research Corporation on Microelectronics Manufacturing Education listed statistics and DOE as two of the top skills needed for new college graduates to be hired as process engineers [4]. There are frequently stand alone classes on statistics and design of experiments (which our µProE majors take to add depth to their skills). However, integrating the statistics and DOE directly into a laboratory class stemmed from a desire for the students to gain a more hands-on understanding of statistics. Six lab modules were designed to provide learning experiences in a range of basic topics in DOE and statistical analysis of data. Table I lists the modules and the schedule for a fifteen week semester. The laboratory experiments are open ended, where teams of students design their own experiments (to answer given questions). The modules increase in complexity of both the DOE and statistics covered as well as the experimental planning required of the students. The first week of each lab module is reserved as a dry lab session. (Module 1 is solely a dry lab exercise.) During this week, the fundamentals of the DOE or statistical analysis of data needed for that module are taught. Students gain mastery of these skills by working through given dry lab exercises (numerical problems related to microelectronics process engineering). Then as a team they design an experimental plan to be carried out in the following weeks. The plan should be based on a specific, assigned question and generate the needed data to be analyzed according to that mo dule’s DOE and statistics principles. TABLE I DOE AND STATISTICAL ANALYSIS OF DATA MODULES USED IN THE LABORATORY CLASS

Lab Module Module 1: Introduction to Statistics Module 2: Determining the Precision of a Measurement Technique Module 3: Determining the Variation Within a Process Module 4: Design of a Single Factor Experiment & Analysis of Variance Module 5: ANOVA of a Single Factor Experiment With Blocking Module 6: Design of a Two-Level Full Factorial Experiment

# of Weeks 1 2 3 3 3 3


Session T2D M ODULE FORMAT A standard format was developed for each module. The goal of this was to create a full laboratory class that could easily be taught by other faculty at San Jose State or exported to other universities. It is also expected that some or all of the modules will be useful in integrating DOE and statistical analysis of data into a range of engineering laboratory classes. Each module consists of: • Laboratory Handout- This is a comprehensive handout given to the students explaining the lab theory and experiment. More details on this are given below. • Dry-lab Tutorial- This is a PowerPoint presentation the faculty can utilize during the first week of the lab module to teach the needed DOE and statistical analysis of data. It contains numerical examples designed specifically for microelectronics process engineering. • Dry-lab Exercises- These are numerical problems specific to microelectronics process engineering that were designed to give the students practice in the DOE and statistical analysis of data principles specific to that module. • Experimental Plan Worksheet- Students fill out a detailed plan for their experiment during the first week of the module (the dry lab week). Prior to beginning in the lab, the students review their experimental plan with the instructor. Aspects such as proper control of variables, use of a statistically valid number of samples, and time management to accomplish the entire experiment in the allotted lab sections are emphasized. • Student Feedback Survey- Each lab module includes a student feedback survey to assess whether the students felt the lab taught them the desired principles and if they view those principles as useful to manufacturing. For each laboratory module, there is a detailed lab handout that is given to the students. The handout includes: • Learning Objectives- The handout provides the students and faculty with a detailed list of learning objectives for that module. The list utilizes Bloom’s taxonomy and represents skills that can be assessed in the lab’s outcomes. They include technical skills on the theory and laboratory equipment as well as mastery of the DOE and statistical analysis of data principles. Table II lists the learning objectives for Module 4: Design of a Single Factor Experiment and Analysis of Variance as an example. • Microelectronics Theory- Any information needed to complement the lectures and textbook in order for the students to fully understand the microelectronics process being studied is presented in this section. • Statistics Theory- This section contains a detailed description of the statistics and DOE. This includes explanations of the equations as well as numerical examples.

•

•

Experimental Goals- Depending on the module, student teams are assigned lab equipment or specific questions to investigate. This section guides them in the factors they need to consider and control when designing their experiments. Lab Outcome- This section details the outcome expected from the laboratory experiment. This is typically a formal written laboratory report or an engineering memo. The content required in the reports is specified. The required data analysis is also clearly detailed. TABLE II

EXAMPLE OF LEARNING OBJECTIVES USED IN MODULE 4: DESIGN OF A SINGLE FACTOR EXPERIMENT & ANALYSIS OF VARIANCE .

# 1 2 3

4

5 6 7 8 9

Learning Objectives Write clear objectives and statement of problem for an experiment. Identify controllable and uncontrollable factors in an experimental set-up. Choose a factor for a single factor ANOVA based on expected outcome and given time and equipment constraints. Determine the appropriate levels to be researched for a factor based on expected outcome, equipment control, metrology precision, and time constraints. Design an experiment using proper replication, randomization, and control of variables. Calculate the variation between levels using a sum of squares method. Calculate the variation using an F-test. Plot data of all the levels to show variation between levels. Organize technical information into a clear and concise formal laboratory report.

M ODULE CONTENT Module 1: Introduction to Statistics Module 1: Introduction to Statistics is solely a one week dry lab exercise. There is a tutorial session in which the students learn (or review) mean, median, mode, standard deviation, and variance. A one sample T-test (to determine if the mean of a process is within standard or given specification) is taught. Students are introduced to the concept of a P value (the smallest level of significance that would lead to the rejection of a hypothesis) and how to use this value to determine if their results are statistically significant. The lab handout details the needed equations along with numerical examples. The students then perform a dry lab experiment where they statistically analyze data from oxidation of silicon wafers. (he students are all familiar with this data and process from their Introduction to IC Processing class.


Session T2D Module 2: Determining the Precision of a Measurement Technique

students statistically analyze their results to determine the variability within a run and from run to run.

Module 2: Determining the Precision of a Measurement Technique is a variation of a laboratory experiment developed by Vanasupa et al. [5]. The difference between precision and accuracy are taught. Students are then given the skills to statistically determine whether a piece of equipment is precis e. This includes simple calculations of the precision (3σ). Students also use an F-test and a paired T-test to compare the variance and mean (respectively) of two processes. In the dry lab portion of the exercise, students practice analyzing experimental results using these DOE principles. The microelectronics process engineering problem they analyze is related to quantifying the difference between two measurement techniques for oxide thickness. For the experiment portion of the lab, the student teams are assigned two pieces of equipment that both measure the same property. We use equipment that measures oxide or metal thickness. The students develop an experiment to determine the precision of each piece of equipment. They develop standard operating procedures to attempt to improve the precision and repeatability of the tool. The students statistically compare the precision of the two tools. This lab is critically important in that it introduces to students that errors in their experiments come not only from the processing but also (and sometimes most significantly) from the measurement tool. It also emphasizes the importance of repeatability and having a standard operating procedure. Students begin to build the statistical skills needed to determine whether the differences between two sets of measurements are statistically significant. This corrects the notion that a lot of students have that if the measurement readings are not exactly the same, one (or both) of them are wrong.

Module 4: Design of a Single Factor Experiment and Analysis of Variance In Module 4: Design of a Single Factor Experiment and Analysis of Variance, students are taught the skills needed to properly design a complex experiment. The dry lab exercise steps them through the process of designing a robust experiment, see Table III [6]. Analysis of variance (ANOVA) is taught so the students have the statistical tools to compare multiple levels of a variable and determine if changing the variable has a statistical impact on the outcome. In the dry lab experiment, the students apply the ANOVA when analyzing the results of photoresist thickness following soft bakes that vary in time. They are able to determine whether time is a statistically significant variable in the experiment. Students then design a single factor (one variable) experiment. They are assigned a microelectronics process and asked to determine if a chosen variable has an impact on the outcome. Students show final mastery of the concept by utilizing ANOVA in their final report to prove whether their variable was statistically significant. TABLE III STEPS USED TO P ROPERLY DESIGN AN EXPERIMENT [6]

# 1 2 3

4 5

Module 3: Determining the Variation Within a Process In Module 3: Determining the Variation Within a Process, students begin to use the statistics and design of experiments they have been learning in the other modules to evaluate results from complex sample sets, that is multiple lots (runs) that each contain multiple samples (wafers). In the dry-lab portion of the exercise, students review the F-test and how it can be applied to multiple runs and samples. The two sample T-test is taught as a way of comparing the mean between two different lots. The students gain mastery of these principles by statistically analyzing given results of varying photoresist thickness (multiple runs and multiple wafers each run). Students then plan and carry out an experiment to statistically evaluate the repeatability of a process. Each team is assigned a microelectronic process (oxidation, photoresist spinning, metal evaporation, plasma etching...). They design a process flow through the equipment and repeat that for two sets of wafers. In their final report,

6 7 8 9

DOE Steps Develop a problem statement. Determine all the variables that impact the process. Determine which are controllable. Choose a factor to investigate (one that may impact the problem in step 1). Choose levels of the factor that are significant (theorized to be measurably different). Control all the other variables. Develop an experimental order that randomizes the sample runs and provides for replication. Develop a plan to carefully monitor your experiment. Determine the steps needed to statistically analyze the results. Review the experimental plan with the instructor. Carry out the experiment and analyze the results.s Module 5: ANOVA of a Single Factor Experiment With Blocking

The DOE skills learned in Module 4 are expanded on in Module 5: ANOVA of a Single Factor Experiment With Blocking. In the previous lab modules, it was emphasized to the students to hold all process parameters other than the one(s) being studied as constant (controls). In a real manufacturing setting, this is not always possible. Uncontrollable changes can happen such as a material lot changes or experiments are assigned to a different (but supposedly equivalent) tool. Blocking is a way of grouping and statistically analyzing the data after such a switch occurs.


Session T2D In the dry lab exercise, students learn and utilize advanced ANOVA calculations that require blocking. They practice those skills in analyzing given photolithography data (resolution of the lithography) for variations in lithography process parameters as well as runs that varied the photoresist supply. They then generate a detailed DOE for an experiment of their own on an assigned microelectronics process. Blocks are purposefully introduced, such as assigning different anneals to different furnaces or using different sources of the raw materials. The students utilize the complex ANOVA calculations to determine if the factors they were studying (the variables that they purposefully changed) as well as the blocks had a statistical impact on the results.

CONCLUSIONS Six laboratory modules were developed for a core course in the microelectronics process engineering program (MatE/ChE 166: Advanced Thin Film Processes). The overall goal of this course is to teach the fundamentals of thin film processing alongside design of experiment and statistical analysis of data. While designed around microelectronics process engineering, the modules could be used to integrate DOE and statistical analysis of data into a range of engineering laboratory classes. The modules were developed using a standard framework that will make them portable to other classes and universities. Contact the authors for a copy of the modules.

ACKNOWLEDGMENT Module 6: Design of a Two-Level Full Factorial Experiment In Module 6: Design of a Two-Level Full Factorial Experiment, a more realistic scenario of a complex engineering experiment is presented, one where there are multiple factors that may affect the outcome and they should be investigated simultaneously because they may also influence each other. In the dry-lab exercises, the calculations for 2k ANOVA are taught. These show how to mathematically compare results when you vary two different factors (such as time, temperature, pressure) and each factor is tested at “k” different levels (such as high, medium, and low). The students practice their ANOVA skills by statistically comparing the results of plasma etching when the power and gas pressure are changed. Using the steps in Table III, students design a complex experiment for an assigned microelectronics process. Their experiment investigates whether two variables in the process have an impact on the final thin film produced. During the following two weeks, they carry out the experiment. Their mastery of 2k DOE is assessed when they use the ANOVA to analyze the results of their experiment.

M ODULE ASSESSMENT The classes in the µProE program were designed around a comprehensive set of program learning objectives. Each class has their own set of learning objectives (on the order of 100 specific learning objectives) that target specific program objectives. Details of the extensive plan for assessment of the program and class learning objectives has been published elsewhere [7]. Assessment of each laboratory is also built into the module. This is done through evaluation of the student performance on the dry-labs, the development of their experimental plans, and the management of their experiment in lab. Student mastery of the DOE and statistical analysis of data is assessed in how the students communicate the results in their laboratory reports. Assessment in the form of student feedback is also collected through student surveys at the end of each lab module.

The curriculum development for these modules was accomplished with the support of the National Science Foundation (DUE#9952707). The laboratory has been enhanced by a Challenge Grant from the Society of Manufacturing Engineers, as well as financial and in-kind support from Intel Corporation, Applied Materials, Novellus Systems, Cypress Semiconductor, Advanced MicroDevices, Project Consultants and Quintel Corporation. Much of the work of the laboratory development was done by Irene Wibowo as part of her Master’s project in Industrial and Systems Engineering. Neil Peters, the Microelectronics Processing Lab Engineer, did an extensive amount of work setting up and running the laboratory sessions.

REFERENCES [1]

Allen, E., Gleixner, S., Young, G., Parent, D., Dessouky, Y. and Vanasupa, L., "Microelectronics Process Engineering: A NonTraditional Approach to MS&E," Procs. Materials Research Society, 684E, 2001, online GG5.1.

[2]

Muscat, A.J., Allen, E.L., Green, E.D.H and Vanasupa, L.S., “Interdisciplinary Teaching and Learning in a Semiconductor Processing Course,” J. Eng. Educ. 87, 1998, 413.

[3]

Parent, D., Dessouky, Y., Gleixner, S., Young, G. and Allen, E., “Microelectronics Process Engineering Program at SJSU”, Biennial University/Government/Industry Microelectronics Symposium Proceedings, 2001, 14th Biennial University/Government/Industry Microelectronics Symposium, 2001, 128.

[4]

SRC Competitiveness Foundation Workshop on Microelectronics Manufacturing Engineering Education, Semiconductor Research Corporation, Research Triangle Park, NC, 1991.

[5]

Vanasupa, L., Smith, H., Gleixner, S.H., Young, G., Allen, E.L., “Dealing with Variations in Measurements and Processes: Experiments for an Undergraduate Laboratory”, Procs. Materials Research Society, 684E, 2001, online GG5.8.

[6]

Montgomery, D.C., Design and Analysis of Experiments, John Wiley & Sons, New York, NY, 1997.

[7]

Young, G., Gleixner, S., Parent, D., Dessouky, Y., Allen, E. and Vanasupa, L., “Course Assessment of the Microelectronics Process Engineering Program at SJSU”, to be published in the American Society for Engineering Education Annual Conference Proceedings, 2002.
