Simulation: a research method

Simulation: a research method Jack P.C. Kleijnen Tilburg University, Tilburg, the Netherlands SIKS (Dutch Research "School for Information and Knowledge Systems") Course on "Research methods and methodology" Doorn (Netherlands), 27 November 2009

Overview What is simulation? Why simulation? Real-life applications of simulation Research challenges in simulation

2

What is simulation? 1. Examples 2. Definition Sub 1: (Details: next slides) a. Single-server queue system (e.g., M/M/1) Extensions: Servers in parallel (e.g., M/M/n) Servers in sequence (e.g., supply chain) Loops/feedbacks (e.g., after tests) Priorities (FIFO, LIFO, SJF, …, MRP, Kanban) b. (s, Q) inventory management system Extensions: Supply chains & networks c. Case studies: my own consulting projects 3

Single-server queue Waiting time w of “job” i + 1 (service and arrival times s & a): w(i + 1) = w(i) + s(i) – a(i + 1) if positive; else 0 M/M/1 (M: Markov / Poisson / exponential): s(i) = -ln r(2i – 1)] / μ with i = 1, 2, … E(s) = 1/μ & pseudo-random number 0 < r < 1 a(i + 1) = -ln r(2i) / λ Start with “empty” system: w(1) = 0 Stop after (say) 100,000 simulated jobs: i = 100,000 Conclusion: DEDS: Discrete Event Dynamic Systems (other type: nonlinear difference equations) Start & stop states of system Inputs: μ, λ (plus “seed” r(0)) 4

(s, Q) Inventory control system I(t) Reorder level s s Order Q if inventory I(t) < s Order delivered at t + L (with L: lead time) I(t) = I(t) – D with Demand D If I(t) < 0: demand lost or backordered?

t

Extensions: 1. Multiple articles (SKUs) "Joint" ordering saves some costs 2. Supply chain: produce instead of buy (next slide) 5

Supply chain: Ericsson (a)

Test

Circuit Board Manufacturing

Test

SMD and Vision Test

Test

Wave Soldering

Function Test Time Test

Frame Soldering Assembly

Final Test

Assembly

See Kleijnen, Bettonvil, Persson (2006): 92 inputs Which are important? Optimize (important) "control" variables Robust optimization: (important) "noise" variables

6

Agent-based simulation • ”Agents”: small computer modules of autonomous decision-makers • Naval Postgraduate School, Monterrey, CA (Susan Sanchez & Master students): military tactics, hardware, etc. • Netherlands: Han La Poutré (TUE), Catholijn Jonker (TUD), Valentin Robu (TUE) • Former Ph.D. student (Jürgen van der Pol): logistics 7

Simulation: definition Simulation: experimentation with computer model of dynamic system Consequences: • Experiment requires statistical methods for design & analysis: DOE, DACE, DASE • Validation of the simulation model: t-test, DOE & regression analysis Note: Experiment with real system • •

Expensive: time & money (few scenarios tested) Dangerous (bankrupt; nuclear accident; etc.)

8

Simulation: goals • • • •

Sensitivity Analysis: what-if? Insight! Optimization of simulated (real) system Verification of analytical methods Example: Queuing approximations Training: “serious” gaming (managers) Also see flight simulator (pilots)

9

Simulation: applications Any scientific discipline studying dynamic systems: from sociology to astronomy My consulting projects (details: next slides): • ROC: logistics of a school (students, teachers, PCs) • Sandia (Albuquerque): disposal of nuclear medical waste • Ministry : social security laws • Gasunie: investment analysis of gas transport on Java • TNO/FEL: sonar search for (explosives) mines (besides consulting, also Master thesis) • VBF: DSS for production planning (also Ph.D., etc.) • RIVM: global warming (also Master & Ph.D. theses) • PTT: telephone network ("grading") capacity • ECT: quay length & crane numbers • IBM: cost-benefit analysis of information systems 10

ROC: logistics of a school Logistics: “balance” between outputs (criteria) - utilization of resources (student, teacher, PC, room) - throughput time (“service” to student) Controllable inputs: - Number & type of resources - Priority rules (SPT, slack time, etc.) Goals of ROC-simulation: • Quantify Critical Performance Criteria (“CPI”) • Identify bottleneck resource • Create insight: evaluate priority rules & scenario’s (resources, student input, etc.) “Flexible learning” = “job shop”? See next slide

11

Job shop: example (Source: Arena textbook) Bron: Arena programmatuur

12

Wageningen: milk robot Ilan Halachmi (2002): optimal number of robots Now: Israel (8 robots, 500 cows)

13

Sandia: nuclear medical waste Building permit for Waste Isolation Pilot Plant (WIPP) in Carlsbad, New Mexico (NM) Client: Department of Energy (DOE) / Environmental Protection Agency (EPA) Consultant: Sandia Labs in Albuquerque, NM Planning-horizon: 10.000 years! Criterion: chance of “disaster” Simulation: physics/chemistry & human activity; see next slide

14

WIPP (source: Jon Helton, Sandia)

15

Sonar search for mines

Simulation model: Sonar system (tilt angle, etc.) Ship (course) & human operator (coffee break) Environment: mine field & accoustics (temp., salinity) Problem: “valid” model? Model modules: input/output behavior matches experts’ qualitative knowledge? Define scenario’s simulate analyse Model as a whole: same detection probabilities in simulation & real-life test? 16

Verenigde BuizenFabrieken Problem: production planning of steel tubes different diameters, thickness Decision Support System: 15 inputs Goal: max. productive machine hours Constraint: delivery time < … days Technical problem: 1 + 2 x 15 = 31 runs 1 run takes 6 CPU hours Solution: 214 - 10 design 16 runs! Analysis: Response Surface Methodology17

RIVM Problem: greenhouse effect (now: Kopenhagen-treaty) Global model: big model 1. Validate per module Tool: sensitivity analysis: Surprising effects! Program “bugs”! 2. Total model: 281 factors Important: 15 (some surprising!) “Monitor” (fight) these 15 factors!

18

Europe Container Terminals Problem: optimize quay length & number of cranes Cranes: planning Ships: arrivals # containers on/off Simulation model: 6 factors (# containers per ship, on/off balance, yearly turnover, stay time) Interactions between factors discovered!

19

Problems & pitfalls Steps in simulation (Law 2007, p. 67): 1. Formulate the real-world problem 2. Collect real-world data incl. expert opinion 3. Construct & verify computer model 4. Make pilot simulation runs ("debug") 5. If model is not valid, then return to step 2 6. Design the simulation experiment 7. Make production simulation runs 8. Analyse simulation’s Input / Output data 9. Present & implement results 20

Step 6: design of experiment Example: VBF’s DSS with 14 "inputs" ("controls") Intuitive design: Change inputs, one-at-a-time Increase / decrease base value Hence: # combinations is 1 + 14 x 2 = 29 (total computer time: 29 x 6 hours = 174 hours) 214 - 10 fractional factorial design: Only 2 values per input Only 24 = 16 combinations suffices to estimate 1 + 14 effects General: orthogonal (Hadamard) n x k matrix 21

Step 6: Curse of dimensionality RIVM’s CO2 simulation submodel: k = 281 inputs Orthogonal matrix with n > k: too many runs Solution: Sequential Bifurcation (Ph.D. Bettonvil) aggregation & binary search Example 1, RIVM: n = 77 (