Multi-fidelity Modeling & Simulation Methodology for Simulation Speed Up Seon Han Choi, Sun Ju Lee, Tag Gon Kim Department of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon, Republic of Korea
[shchoi, sjlee]@smslab.kaist.ac.kr,
[email protected] field, analysis of future operational capability through battle experiments [2], performance acquisition of the next generation weapons system, and analysis of tactics [3] are achieved with the M&S method. Problem solving processes using M&S on general, especially the M&S-based analysis, have been performed in simulation experiments of full-factorial design as a “what-if” analysis causing the simulation to be extremely time-consuming (see Table 1). To reduce the simulation time (i.e. to enhance simulation speed), many researches are conducted in various aspects, such as Event-based DEVS Execution Environment [4] and Simulation-based optimization [5].
ABSTRACT M&S-based analysis has been performed for simulation experiments of all possible input combinations as a ‘what-if’ analysis causing the simulation to be extremely time-consuming. To resolve this problem, this paper proposes a multi-fidelity M&S methodology for enhancing simulation speed while minimizing accuracy loss and maximizing model reusability, in the M&Sbased analysis. Target systems of this methodology are continuous and discrete event system. The proposed multi-fidelity M&S methodology consists of 4 steps: 1) target model selection and Interest Region definition, 2) low-fidelity model development, 3) multi-fidelity model composition, 4) selected target model substitution. Also this methodology proposes structure of multifidelity model and its mathematical specifications for the third step. This methodology is applied without any modification of existing models and simulation engine for maximizing model reusability. Case study applies this methodology to Torpedo Tactics Simulation model and the Vehicle Allocation Simulation model. The result shows that simulation speed increases at least 1.21 times with 5% accuracy loss. We expect that this methodology will be applicable in various M&S-based analysis for enhancing simulation speed.
Table 1. Example of M&S-based analysis: Anti-Torpedo Sim [3] Combat Entity Submarine Warship Torpedo
Categories and Subject Descriptors
Decoy
I.6.5 [SIMULATION AND MODELING]: Model Development – Modeling methodologies; I.6.8 [SIMULATION AND MODELING]: Types of Simulation – Continuous, Discrete event
Total Case Execution Time
General Terms
Experimental Case (Factor) Experimental Level of Description Factor factor F1 Wire-guided tactic 2 F2 Detection range 4 F3 Avoidance angle 2 F4 Delivery angle 3 F5 Torpedo speed 3 F6 Source level 2 F7 Operating time 4 F8 Decoy speed 3 23×33×42×50 = 3456×50 = 172800 172800×20(sec) = 3456000(sec) = 40(day)
This paper centers on the multi-fidelity M&S concepts to enhance the simulation speed in the M&S-based analyses. The fidelity is defined as the degree of similarity between a model and the system properties being modeled [6]. A high-fidelity model has high accuracy in model output and simulation speed is slow; a low-fidelity model has low accuracy in model output and simulation speed is relatively high. Accordingly, multi-fidelity M&S means bring the models with various fidelity levels under certain conditions during simulation. Due to the use of appropriate low-fidelity models, simulation speed increases with minimization of accuracy loss.
Algorithms, Design, Performance, Theory
Keywords Multi-fidelity M&S; Simulation speed up; Interest Region; Model reusability
1. INTRODUCTION The Modeling and Simulation (M&S) method has been widely used for solving problems in real world that cannot be solved with numerical and analytical methods [1]. Problem solving by M&S provides new insights that other methods cannot in design/analysis/acquisition of system. For example, in defense
In recent years, the multi-fidelity M&S has been applied in many application fields such as aerospace engineering [7][8], submarine system [9], computational fluid dynamics [10][11], and supply chain system [12]. However, all of the researches focused on practical issues and on ad hoc approaches. In other words, there has been no formal representation for the multi-fidelity model and no model reusability of existing simulation models. Therefore, this paper proposes a multi-fidelity M&S methodology for enhancing simulation speed while minimizing accuracy loss and maximizing model reusability.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from
[email protected]. SIGSIM-PADS '14, May 18 - 21 2014, Denver, CO, USA Copyright 2014 ACM 978-1-4503-2794-7/14/05…$15.00. http://dx.doi.org/10.1145/2601381.2601385
139
Figure 1. Interest Region concept of multi-fidelity M&S The target model for applying this methodology is an existing model which consumes too much time due to many requisite experiments. In order to enhance simulation speed without significant loss in accuracy, it is important to know when to use low-fidelity model during simulation. For this, we firstly define an important concept called Interest Region. Interest Region is a region where the model output has a serious effect on the overall simulation result. For example (see Figure 1), when Interest Region of the sub-model C is determined, multi-fidelity M&S uses high-fidelity model C in Interest Region and low-fidelity model C in the outside of Interest Region. However, overall simulation result of multi-fidelity M&S is almost same as onefidelity M&S (Existing method), which uses high-fidelity model C in overall simulation region. The reason is that output of the low-fidelity model has a little influence on the overall simulation result in the outside of Interest Region. Furthermore, because using low-fidelity model whose simulation speed is relatively faster than high-fidelity model’s, multi-fidelity M&S can enhance simulation speed.
, , , , - , are the set of inputs and outputs. - is the set of states. -
:
-
:
, ,
, ,
is the state transition function. is the output function.
2.1.2 Discrete event system Discrete event system is described by the DEVS (Discrete EVent system Specification) formalism [14] and state variables of model have discrete value while time increases continuously (e.g. War game). Below is the formal representation of discrete event system. , , , , , , - , are the set of inputs and outputs. - is the set of states. - , | ∈ , 0 : ⟶ is the internal transition function. : × ⟶ is the external transition function. - λ : ⟶ is the output function. - : ⟶ is the time advance function.
To maximize the model reusability, the target model (existing model) is considered as a high-fidelity model, which is not modified when applying this methodology. Also low-fidelity model is developed from the high-fidelity model. Its simulation speed is faster than the high-fidelity model’s while accuracy of output is lower than the high-fidelity model’s. Also this methodology includes the formal representation of multi-fidelity model with high/low-fidelity models and is applied to the existing model without any modification of simulation engine.
Discrete event model consists of 3 sets (input, output, and state) and 4 functions (internal/external state transition function, output function, and time advance function). Input and output are events consisting the occurrence time and value. For example, the event called ‘package arriving’ consists of the arriving time (occurrence time) and contents of the package (value). State has discrete values and it is changed by two cases. When input comes, state variables are changed, and if none of the input comes until certain time, output occurs and state variables are changed. The former is called external transition and conducted by the external transition function. The latter is called internal transition and conducted by the internal transition function with the output function. The amount of time (state time) that state can remain for is decided by time advance function.
This paper is organized as follows. Section 2 provides a background and Section 3 introduces the proposed multi-fidelity methodology. The case study is described in Section 4 and a conclusion is given in Section 5.
2. BACKGROUND 2.1 Target System 2.1.1 Continuous system Continuous system is described by a differential equation, and variables in the model are changing continuously as time increases (e.g. Analog circuit). Below is the formal representation of continuous system [13]. Continuous model consists of 3 sets (input, output, and state) and 2 functions (state transition function, and output function). Input, output, and state variable have continuous values according to the time. The state transition function is represented by a differential equation and the output function is described with state and input variables. The execution of continuous model solves the state transition function and calculates the output function, as time increases in simulation engine.
2.2 Fidelity of Model
Figure 2. Fidelity of model
140
For the models that represent the same target system component, fidelity is compared by their output for the same input. Highfidelity model has lower output error than low-fidelity model, when it is compared by reference output (see Figure 2).
discrete event model, value of output is decided in output function and time of output is decided in time advance function. Therefore, these two functions are related to determine the fidelity of discrete event model. Also input, output, and state variables of highfidelity model and low-fidelity model are same; because the fidelity is compared in output for the same input (changing of variables in model is related to resolution of model [14][15]).
Table 2. Measurement method of output error Error
Continuous system
Discrete event system
3. MULTI-FIDELITY M&S METHODOLOGY Value Error Value Error
Value Error
Time Error Time Error
Time Error |
′|
Figure 3. Proposed multi-fidelity M&S methodology Proposed multi-fidelity M&S methodology consists of 4 steps: 1) target model selection and Interest Region definition, 2) lowfidelity model development, 3) multi-fidelity model composition, 4) selected target model substitution (see Figure 3). The target systems of this methodology are continuous and discrete event system. To maximize model reusability, this methodology is applied without any modification of existing models and simulation engine. The following sections will explain each step in detail.
To compare the fidelity between two models quantitatively, this paper suggests ϵ which means the difference between two outputs (i.e. output error). It is measured in two aspects, because the output of continuous model and discrete event model consists of time and value. In continuous model, Time Error (i.e. delay) is defined to be the time difference between two same values of output, and Value Error is defined as the value ratio between two output values in same time. In discrete event model, Time Error means the difference between two ET (time interval from input event to output event) of same output events and Value Error is defined to be the value ratio between two same events in the same time (see Table 2). For each aspect, ϵ is calculated by RMSE (Root Mean Square Error). ϵ
3.1 Target Model Selection and Interest Region Definition The first step consists of selecting a target model from the entire model and defining Interest Region of the selected target model. The selection criteria for the target model are below.
∑
- A model based on continuous model formalism or discrete event model formalism
1
In the formula, the Error can be the Value Error or Time Error, and it is decided according to simulation objective. N is the number of samples. For example, when the Error is the Value Error in continuous model, N is the number of time samples. When the Error is Time Error in discrete event model, N is the number of events. The range of ϵ is 0, ∞ . 0 means the output of two models are exactly same in condition of same input. The larger ϵ is, the more difference is. In other words, large ϵ means low-fidelity model about the reference model possessing reference output. The reference model can be real system or validated model. We assume that the target model is valid, and it is reference model.
- A model executed frequently during simulation and needed a high computation power (The model which has high ETR) - A model with Interest Region smaller than overall simulation region The first criterion is needed to maximize the model reusability. When the model does not meet the second criterion, we cannot get meaningful simulation speed up even if the methodology is applied to that model. Section 3.4 will explain it in detail. If Interest Region is the same as overall simulation region, we cannot use the low-fidelity model. To find the target model efficiently, the criteria are applied step by step. First, make the list of candidate models fulfilling the first criterion and sort the list in ascending order of ETR (Execution Time Ratio: The ratio of the model execution time to overall execution time). From the top of the list, define Interest Region of the model and apply the methodology if it meets the third criterion.
In continuous system, value of output is determined by the value of state variables, and these state variables are determined in state transition function. Time of output is decided in output function. Therefore, state transition function and output function are related to determine the fidelity of continuous model. However, in
141
In order to define Interest Region of model, first, describe the overall simulation region with several variables that the model can access. The several variables are the set of Interest Region Variable (IRV) which can be time variable (logical simulation time), input variables, state variables, or new variables representing a combination of input variable and state variable.
Figure 4. Interest Region definition with IRV and R Figure 6. Example of low-fidelity model development: Torpedo maneuver model [16]
After that, according to the simulation objective, define the Interest Region (R in Figure 4) of IRV. R must be smaller than the range of IRV. IRV can have many R and experiment is needed to decide optimal size of R in some cases. The important point is that IRV and R will be Fidelity Change Condition (FCC) of multifidelity model in the third step.
The state transition function of low-fidelity has fewer terms than that of high-fidelity model. As the more terms are eliminated, the accuracy of function decreases (see Figure 6), while the calculation speed increases. Therefore, the low-fidelity model has Value Error in comparison with the high-fidelity model (see Figure 6). Table 3 shows the execution time and ϵ for the same input. As the fidelity decreases, execution time of model decreases and ϵ increases.
3.2 Low-fidelity Model Development The second step is to develop the low-fidelity models from the target model selected in the first step. The low-fidelity models should have lower accuracy of output and lower execution time than the target model. The following sub-sections will explain this in detail for continuous model and discrete event model.
Table 3. Comparison of models: Torpedo maneuver model
3.2.1 Continuous model
Execution Time (Sec) ϵ
As mentioned in background, the fidelity of continuous model is determined by the state transition function and the output function. Therefore, low-fidelity model development is to simplify these two functions. Because this is the modeling issues, there are various methods. This paper suggests two methods: Elimination and Projection. Elimination means deleting the terms, which have little effect to the accuracy of output in the state transition function. On the other hand, Projection means fixing the value of variables in the state transition function. Following example will explain low-fidelity model development.
High-fidelity
Mid-fidelity
Low-fidelity
8.01
7.50
4.08
0
0.08
0.22
3.2.2 Discrete event model The fidelity of discrete event model is determined by the output function and time advance function. Therefore, low-fidelity model development is to make these two functions simple. Specifically, that is to simplify the algorithms in these two functions. These algorithms decide the time of each state in the time advance function and the output value in the output function. As is the case of continuous model, there are various methods. Following example will explain low-fidelity model development.
Figure 5. Example model: Torpedo maneuver The example model is the Torpedo maneuver model [16] (see Figure 5). Input and output variables of this model are Elevator ) and Depth (Z ) of torpedo and the state variable is Pitch ( ( ). The state transition function of the model is differential equation which consists of many terms. In this model, value of output is important, according to simulation objective. Therefore, developing low-fidelity model is to make state transition function simple. Depending on the level of eliminating terms, mid-fidelity model and low-fidelity model can be developed by using Elimination method (see Figure 6).
Figure 7. Example of low-fidelity model development: Vehicle model
142
The example model is the Vehicle model (see Figure 7). In this model, time of ouptut is important, according to simulation objective. Therefore, developing low-fidelity model is to make time advance function simple. When input (Departure) comes, the state of vehicle model change to MOVE from WAIT. Then the vehicle model makes the output after the state time of MOVE. High-fidelity model calculates the state time of MOVE by using Dijkstra algorithm [17], which finds the shortest path with given the Start and the End, in graph. However, the Dijkstra algorithm needs too much time to calculate. Low-fidelity model is developed by making this algorithm simple. The low-fidelity model uses the simple algorithm which just calculates distance between the Start and the End. The accuracy of simple algorithm is lower than Dijkstra algorithm’s, while its calculation speed is much faster than Dijkstra algorithm’s. Therefore, the low-fidelity model has Time Error in comparison with the high-fidelity model (see Figure 7). For the same input (2000 Departure event), Table 4 shows the execution time and ϵ. The low-fidelity model has 0.21 output error, while the execution time of it is lower than highfidelity model’s.
MFM consists of the internal models (the target model and lowfidelity models) and SM. Since MFM is the substitution of the target model, input and output are the same as the target model. The SM changes internal models according to Fidelity Change Conditions (FCC) during simulation. FCC is composed of IRV and R which were decided in the second step. SM has two important roles. First, when input comes, SM decides whether the current internal model changes with FCC, and if the model is not changed, SM sends the coming input to the model (i.e. Input Bypass). Otherwise, SM changes the model, copies the state from the current internal model to the next internal model for continuing simulation, and sends the coming input to the next internal model (i.e. Model Change with State Copy). For this, ) from the current internal model state transfer message ( . and state copy message ( . ) to the next internal model are added. In continuous model, SM is expressed as algorithm model; in discrete event model, SM is expressed as atomic model. The following sub-sections will explain this in detail.
3.3.1 Continuous model
Table 4. Comparison of models: Vehicle model Execution Time (Sec) ϵ
High-fidelity 2.79 0
Low-fidelity 0.41 0.21
In summary, the low-fidelity model that we developed should have lower accuracy of output and lower execution time than the high-fidelity model. In continuous model, development of lowfidelity model is to make transition function and output function simple. In discrete event model, that is to make time advance function and output function simple. Someone who knows well about the high-fidelity model or target system can develop lowfidelity models efficiently.
Figure 9. Structure of multi-fidelity model: Continuous model Figure 9 shows the structure of multi-fidelity model of continuous model. The internal models are expressed as existing continuous model specifications mentioned in the background, and SM is expressed as algorithm model which existing continuous simulation engine can simulate (see Figure 10).
3.3 Multi-fidelity Model Composition After selecting the target model in the first step and developing low-fidelity models based on target model in the second step, the next step is to composite a Multi-fidelity model (MFM) automatically. For automatic composition, the formal structure and specification are needed (see Figure 8).
Figure 10. Structure of selection model: Continuous model The set of inputs of SM consists of external input sent to the current internal model, and state input transferred from the current internal model. The set of outputs of SM consists of the external output for Input Bypass, and activate output for Model Change with State Copy. SM has two sets of states, FCC and IM. IM represents the current internal model, and consists of the current internal model, the state value of the model, and the Boolean variable representing a change of the model. The two important roles of SM are conducted by a Selection Function (sf) mapped to state transition function and an Output Function (of). The sf decides whether the current internal model is changed with FCC, when external input comes. If the current internal model is not changed, the of sends the external input to the model. Otherwise, the of sends the state of the current internal model and the external input, to next internal model (see Figure 11).
Figure 8. Structure of multi-fidelity model
-
, , , , is the set of inputs is the set of outputs , … , is the set of models with various fidelity is the Selection Model
⊆
∪ ∪ is the model coupling scheme ⊆ × . is the external input coupling ⊆⋃ . × is the external output coupling ⊆ . ×⋃ . ∪ ⋃ . × . is the internal coupling
143
∪
-
-
-
,
,
,
. ,…, . is the set of inputs. is the state input of internal model( ))
.
(
, ,
. ,…, . . ,…, . ∪ is the set of outputs. ( . is the bypass output of . is the activate output of ) | ∈ , , , is the state value of ∈ , is the set of internal model states , | ∈ , ∈ is the set of fidelity change conditions. : × × ⟶ is the selection function. : × ⟶ is the output function
, ,
Figure 12. Structure of multi-fidelity model: Discrete event model Except for the stop message, the set of inputs and outputs are the same as SM of continuous model. SM has two sets of state, FCC and IM. FCC is the same as SM in continuous model, while IM is slightly different. IM consists of the input (Xin), current internal model, the state value of the internal model, and four steps for message delivery. In discrete event model, these IM are mapped , . , . , and to 4 phases which are . . . Unlike the continuous model, the two important roles of SM are conducted by series of phase transition. The basic phase is . , according to the current internal model . When input comes, the phase moves to . and SM checks the whether the changes with sf and FCC. Also if the model is not changed, the phase changes to . and SM sends the to the model. Otherwise, the phase shifts to , according to next internal model , and SM sends the stop message to the . Then, the phase moves to . , and SM sends the activate message to the . , and SM sends the to Finally, the phase changes to . the current internal model . After that, SM is waiting until the comes. The series of phase transition of Input Bypass is below. (brackets mean the output message, ⟹ means the external transition, and ⟶ means the internal transition)
Figure 11. Two important roles of selection model: Continuous model
3.3.2 Discrete event model Figure 12 shows the structure of multi-fidelity model of discrete event system. The internal models are expressed as DEVS mentioned in background, and SM is expressed as DEVS Atomic model (see Figure 13). Unlike the continuous model, discrete event model can be executed when it has no input or internal transition. Therefore, MFM of discrete event model have ) which stops execution of current additional message, ( . internal model when it is changed. Furthermore, to stop the internal transition, a state (Stop state) whose state time is infinite, is added to the internal model. When the internal model get a stop message, its state changes to the Stop state and it waits until getting an activate message. When it receives the activate message, its state shifts to some state included in the message. ∪
( -
. .
.
,…,
.
, ,
,
,…, .
.
.
-λ: . - : .
⟶ ,
.
⟶ ⟶
. .
,
, ⟶
,
.
, .
,
. .
⇒
,
.
⟶
.
,
,
⟶ →
.
.
.
144
. →
.
))
is the set of outputs. is the stop output of , ,
.
,
,
)
is the set of
.
Model change ⟶
⟶0
, .
,
.
.
⟶ .
→
, ,
∪ . ,…, . is the activate output of , .
⟶ . , .
⟶
is the state input of internal model(
. ⟶ . No model change
.
.
.
→
, , , | ∈ , is the state value of , ∈ internal model states. Phase . ≡ , , , | , ∈ , ∈ is the set of fidelity change conditions. : × ⟶ is the selection function. : , . ⟶ . , . , . ⟶ . : .
⇒
The series of phase transition of Model Change with State Copy is below.
,
is the set of inputs. (
,…, . . ∪ is the bypass output of
.
.
No model change
.
Model change
Figure 13. Structure of selection model: Discrete event model The MFM substitutes the target model without any modification of existing simulation engine, and other models. This increases the model reusability. Small modifications used to add more messages and the Stop state in case of discrete event model are unrelated to model logic, since they are just subsidiary functions. If IRV with R and internal models are given, the MFM is composited automatically and it leads to reducing time for applying this methodology.
MFM. 1 : The ratio of 0 1
execution time to target model execution time.
: The ratio of 0 1, ∑
simulation region to overall simulation region. 1
: The overhead rate of model exchange in the MFM. E ∝ # of model exchange, 0 1
3.4 Simulation Speed Evaluation
The maximum speed up is using only the fastest low-fidelity model during overall simulation.
This section suggests the formula of speed up when applying the proposed methodology, and tells that which parts should be modified to get the maximum speed up.
1 1
1
× min
The efficiency means the ratio of actual speed up to maximum speed up when applying the proposed methodology. 1 1
Figure 14 illustrates that how to reduce the execution time when applying the multi-fidelity methodology. The time reduction means that reducing the execution time of target sub-model. Therefore, if the ETR of target sub-model is low, we can’t get meaningful simulation speed up when the methodology is applied and are defined as the to that model. overall execution time of high-fidelity model and multi-fidelity model, respectively. Speed up is defined as below.
4. CASE STUDY 4.1 Torpedo Tactics Simulation 4.1.1 Overview of torpedo tactics simulation model
1 1
1
× min ∑
Based on these formulae of speed up, to maximize the speed up, first, select the target model with large ETR. Second, simplify the C and sf in SM and that leads to reducing A. Third, reduce the Interest Region (i.e. increase the simulation region where lowfidelity is used) and that is related to increasing and reducing . The last is to minimize the frequency of model exchange and it is related to reduce E. The third and fourth are related to maximization of the efficiency.
Figure 14. Execution time reduction of multi-fidelity methodology
1 1
∑
: The ratio of the target model execution time to overall execution time. 0 1
Figure 15. Scenario of TTS
: The coefficient of structure overhead (Input Bypass) in the
145
The example of continuous model is Torpedo Tactics Simulation (TTS) whose objective is to get the torpedo hit rate according various parameters such as firing range, type of torpedo, speed of torpedo, type of decoy, number of decoy, and so on. Because of various parameters with large range, considerable time is needed to get the result. Therefore, we apply the proposed methodology to TTS for simulation speed up. Figure 15 shows a scenario of TTS. A submarine placed a distance away from a warship launches a torpedo and it finds the warship by using snake search tactic. If it finds the warship, it chases and hits the warship. When detecting it, the warship drops several decoys with evasive maneuvers. If hit target by torpedo is decoy, it finds the warship again by using circle search tactic. If it does not hit the target until the limit moving distance, it explodes itself.
models which meet the second criterion, we got ETR of sub models (see Figure 17).
Figure 18. Interest Region of Torpedo maneuver model Because the torpedo maneuver model has the highest ETR, we defined Interest Region of the maneuver model, to identify whether it meets the third criterion. To define Interest Region, we described overall simulation with ‘bDetect’ which is a Boolean variable and represents whether the torpedo detects the target. When the ‘bDetect’ is true, output of the maneuver model has a serious effect on the overall simulation result (torpedo hit rate). Therefore, Interest Region of the maneuver model is when the ‘bDetect’ is true, and the R is ‘true’. In this case, the experiment to decide optimal size of R is not needed because the IRV has only two values. The torpedo maneuver model meets the three criteria. Therefore, it is the target model for applying this methodology. The second step is to develop the low-fidelity models from the torpedo maneuver model. The maneuver model has 6 state variables ( , , , , , ) which represent velocity and angular velocity of each axis ( , , ). The state transition function of the maneuver model is very complex differential equation which is based on the Newton Equation, and calculates the change amount of state variables with various parameters such as thrust force, gravity force, drag force and so on [16]. Developing the low-fidelity models means to simplify the state transition function. For this, Elimination and Projection method mentioned in 3.2.1 are used. The third step is to composite a MFM automatically. The internal models are the existing torpedo maneuver model and low-fidelity model developed in the second step. Also FCC is composed with IVA and R decided in the first step. The last step is to substitute the maneuver MFM composed in previous step for the maneuver model. Figure 19 shows the overall process of applying of proposed methodology to TTS.
Figure 16. Overall model structure of TTS model The TTS model consists of many sub models (see Figure 16). All of the models are based on continuous model specification and implemented with C++ language.
4.1.2 Apply the multi-fidelity M&S methodology
Figure 17. ETR of sub models in TTS The first step is to select the target model and define Interest Region. TTS is based on the continuous model specification, so all of the sub models satisfy the first criterion. To find the sub
Figure 19. Overall process of multi-fidelity M&S methodology: Torpedo Tactics Simulation
146
4.1.3 Simulation result
Figure 21. Scenario of VAS on several vertexes. Customers who have the own destination, are generated in each vertex with some weight. If station is within some distance of the vertex where the customer is generated, the customer goes to the station and takes a vehicle. Otherwise, the customer calls a vehicle and the closest vehicle from the customer is allocated. If there is no vehicle in the station, the customer waits for a vehicle. After some time later, the customer calls a vehicle. If a vehicle picks up a customer who waits in a station or calls the vehicle, it goes to destination of the customer. After arriving, it goes to the nearest station and waits for a customer. It goes to the popular station, after some time has passed. A vehicle can take a call from customers in moving and waiting without picking up a customer.
Figure 20. Simulation result of TTS Torpedo hit rate can be measured with various parameters, but in this case study, it is measured according to the warship detect range. Figure 20 shows the torpedo hit rate and execution time according to the warship detect range. To accomplish the standard error is within 0.031 with confidence level 95%, torpedo hit rate and execution time are calculated in 1000 times launching. In the graph, High-fidelity model means simulation with existing TTS model, Low-fidelity model means simulation with TTS model whose maneuver model is substituted with low-fidelity model, and multi-fidelity model means simulation with TTS model whose maneuver model is substituted with the maneuver MFM. The torpedo hit rate of High-fidelity model and Multi-fidelity model are almost same, but Low-fidelity model has much accuracy loss. However, execution time of Multi-fidelity model is lower than the one of High-fidelity model, i.e., simulation speed increases about 1.25 times with 70% efficiency (see the section 3.4). Therefore, the result shows that proposed methodology can increase the simulation speed without significant loss in accuracy. Furthermore, without modification of simulation engine, and other existing models, the proposed methodology is applied and that means maximizing model reusability.
Figure 22. Overall model structure of VAS model
4.2 Vehicle Allocation Simulation
The VAS model consists of many sub models (see Figure 22). All of the models are based on DEVS formalism and implemented with DEVSim++ [18].
4.2.1 Overview of vehicle allocation simulation model
4.2.2 Apply the multi-fidelity M&S methodology
The example of discrete event time model is Vehicle Allocation Simulation (VAS) which has the objective to decide the optimal number of vehicle by measuring the average waiting time of customer and average utilization rate of vehicle according various parameters such as organization of map, distribution of customer, speed of vehicle, and so on. As is the case of the TTS, it takes lots of time to get the simulation result. Therefore we apply the proposed methodology to VAS for simulation speed up.
Figure 23. ETR of sub models in VAS
Figure 21 shows a scenario of VAS. The map is described as graph. A vertex represents a crossing and an edge represents a road. Customers and vehicles can be on a vertex, and stations are
The first step is to select the target model and define Interest Region. VAS is based on the DEVS formalism, so all of the sub models satisfy the first criterion. To find the sub models which
147
meet the second criterion, we got the ETR of sub models (see Figure 23). Because the vehicle model has the highest ETR, we defined Interest Region of the vehicle model to know whether it meets the third criterion. To define Interest Region, we described overall simulation with ‘Distance’ which is positive number and means expected distance between departure and destination of a customer. The simulation results (average waiting time and average utilization rate) are more affected when ‘Distance’ has high value. Therefore the Interest Region of the vehicle model is when the ‘Distance’ has high value. In this case, the experiment to decide optimal size of R is needed because the size of R has various values. Large Interest Region, which means high-fidelity model is used in simulation for much time, makes the simulation result more accurate and the execution time longer. While, smaller Interest Region makes the simulation result inaccurate and the execution time shorter. Therefore, by using the experiment, the optimal size of R is decided according to an optimal condition that is set by user.
Interest Region decrease), execution time is short, while accuracy of simulation result is high.
Figure 25. Experiment result for deciding optimal size of R
Figure 24. Interest Region of Vehicle model
Based on the experiment result, we should decide the optimal size of R. The optimal condition is as follows.
The second step is to develop the low-fidelity model from the vehicle model. The developed low-fidelity model is the same as one mentioned at 3.2.2. To calculate the state time of MOVE in time advance function, the high-fidelity model use the Dijkstra algorithm, while the low-fidelity model use the simple algorithm which calculates distance between the Start and the End. The third step is to composite a MFM automatically. The internal models are the existing vehicle model and the low-fidelity model developed in the second step. Also in this step, the experiment deciding optimal size of R is conducted. Table 5 shows the experimental design. This experiment checks the measurement values according to the various Start points of R from 1000 to 3500. The experiment group is multi-fidelity model whose vehicle model is substituted with the vehicle MFM, and the control group is high-fidelity model (existing VAS) and low-fidelity model whose vehicle model is substituted with the low-fidelity model.
The optimal size of R has minimum execution time among the candidates whose accuracy of simulation result is within a permissible error. If the permissible error is 0.05, the set of candidates are 1000, 1500, and 2000. Among the candidates, 2000 has the minimum execution time. Therefore, the optimal size of R is (2000, 10000) in the condition of this permissible error. The optimal size of R will be changed with the permissible error. However in this case study, we decide that the optimal size of R is (2000, 10000). The last step is to substitute the vehicle MFM composed in previous step for the vehicle model. Figure 26 shows the overall process of applying proposed methodology to VAS.
4.2.3 Simulation result To increase the reliability of simulation result, real data is used to decide the value of several parameters. First, the map is constructed based on real map data [19] in scale of 400m. Major crossings correspond to vertex, and roads and its real distance between each crossing correspond to edge and its weight. The number of customer, destination of customer, and speed of vehicle are decided based on the government report [20]. Based on these parameters, Figure 27 shows the average waiting time, average utilization rate, and execution time according to the number of vehicle. The meanings of High-fidelity model, Low-fidelity model, and Multi-fidelity model are the same as TTS. The simulation result of High-fidelity model and Multi-fidelity model are same, but Low-fidelity model has much accuracy loss. However, execution time of Multi-fidelity model is lower than the one of High-fidelity model, i.e., simulation speed increases about 1.21 times with 65% efficiency (see the section 3.4). The efficiency of VAS is lower than the one of TTS because of the high frequency
Table 5. Experiment design for deciding optimal size of R Group Experiment Group
Control Group
Model Multifidelity model Highfidelity model, Lowfidelity model
IRV
Start point
Measurement value
Distance
1000, 1500, 2000, 2500, 3000, 3500
Accuracy of simulation result (average waiting time), Execution time (sec)
Figure 25 shows the trade-off relation between execution time and accuracy of simulation result, according to the Start point of R. As is mentioned, if the Start point of R increases (i.e. the size of R or
148
Figure 26. Overall process of multi-fidelity M&S methodology: Vehicle Allocation Simulation. of model exchange in the vehicle model (see Figure 28). The high frequency of model exchange decreases the efficiency of this methodology, as mentioned. Nevertheless, the result shows that proposed methodology can increase the simulation speed without significant loss in accuracy. Furthermore, without modification of simulation engine, and other existing models, the proposed methodology is applied and that means maximizing model reusability.
Figure 27. Simulation result of VAS
Figure 28. Frequency of model exchange in a vehicle model
5. CONCLUSION This paper proposes the multi-fidelity M&S methodology for enhancing simulation speed while minimizing accuracy loss and maximizing model reusability, in M&S-based analysis. The target systems of this methodology are continuous and discrete event systems. The proposed methodology consists of 4 steps: 1) target model selection and Interest Region definition, 2) low-fidelity model development, 3) multi-fidelity model composition, 4) selected target model substitution. To maximize model reusability, this methodology is applied without any modification of existing models and simulation engine. The case studies show effectiveness of the proposed methodology in simulation speed up and model reusability. Examples of continuous system and discrete event system are the Torpedo Tactics Simulation model and the Vehicle Allocation Simulation model. The proposed methodology enhances the simulation speed about 1.25 times and 1.21 times in each example, without any modification of existing models and simulation engine.
149
[9] Molina-Cristobal, Arturo, Patrick R. Palmer, and Geoffrey T. Parks. 2011. Multi-fidelity Simulation modeling in optimization of a hybrid submarine propulsion system. In Proceedings of the European Conference on Power Electronics and Applications (Birmingham, UK, Aug. 30Sept. 1, 2011). 1-10.
6. ACKNOWLEDGMENTS This work was partially supported by Defense Acquisition Program Administration and Agency for Defense Development under the contract. (UD110006MD)
7. REFERENCES [1] Kim, T.G. 2007. Modeling and Simulation Engineering. Communication of the Korea Information Science Society. 14(6), 3-17.
[10] Peng, W., Yun, Z., Zhengping, Z., Lei, Q., and Zhixiang, Z. 2013. A Novel Multi-fidelity Coupled Simulation Method for Flow Systems. Chines Journal of Aeronautics, 26(4), 868875.
[2] Kim, J.H., Moon, I.C., and Kim, T.G. 2012. New insight into doctrine via simulation interoperation of heterogeneous levels of models in battle experimentation. Simulation: Transactions of The Society for Modeling and Simulation International. 88(6), 649-667.
[11] Zhou, Z., Ong, Y. S., Nair, P.B. Keane, A. J., and Lum, K. Y. 2007. Combining global and local surrogate models to accelerate evolutionary optimization. System, Man, and Cybernetics, Part C: Applications and Review, IEEE Transactions on, 37(1), 66-76.
[3] Seo, K.M., Song, H.S., Kwon S.J., and Kim, T.G. 2011. Measurement of Effectiveness for an Anti-torpedo Combat System Using a Discrete Event Systems Specification-based Underwater Warfare Simulator. The Journal of Defense Modeling and Simulation: Applications, Methodology, Technology, 8(3), 157-171.
[12] Celik, N., Lee, S., Vasudevan, K., and Son, Y.J. 2010. DDDAS-based multi-fidelity simulation framework for supply chain systems. IIE Transactions, 42(5), 325-341. [13] Lim, S.Y., Kim, T.G. 2001. Hybrid Modeling and Simulation Methodology based on DEVS Formalism. In proceedings of Summer Computer Simulation Conference (Vancouver, Canada, July 16-20, 2001). 188-193.
[4] Kwon, S. J., and Kim, T.G. 2012. Design and Implementation of Event-based DEVS Execution Environment for Faster Execution of Iterative Simulation. In Proceedings of the Spring Simulation Multiconference (Orlando, USA, March 26-29, 2012). p. 14.
[14] Zeigler, Bernard P., Herbert Praehofer, and Tag Gon Kim. 1976. Theory of modeling and simulation. John Wiley, New York, 510 pages.
[5] Hong, J.H., Seo, K.M., and Kim, T.G. 2013. Simulationbased optimization for design parameter exploration in hybrid system: a defense system example. Simulation: Transactions of The Society for Modeling and Simulation International. 89(3), 362-380.
[15] Hong, S.Y., and Kim, T.G. 2012. Specification of multiresolution modeling space for multi-resolution system simulation. Simulation: Transactions of The Society for Modeling and Simulation International. 89(1), 28-40. [16] Fossen, T.I. 1994. Guidance and control of ocean vehicles. Wiley, England.
[6] IEEE. 2011. IEEE Recommended Practice for Validation of Computational Electromagnetics Computer Modeling and Simulation. IEEE Std 1597.2TM-2010, p. 113.
[17] Dijkstra, Edsger W. 1959. A note on two problems in connexion with graph. Numerische mathematic, 1(1), 269271.
[7] Vitali, R., Haftka, R.T., and Sankar, B.B. 2002. Multifidelity design of stiffened composite panel with a crack. Structural and Multidisciplinary Optimization, 23(5), 347356.
[18] Tag Gon Kim. 2006. DEVSIM++ v3.0 Developer’s Manual, http://smslab.kaist.ac.kr, accessed at Jan. 2014. [19] Naver Map. http://maps.naver.com, accessed at Jan. 2014.
[8] Gano. S.E. 2005. Simulation-based design using variable fidelity optimization. Doctoral Thesis, University of Notre Dame, 14-30.
[20] Gyeonggi Research Institute. 2011. Analysis of O/D passenger traffic in capital area. http://www.gri.re.kr, accessed at Jan. 2014.
150
