Development of a Parallel Electron and Photon

Journal of the Korean Physical Society, Vol. 47, No. 4, October 2005, pp. 716∼725

Development of a Parallel Electron and Photon Transport Code (PMCEPT) I: Method and Absorbed Dose Computation in Water Oyeon Kum∗ Department of Chemistry, Clemson University, Clemson, SC 29634, USA

Sung-Woo Lee Department of Radiation Oncology, Henry Ford Hospital, Detroit, MI 48202, USA (Received 22 December 2004) The Monte Carlo (MC) method for high-energy photon and charged particle transport is the most accurate for dose calculations in radiotherapy. However, the large amount of computing time required by general purpose MC codes has prevented their use for routine dose distribution calculations for customized radiation treatment planning. One of the best ways to provide an accurate dose distribution within an acceptable time limit is to develop a parallel MC simulation algorithm on a beowulf PC cluster. We developed a parallel MC electron and photon transport simulation code (PMCEPT) based on the standard message passing interface (MPI). Our MC results agreed with those of the MCNP5 code developed by the Los Alamos National Laboratory and widely used in many hospitals. By preventing the superposition of a series of random numbers on different processors, our parallel results agreed well with the serial ones. The parallel efficiency approached 100 %, as was expected. A feasibility study for building a customized cancer radiation treatment planning simulation system showed that a Linux beowulf PC cluster with 35 up-to-date processors is adequate. PACS numbers: 87.53.W, 21.60.Ka, 02.70.Lq Keywords: Monte carlo method, Parallel electron and photon transport code (PMCEPT), Customized cancer radiation treatment, Absorbed dose, Beowulf PC cluster

1987 suggest that the two important components for a simulator are a computerized tomograph (CT) and an accurate and fast absorbed-dose-computing algorithm. A MC simulation code is currently the best tool for accurate absorbed dose calculations because it improves the simplified dose calculation methods used today, which model the body as a virtual homogeneous “bucket of water.” In these simplified methods, inhomogeneities, such as bone and airways, are ignored or highly simplified. Furthermore, interpolated data from dose measurements made in water are used to calculate radiation treatments [5,6]. These calculations are also based on a variety of simplifications of the way radiation is produced by the source. Some tumors, such as cancers of the head and neck, lungs, and reproductive organs, are particularly difficult to treat with radiation because of their proximity to vital organs, the abundance of different tissue types in the area, and the differences in their susceptibilities to radiation [2,7]. Even though current CT scans can provide radiation planners with a three- or four-dimensional (4D) (including time dimension) [8] electron-density map of the body, it is difficult to find an accurate computing algorithm that solves the clinical challenges. Thus, a MC

I. INTRODUCTION Because of the fast advancement of computer hardware and network technologies, the radiation transport Monte Carlo (MC) simulation technique is now a useful tool in the fields of nuclear medicine, X-ray diagnostics, radiotherapy physics and dosimetry, and radiation protection calculations [1]. Among its many applications, simulations for customized cancer radiation treatment planning for each patient are very important for both patient and doctor because they make possible the most effective treatment with the least possible dose to patients [2–4]. Inadequate planning, providing either too little radiation to the tumor for a cure or too much radiation for nearby healthy tissues, results in complications and sometimes death. The best solution for this problem is to build an accurate simulator to provide better treatment strategies. The overall and thorough external beam radiotherapy procedures described in ICRU Report 42 in ∗ E-mail:

[email protected]; Present address: Com2 Mac, Dept. of Math., Pohang University of Science and Technology, Pohang 790-784

-716-

Development of a Parallel Electron and Photon Transport· · · – Oyeon Kum and Sung-Woo Lee

code with acceptable performance is the essential ingredient for the simulator. Our MC dose calculation code can be used for all types of radiation therapy and can exactly model the radiation beam delivery system being used for each treatment by using each patient’s CT image as the target data [5]. A useful clinical radiation therapy planning system must provide absorbed dose distributions within a 2 – 3 % error range for a particular patient within an acceptable time limit of a few minutes, for example, five. The optimization procedure consists of selecting the best treatment plan according to a number of predefined criteria [2]. In this paper, we report our new parallel MC electron and photon transport code for absorbed dose calculations, called the PMCEPT code, which has the potential to be a MC algorithm for a useful simulator. We also studied the feasibility of building a valuable and economical treatment planning simulator using the PMCEPT code. In Section II, we summarize briefly the high-energy transport physics of photons and charged particles while in Section III, we describe our parallel MC algorithms. The results and discussion can be found in Section IV, and finally, Section V offers the conclusions of this study.

II. INTERACTIONS OF ELECTROMAGNETIC RADIATION AND CHARGED PARTICLES WITH MATTER Ever since Bhabha and Heitler [9] suggested that the multiplicative process of bremsstrahlung and pair production played a dominant role in the propagation of high-energy electrons and photons through matter, much work, both theoretical and experimental, has been conducted to deduce the characteristics of the cascade produced. This coupled electron-photon cascade, which is an inherent stochastic process, can be described completely in statistical terms. Since many papers and books [10–18] have provided the detailed characteristics of this cascade, in this paper, we summarize briefly their statistical nature and adaptiveness to simulations on a digital computer. A transport equation is given by the following linear integro-differential form: 1 ∂Φ ~ + Ω · ∇Φ + σ(E)Φ v ∂t Z Z ∞

~ 0 Φ(E 0 , Ω ~ 0 , ~r, t)ψ(E 0 , Ω ~ 0 ; E, Ω), ~ (1) dΩ

dE 0

= E

4π

~ ≡ where v is the p particle velocity, t is the time, Ω p ( 1 − µ2 cos φ, 1 − µ2 sin φ, µ) is the unit vector in the direction of the incident beam with µ = cos θ and θ being the scattering angle, ~r is the position vector, E is the energy of the particle at the position ~r, σ(E) is the probability per unit path-length of an interaction of any type

-717-

~ ~r, t)dEdΩ ~ between the particle and the medium, Φ(E, Ω, is the flux of particles at time t and position ~r with energies in the interval (E, E + dE) and with directions ~ Ω ~ + dΩ), ~ and ψ(E 0 , Ω ~ 0 ; E, Ω)dEd ~ ~ is in the interval (Ω, Ω the probability per unit path-length that a particle with ~ 0 will, as the result of a collienergy E 0 and direction Ω sion, acquire an energy in the interval (E, E + dE) and ~ Ω ~ + dΩ). ~ a direction in the interval (Ω, If the flux of secondary electrons is also of interest, ψ must take into account their product and must then be interpreted as a production rate. This integro-differential transport equation is prohibitively complicated, not allowing for an analytical treatment except under severe approximations. The MC method is the only available technique so far that can solve the equation without further approximations [10]. Though a description of the diffusion process in terms of the transport equation is analogous to the use of Eulerian coordinates in hydrodynamics, the MC method uses Lagrangian coordinates: A diffusing particle has a specific label and its trajectory is followed. Thus, the PMCEPT code embodies physical reality by following the trajectory with particles being created according to the distribution described by the source. The particles travel the collision distance, which is determined by the probability distribution and depends on the total crosssection, and scatter into another energy and/or direction according to the corresponding differential cross-section. New particles can be born when the mother particle interacts with the medium. These newly born particles are also transported in the same way. This procedure continues until all particles are absorbed or leave the region under consideration. Essentially, there are twelve possible processes by which the electromagnetic field of a photon may interact with matter [19] as seen in Table 1. As Table 1 shows, the major processes, such as Compton scattering [20, 21], pair production [22–24], and photoelectric absorption [23,25], are put in boxes while the minor processes (≥1 % contribution over certain energy intervals), such as Rayleigh scattering [23, 26, 27], are underlined; the rest are negligible processes. Note that some processes have been completely omitted because of their rare occurrences. The PMCEPT code includes the four primary scattering processes for photon interactions with matter: pair production, photoelectric absorption, Compton scattering, and Rayleigh scattering. Pair production which is closely related to bremsstrahlung, also known as materialization, is the mechanism by which a photon is transformed into an electron-positron pair. The principle of conservation of momentum and energy prevents this scattering from occurring in free space. There must be a nucleus or an electron present for this process to occur. Because this process is closely related to bremsstrahlung, the algorithm is similar to that of bremsstrahlung, which is explained in detail later in this section. Photoelec-

-718-

Journal of the Korean Physical Society, Vol. 47, No. 4, October 2005

Table 1. Simplified classification of photon interactions. Type of interaction

Scattering Absorption

Interaction with

Elastic (Coherent)

Inelastic (Incoherent)

Atomic Electrons

Photoelectric Effect

Rayleigh Scattering

Compton Scattering

Nucleons

Photonuclear Reactions

Elastic Nuclear Scattering

Nuclear Resonance Scattering

Electric Field of Surrounding Charged Particles

Pair Production

Delbruck Scattering

Mesons

Photomeson Production

tric absorption is a mechanism by which the photon disappears and charged particles are ejected from the surrounding atoms. This scattering occurs between a photon and an atom in which complete absorption is also possible. In Compton scattering, an incident photon is scattered by a loosely bound (or virtually free) electron. This process is an inelastic one in that some of the initial kinetic energy of the photon is needed in order to overcome the binding energy of the electron to the atom, and, therefore, does not appear as the kinetic energy of the products. The differential and total Compton scattering cross-sections are taken from the formulae originally due to Klein and Nishina [20]. Rayleigh scattering, also called electron resonance scattering, is an atomic process in which the incident photon is absorbed by a bound electron. This electron is raised to a higher energy state, and a second photon at the same energy level as the incident photon is then emitted, with the electron returning to its original state (it is not excitation). In effect, the photon is scattered by the atomic electron cloud, with the entire atom recoiling. The photon undergoes only a slight change in its direction and incurs only an insignificant decrease in energy because of the small recoil energy absorbed by the atom. This process is elastic. The total cross-sections for this process are taken from Storm and Israel [23]. Rayleigh scattering dominates at very low energies (lower than a few keV), and pair production dominates at high energies. Photoelectric absorption and Compton scattering occur at intermediate energies. However, photoelectric absorption dominates at lower energies than Compton scattering does. The theoretical recipes for calculating total and differential cross-sections for pair production, Compton scattering, and Rayleigh scattering are given in the above-listed respective references in detail. A charged particle moving through a medium interacts with the medium in basically three different ways, depending on the particle’s kinetic energy and the dis-

Fig. 1. Schematic diagram of the electron and positron interactions with atoms.

tance of the particle from the atom with which it interacts: (1) by collision with an atom as a whole, (2) by collision with an electron, and (3) by radiative processes (bremsstrahlung). As seen in Figure 1, when the distance of closest approach to the atom is large compared with the atomic dimensions, the moving charged particle interacts with the atom as a whole. The Coulomb force is the primary interaction force, resulting in the excitation or ionization of the atom. This distance encounter, sometimes called a soft collision, is described by either single or multiple elastic scattering [28–33]. A single elastic scattering simulation is an event-by-event simulation of electron transport and is often not possible due to limitations in computing power because of the large number of interactions with surrounding matter. To avoid this difficulty, Berger [10] developed the condensed history technique, called multiple scattering. This technique is valid whenever traveling particles undergo successions of similar processes that change the direction of motion and


the successive scatterings are statistically independent or almost independent. Thus, it is essentially equivalent to the diffusion problem. Lewis [28] showed the exact solution method by solving the integro-differential diffusion equation, which leads to expressions for the various moments of the spatial and angular distributions. The algorithm implemented in the PMCEPT approximates the Lewis solutions exactly up to the second-order moments. If the distance of closest approach to the atom is of the order of the atomic dimension, the moving charged particle interacts with one of the atomic electrons. This interaction results in the ejection of an electron from the atom with considerable energy and is often called a hard collision. In general, the energy transmitted to the secondary electron is large compared with the binding energy, and the process can be treated as a free electron collision, but the intrinsic magnetic moment (spin) [34] of the charged particle must be taken into account in the collision probability. In particular, when the two particles are identical, exchange phenomena occur and become important if the minimum distance of approach is of the order of the deBroglie wavelength, λ = h/p, where h is Planck’s constant and p is the momentum. When a high-energy charged particle pierces the electron cloud of an atom and approaches close to the nucleus of the atom, the electric field of the nucleus strongly deflects the particle’s trajectory. This deflection process results in radiative energy loses and the emitted radiation (bremsstrahlung) [35–39] covers the entire energy spectrum up to the maximum kinetic energy of the charged particle. Quantum electrodynamics (QED) [40] explains this phenomena in two ways: the emitted radiation usually consists of a number of low-energy (soft) quanta that are less than the kinetic energy of the charged particle; however, occasionally, a photon may be emitted with energy comparable to the incident particle energy. Thus, the two principal mechanisms by which electrons lose energy via their interaction with matter are inelastic collisions with the electrons of the atom and radiative collisions with nuclei [41]. The radiative collisions, which occur in the form of bremsstrahlung and positron annihilation [42, 43], transfer energy back to the photons, leading to the coupling of the electron and the photon radiation fields. In the case of bremsstrahlung, an electron or positron is scattered by two photons, a virtual photon from the atomic nucleus and another free photon that is created by the process. The basic formulae for this process are taken from the review article by Koch and Motz [35]. Butcher and Messel’s idea [21] for mixing the cross-sections for sampling of the secondary spectra is added. Below 50 MeV, the Born approximation cross-sections are used with empirical corrections added to get agreement with experiment. Above 50 MeV, the extreme relativistic Coulomb-corrected cross-sections are used. Although a single photon positron annihilation process in the nuclear field is possible, Messel and Crawford [42] pointed out that the ratio of one photon annihilation to two photon annihilation is small until higher

-719-

energies are reached, at which point the absolute value of the cross-section is small. Thus, the single photon annihilation process is ignored. Positron annihilation to three or more photons is even less likely than one photon annihilation and, therefore, also ignored. The high rate of inelastic collisions with atomic nuclei leads to frequent changes in the direction of electron transport. Radiative collisions dominate at high energies while inelastic collisions [42,44,45] dominate at low energies. Inelastic electron collisions and photon interactions with atomic electrons lead to excitations and ionizations of atoms, and highly excited atoms, with vacancies in the inner shells, are relaxed through the emission of photons and electrons of characteristic energies [46]. In the photoelectric absorption process, a photon is absorbed by an atom, and an electron is emitted with an energy given by the incident photon energy minus its binding energy. The atom, left in an excited state with a vacancy in the ionized shell, relaxes via the emission of fluorescent photons and Auger and Coster-Kronig electrons. Thus, a relaxation simulation is added which allows creation and the fluorescent photons from the K, L, M shells, the Auger electrons, and the Coster-Kronig electrons to be followed. For incident photon energies below the K−shell binding energy, the entire photon energy is deposited locally. The K−shell binding energy is always subtracted from the energy of the electron set in motion, even though there is a certain probability that the photon-absorption process takes place with a shell other than the K−shell (for high-Z materials this probability is of the order of 20 %). The total photoelectric cross-sections are taken from Storm and Israel [23] and are available for the elements 1 through 100.

III. PARALLEL STRUCTURE OF THE PMCEPT CODE Since a coupled electron-photon cascade, which is an inherent stochastic process, can be described completely in statistical terms, it readily lends itself to simulation on a digital computer. With the advent of high-speed digital computers, the direct simulation of a cascade by using the Monte Carlo (MC) method is possible. Wilson [47] used a hand simulation with the aid of a wheel of chance. However, the speed and flexibility of a digital computer is needed to obtain relatively accurate and comprehensive results. Also, all aspects of the cascade can be simulated if accurate cross-sections are used in an exact three-dimensional model. Furthermore, the type of information that be extracted from the simulated cascade is limited only by the ingenuity of the programmer in programming the computer. The disadvantage of this computer simulation is that even on the fastest computers, many hours are required to obtain statistically significant results [5]. One way to overcome this disadvantage is to use multi-processor parallel computing algorithms.

-720-


Similar to parallel replica dynamics [48], the generation of the trajectory ensemble is intrinsically a parallel task. Figure 2 shows the block diagram of the PMCEPT code. In parallel computing, the processors are divided into two classes: one master processor and the remaining slave processors. The master processor controls the entire job, as well as its assigned job, so that the appropriate parallel program has a single program multiple data (SPMD) structure. The message passing interface (MPI) [49] is implemented for the communication between processors. The first block describes the initiation of the MPI message passing network. Once the network is completed, the master processor prepares the input data. These data are acquired in two ways: by reading the provided data from the outside input files and by uploading the data from the physical library data files. Job control data, target geometry data, and material information data, including the corresponding cross-section data, are provided from outside input files. Additional data, such as spin effect corrections, photoelectric corrections, relaxation corrections, inelastic Compton corrections, and particle life time data files, are stored in the physical library and uploaded when the system starts to run. If all data are prepared, the master processor broadcasts them to all the slave processors. When all the processors have the same information, they start to

work simultaneously. Each processor generates its own set of particle trajectories. No inter-processor communication is necessary while the trajectory ensemble is being developed because all the processors have the same target information and source generation instructions. The cascade showers are divided into two paths, depending on the input particle types, photon or electron/positron. In the most probable energy range of a photon from approximately 10 keV up to about 100 GeV, the most important interactions governing the transport behavior of a photon are the photoelectric effect [23,25], pair production [22–24], Compton scattering [20, 21], and Rayleigh scattering [23, 26, 27] with relaxation [46]. On the other hand, the charged particle, in the course of traversing the medium, will be involved in an enormous number of collisions, resulting in small energy loses and deflections (single and multiple scattering) [28–33], and a relatively small number of catastrophic collisions (Møller, Bhabha, bremsstrahlung, and positron annihilation) [35–39,42–45] in which it may lose a major fraction of its energy or may be deflected through a large angle. When a charged particle travels through a condensed medium, it loses energy both via the hard collisions explained in the previous section and via the average stopping power effect (soft-collision term) [30,50]. The collision and radiative stopping powers and the range tables are taken from the National Institute of Standard and Technology (NIST) data bank Web site and the widely accepted International Commission on Radiation Units (ICRU) tabulations [50, 51]. To describe the soft-collision energy loss process, we used the continuous-slowing-down-approximation (CSDA), which is assumed to be equal to the stopping power, in the PMCEPT code. In this approximation, fluctuations of the energy loss are disregarded, and the energy of the particle is taken to be a deterministic function of the path-length traveled: Z s dE 0 0 E(s) = E0 − (2) ds (s ) ds , 0

dE ds

where is the mean energy loss per unit path-length (stopping power). In many practical situations, the CSDA range is a close approximation to the mean pathlength travelled by the particle in the course of slowing down [50]. The path-length traveled by the particle may be used as a clock to measure time by using the following equation: Z t s= v(t0 )dt0 . (3) 0

Fig. 2. Flow diagram of the PMCEPT code. The parallelization is accomplished by evenly dividing the number of primary histories into the number of processors. Final results are reduced to the master processor. The small letters represent the MPI message passing functions. The reading of input data and the writing of output data are accomplished by the master node.

For the charged particle transport, the boundarycrossing algorithm is also important if the target material is not homogeneous. The PMCEPT code uses an exact boundary-crossing algorithm [52,53] that activates a single elastic scattering mode when the charged particle crosses the boundary. Note also that the series of random numbers generated from all the processors must be different. The PM-


CEPT code has a Knuth algorithm for random numbers [54], which has almost an infinite period (1030 ). The PMCEPT code calls for approximately 1300 random numbers per history (trajectory) for the transport of a 20-MeV electron through a water phantom. Thus, the random number generator has a long enough period for a simulation of a few tens of million trajectories per processor using a few hundred processors. In order to generate non-superposed random number series in the different processors, we divided the entire period evenly among the processors and assigned the appropriate seeds (55 congruential pairs and 4 relevant variables) to each. These seeds reproduced a completely disjointed series of random numbers. Each processor accumulates relevant raw data from its own trajectory set. As soon as all processors complete their calculations, the master processor reduces the results from all slave processors, calculating the ensemble average and variance based on the collected results. The master processor writes the resulting output and finally, the program is terminated on all processors.

IV. APPLICATION TO A HEXAHEDRAL ICRU WATER PHANTOM Chemical and biological changes in a medium exposed to ionizing radiation depend on the energy deposited in the medium by the radiation. The energy absorbed in the medium from any type of ionizing radiation is given by ∆ED D= , (4) ∆m where ∆ED is the mean energy imparted by the ionizing radiation and ∆m is the mass of the medium in a volume element. The quantity of radiation is described in the unit of rad, an acronym for radiation absorbed dose, or Gray (Gy = 100 rad). According to Ref. 5, water is recommended as the standard medium for absorbed dose measurements because, for electron-beam irradiations, the absorbed dose distributions in water and human soft tissue (muscle) [55] are very similar. Furthermore, the constant chemical composition and density of water present a distinct advantage. Thus, it is recommended that other material phantom data be converted to “in-water-data” for clinical uses. For this benchmark study, the shape of the target data had to be the same as the available representation of the patient’s data. Clinical status of the patient was obtained from several different types of examinations: inspection and palpation, X-ray examination, computerized tomograph (CT) scanning, and ultrasound scanning [2]. More recent developments in the field of imaging devices, such as magnetic resonance imaging (MRI), positron emission tomography (PET), and single photon emission tomography (SPECT), can also provide a representation of the patient’s data. In using these data, we

-721-

Fig. 3. ICRU water target geometry and beam distribution area. The left figure shows the grid and beam area on the top plane at z = 0.

must remember that volumes, not areas, are being considered. In particular, the popular CT images can easily be converted into three-dimensional cubes or voxels by mapping the densities of the regions [56–58]. In this study, the target is described in a hexahedral water phantom as having many small voxels in a threedimensional cartesian coordinate system because it simulates the CT representation of the patient’s data. Figure 3 shows the initial geometry. The standard size recommended is, as for photon beams, a 30-cm cube [6]. Such a phantom is convenient for any energy used and for most clinical situations. We simulated two sizes of the target: 10.5 × 10.5 × 20 cm3 and 30.1 × 30.1 × 22 cm3 . The former is used for the evaluation of the PMCEPT code by comparing the absorbed doses with those of the MCNP code. The latter is used for studying the feasibility of building a radiation treatment planning simulation system. The cell size of the former is 0.5 × 0.5 × 0.5 cm3 , and that of the latter is 0.1 × 0.1 × 0.1 cm3 . The number of voxels is 17,641 and 19,932,221, including the default medium of vacuum, respectively. The odd dimensions are to ensure a voxcel on the central axis. Reference 5 introduces four different principal types of radiation sources and beam geometries. Among them, the most elementary type of electron beam is the pencil beam or, more specifically, the mono-directional beam from a point source. However, we chose the planeparallel beam shown in the Figure 3 for this study because it describes the situations more realistically. The beam distribution area is 2.5 × 4.5 cm2 in the x-y plane at z = 0.0. This area is smaller than the size of the phantom. The beam is assumed to be distributed uniformly on this area. The initial kinetic energy of each electron beam is 20 MeV. This energy corresponds to the upper bound of the clinically used beam therapy energy. We chose a high energy beam for this study. In general, the computing time increases as the initial beam’s kinetic energy increases, and the statistical convergence becomes slower as the beam’s distribution area becomes larger. The electron’s stopping power and mean free path for

-722-


Fig. 6. Isodose curves in the y-z plane at x = 0. The light curves represent the MCNP results and the dark curves the PMCEPT results. The unit of dose is 10−12 Gy·cm2 /particle.

Fig. 4. Stopping power and discrete mean free paths in the water phantom. The horizontal axes are log scales.

Fig. 7. Isodose curves in the x-z plane at y = 0. The light curves represent the MCNP results and the dark curves the PMCEPT results. The unit of dose is 10−12 Gy·cm2 /particle.

Fig. 5. Dose distributions of the PMCEPT and the MCNP5 codes at the central column of the target geometry. The central column of the target is shown.

the target medium are shown in the Figure 4. We used a small target for comparison with the MCNP5 code developed by Los Alamos National Laboratory. This MCNP5 code, a general purpose Monte Carlo N-Particle code, is widely used for clinical calculations in many hospitals. We used the MCNP5 version 5 (2003) in this study, and the computing time for the MCNP5 code run on a PC having 2600+ AMD Athlon cpu and 256-MB ram memory operated by Windows NT was approximately 900 minutes for the calculations of

one million particle histories with no variance reduction technique. On the other hand, the PMCEPT code run on a Linux PC having 1-GHz Pentium III cpu and 1GB ram memory was approximately 110 minutes for the same calculations . Thus, the computing time of the PMCEPT code was about nine times faster than that of the MCNP5 code, even on the slower computer [59]. Figure 5 shows the dose distributions along the central voxels. Each voxel volume in the central column (shown in the figure) is 0.5 × 0.5 × 0.5 cm3 . The two results agree with each other within a 1 % error. Figure 6 shows the isodose curves in the y-z plane at the center of the x-axis (x = 0.0). Figure 7 shows the isodose curves in the x-z plane at the center of the y-axis (y = 0.0). The PMCEPT results agree well with those of the MCNP5, suggesting that the PMCEPT code can provide clinically acceptable and accurate solutions faster than the MCNP5 code.


Fig. 8. Isodose curves in the y-z plane at x = 0. The light curves represent the serial results and the dark curves the parallel results using 10 processors. The unit of dose is 10−12 Gy · cm2 /particle.

Fig. 9. Isodose curves in the x-z plane at y = 0. The light curves represent the serial results and the dark curves the parallel results using 10 processors. The unit of dose is 10−12 Gy · cm2 /particle.

We also compared the parallel results calculated from ten processors with the serial results for a small target. Figure 8 shows the isodose curves in the y-z plane at x = 0.0. Figure 9 shows the isodose curves in the x-z plane at y = 0.0. In both plots, half of the target length in the z-direction is not shown because only half of the z-axis shows appreciable doses. Both plots compare the serial outputs (light curves) with the parallel outputs (dark curves). The maximum dose points agreed as shown in Figure 8 though there was a slight difference on the next maximum isodose curve. The agreement was perfect in the x-z plane. Overall, all the isodose curves agreed well with each other. The parallel speedup for one million primary particle trajectories is shown in Figure 10. The parallel speedup

-723-

Fig. 10. Parallel efficiency. The total elapsed times are 6411.8, 6432.6, 6433.2, and 6452.1 seconds for 1, 2, 5, and 10 one-GHz Pentium III processors in the beowulf PC cluster, respectively.

resulting from N processors is defined as the ratio of the serial run-time to the average run-time of the N processors. We used the elapsed time (wall-clock time) in this study. The parallel efficiency approached 100 %, as was expected, for the Linux beowulf PC cluster, which is the most economic high-performance computer. We studied the feasibility of building a cancer radiation therapy planning simulation system by using a large water phantom. The target size used in this study was the realistic clinical size recommended from ICRU Reports [5, 6, 55]. We used 6-, 12-, and 20-MeV electron beams. In this study, the acceptable criteria for accuracy was within a 0.21 % error, and the computing time was five minutes. For an accuracy within a 0.01 % error, primary trajectories of a million order were required. We estimated the number of processors to meet these criteria in a Linux beowulf PC cluster. The result was that approximately 100 730-MHz Pentium III processors, 85 1-GHz Pentium III processors, or 35 2.8-GHz Pentium IV processors were necessary. This result suggests that PC clusters with 30 to 40 up-to-date processors can build a useful cancer radiation therapy planning simulation core system. Considering the fast advancement in electronics and network technologies, we expect that even a smaller system will be able to perform this function.

V. CONCLUSIONS We developed a parallel Monte Carlo electron and photon transport simulation code (PMCEPT code) based on the standard MPI message passing interface in a threedimensional cartesian coordinate system. The absorbed dose calculations agreed with those of the MCNP5 code within a 1 % error. The computing time of the PMCEPT code was faster than that of the MCNP5 code. The parallel results agreed with the serial results, with the parallel efficiency approaching 100 % on a Linux beowulf PC cluster. The feasibility study of a customized radiation

-724-


therapy planning simulation system showed that even a beowulf PC cluster having 35 up-to-date processors was adequate. Many other applications [60,61], including all areas of nuclear medicine and biology, chemistry, electronics (detectors), and high energy physics, can also be expected to use the PMCEPT code. The PMCEPT code consists of two parts: a preprocessor that calculates the total and the differential scattering cross-sections for the given material and the major simulation code that calculates the interactions of the photons and charged particles with the target materials. An immediate next step is to develop the software for transforming CT image data into useful target data. Subsequently, the capability of the PMCEPT code will be extended to include simulations with additional particle types, such as protons, neutrons, and heavy ions. Both of these studies are currently underway.

ACKNOWLEDGMENTS We thank Bill Hoover for his careful proofreading and thoughtful comments. We also thank Steve Stuart for his helpful conversations and support. This work was supported, in part, by the SRC/ERC program of MOST/KOSEF (grant number: R11-1999-054).

REFERENCES [1] Pedro Andreo, Phys. Med. Biol. 36, 861 (1991). [2] Internal Commission of Radiation Units and Measurements, ICRU Report 42 1987. [3] J. Sempau, S. J. Wilderman and A. F. Bielajew, Phys. Med. Biol. 45, 2263 (2000). [4] S. J. Lie, T. Pawlicki, J. Deng, S. B. Jiang, E. Mok and C. M. Ma, Phys. Med. Biol. 45, 2969 (2000). [5] Internal Commission of Radiation Units and Measurements, ICRU Report 35 1984. [6] Internal Commission of Radiation Units and Measurements, ICRU Report 24 1976. [7] Internal Commission of Radiation Units and Measurements, ICRU Report 36 1983. [8] P. J. Keall, J. V. Siebers, S. Joshi and R. Mohan, Phys. Med. Biol. 49, 3639 (2004). [9] H. J. Bhabha and W. Heitler, Proc. Roy. Soc. (London) A159, 432 (1937). [10] B. Alder, S. Fernback and M. Rotenberg (Eds.), Method in Computational Physics, Vol. 1 (Academic Press, New York, 1963). [11] W. R. Nelson and T. M. Jenkins (Eds.), Computer Techniques in Radiation Transport and Dosimetry (Plenum Press, New York, 1980). [12] H. M. Colbert, Sandia Laboratory (Livermore) Report Number SCL-DR-72019 1973. [13] R. L. Ford and W. R. Nelson, SLAC-210 (June 1978). [14] C. L. Hartmann Siantar and E. I. Moses, Europ. J. Phys. 19, 513 (1998).

[15] X-5 Monte Carlo Team (Eds.), Los Alamos National Laboratory Report LA-UR-03-1987 (2003). [16] Application Software Group (Eds.), CERN Program Library Long Writeup W5013 (CERN, Geneva, 1993). [17] M. B. Emmett, ORNL-TM-4972, Oak Ridge National Laboratory (Oak Ridge, TN, 1975). [18] J. Sempau, A. Sanchez-Reyes, F. Salvat, H. Oulad ben Tahar, S. B. Jiang and J. M. Fernandez-Varea, Phys. Med. Biol. 46, 1163 (2001). [19] U. Fano, L. V. Spencer and M. J. Berger, Encyclopedia of Physics, Vol XXXVIII/2, edited by S. Flugge (Springer, Berlin/Gottingen/Heidelberg, 1959). [20] O. Klein and Y. Nishina, Z. f¨ ur Physik 52, 853 (1929). [21] J. C. Butcher and H. Messel, Nucl. Phys. 20, 15 (1960). [22] J. W. Motz, H. A. Olsen and H. W. Koch, Rev. Mod. Phys. 41, 581 (1969). [23] E. Storm and H. I. Israel, Atomic Data and Nucl. Data Tables 7, 565 (1970). [24] A. F. Bielajew, NRCC Report, IPRS-0287R (1994). [25] D. E. Cullen, M. H. Chen, J. H. Hubbell, S. T. Perkins, E. E. Plechaty, J. A. Rathkopf and J. H. Scofield, LLNL Report UCRL-50400, Vol. 6, Revision 4 (Livermore, California, 1989). [26] J. H. Hubbel and I. Øverbø, J. Phys. Chem. Ref. Data 9, 69 (1979) [27] R. G. Jaeger, E. P. Bilzard, A. B. Chilton, M. Grotenhuis, A. H¨ ong, Th. A, Jaeger, H. H. Eisenlohr (Eds.) Engineering Compendium on Radiation Shielding, Vol. I (Springer-Verlag, New York, 1968). [28] H. W. Lewis, Z. Phys. Rev. 78, 526 (1950). [29] G. Z. Moli´ere, Z. Naturforsch 2a, 133 (1947). [30] H. A. Bethe, Phys. Rev. 89, 1256 (1953). [31] W. T. Scott, Rev. Mod. Phys. 35, 231 (1963). [32] S. Goudsmit and J. L. Saunderson, Phys. Rev. 58, 36 (1940). [33] L. V. Spencer, Phys. Rev. 98, 1597 (1955). [34] J. W. Motz, H. A. Olsen and H. W. Koch, Rev. Mod. Phys. 36, 881 (1964). [35] H. W. Koch and J. W. Motz, Rev. Mod. Phys. 31, 920 (1959). [36] Y. S. Tsai, Rev. Mod. Phys. 46, 815 (1974). [37] S. M. Seltzer and M. J. Berger, Nucl. Instr. Meth. B12, 95 (1985). [38] S. M. Seltzer and M. J. Berger, Int. J. Appl. Radiat. Isot. 33, 1219 (1985). [39] S. M. Seltzer and M. J. Berger, Atomic Data and Nuclear Data Tables 35, 345 (1986). [40] C. Claude, D. Jacques and G. Gilbert (Eds.), Photons and Atoms: Introduction to Quantum Electrodynamics (Wiley, New York, 1989). [41] W. H. Barkas and M. J. Berger, NASA-SP-3013 (National Aeronautical and Space Administration, Washington, D.C., 1964). [42] H. Messel and D. F. Crawford, Electron-Photon Shower Distribution Function (Pergamon Press, Oxford, 1970). [43] W. Heitler, The Quantum Theory of Radiation (Clarendon Press, Oxford, 1954). [44] C. M¨ oller, Ann. Physik 14, 531 (1932). [45] H. J. Bhabha, Proc. Roy. Soc. A 154, 195 (1936). [46] S. T. Perkins, D. E. Cullen, M. H. Chen et al., Lawrence Livermore National Laboratory Report UCRL-50400, V. 30 (Livermore, California, 1991). [47] R. R. Wilson, Phys. Rev. 86, 261 (1952).

Development of a Parallel Electron and Photon Transport· · · – Oyeon Kum and Sung-Woo Lee [48] O. Kum, B. Dickson, S. Stuart, B. Uberuaga and A. Voter, J. Chem. Phys. 121, 9808 (2004). [49] MPI document group, MPI-2: Message Passing Interface, http://www-unix.mcs.anl.gov/mpi/ (2004). [50] Internal Commission of Radiation Units and Measurements, ICRU Report 37 1984. [51] http://physics.nist.gov/cgi-bin/Star. [52] A. F. Bielajew, D. W. O. Rogers and A. E. Nahum, Phys. Med. Biol. 30, 419 (1985). [53] B. J. Foote and V. G. Smyth, Nucl. Inst. Meth. B 100, 22 (1995). [54] W. H. Press, B. P. Flannery, S. A. Teukolsky and W. T. Vetterling, Numerical Recipes (Cambridge University Press, Cambridge, 1986). [55] International Commission on Radiation Units and Measurements, ICRU Report 44 1989.

-725-

[56] I. Kawrakow, M. Fippel and K. Friedrich, Phys. Med. Biol. 23, 445 (1996). [57] L. Wang, M. Lovelock and C. S. Chui, Med. Phys. 26, 2626 (1999). [58] A. Leal, F. S. Doblado, M. Perucha, E. Carrasco, M. Rincon, R. Arrans and C. Berbal, Comp. in Sci. & Eng. 60, 1 (2004). [59] Later, one of the authors found that the computing time of the PMCEPT code was approximately twenty times faster than that of the MCNP5 code on the IBM ThinkPad X40 (laptop) with 1.2-GHz cpu and 512-MB ram memory operated by RedHat Linux 9.0 (absoft Fortran compiler) for the same small-size problem. [60] M. Son, J. Rhee, J. Lee and H. Hwang, J. Korean Phys. Soc. 45, 540 (2004). [61] H. Cho, et al., J. Korean Phys. Soc. 46, 418 (2005).