A Survey of Imaging across the Electromagnetic

A Survey of Imaging across the Electromagnetic Spectrum Ziyi Zheng

Center for Visual Computing (CVC) Department of Computer Science Stony Brook University

Advisor: Klaus Müller February 8, 2010

Abstract Various imaging technologies have been extending human perception into the full range of the light spectrum. Existing and emerging imaging technologies follow a similar trend, moving from 2D imaging to 3D tomography. The fundamental goal of 3D tomography is to precisely reconstruct a real world 3D volumetric object. Applications are in medicine and biology, but also computational science, the arts and design, and entertainment. The introduction of accurate modeling of physical phenomena into the tomographic reconstruction procedure promises great leaps to be made with regards to the delity of the reconstructed object. This requires iterative reconstruction methods since analytical tomography theory fails to incorporate physical phenomena such as scattering, refraction, and the like. Iterative reconstruction involves a repeated sequence of forward and backward projections. Here the forward projection must be an accurate high-quality rendering since reconstruction is the inverse problem of rendering. Recently developed advanced rendering techniques can provide fast and accurate forward models which we believe will signicantly benet tomographic reconstruction. The physically based photon transport model, shown as a feasible solution for providing high quality illumination eects, has been extensively investigated in past decades. This report starts by reviewing local illumination and global illumination models in computer graphics. More generally, we address physics property of electromagnetic waves at various wavelengths, from microwave to visible light to X-rays. We then give an overview of existing tomographic algorithms that reconstruct objects via dierent electromagnet wavelengths. Following, we present on-going research work. We conclude with potential research directions on high-quality reconstruction approaches emerging from the concepts studied in this report.

i

ii

CONTENTS

CONTENTS

Contents 1 Introduction

1

2 Local Illumination

3

2.1

Attenuation and Emission

. . . . . . . . . . . . . . . . . . . . . . .

3

2.2

Scattering Approximation

. . . . . . . . . . . . . . . . . . . . . . .

3

2.3

Ray Integration Method

. . . . . . . . . . . . . . . . . . . . . . . .

4

2.4

Wavelength View

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

6

3 Global Illumination 3.1

3.2

7

Refraction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

3.1.1

Wave Model . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

3.1.2

Particle Model

8

3.1.3

Parameterization

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

8

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

10

3.2.1

Rayleigh Scattering . . . . . . . . . . . . . . . . . . . . . . .

10

3.2.2

Compton Scattering

. . . . . . . . . . . . . . . . . . . . . .

11

3.2.3

Phase Function

. . . . . . . . . . . . . . . . . . . . . . . . .

11

3.2.4

Monte Carlo Integration

Scattering

. . . . . . . . . . . . . . . . . . . .

12

3.3

Simulation Methods . . . . . . . . . . . . . . . . . . . . . . . . . . .

13

3.4

Wavelength View

16

. . . . . . . . . . . . . . . . . . . . . . . . . . . .

4 Tomographic Reconstruction 4.1 4.2

4.3 4.4

Gamma-ray

17

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

17

X-ray . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

19

4.2.1

Filtered Back Projection . . . . . . . . . . . . . . . . . . . .

19

4.2.2

Iterative Reconstruction

20

4.2.3

Better Forward Model

Optical

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

21

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

23

Diuse Optical

. . . . . . . . . . . . . . . . . . . . . . . . . . . . .

24

4.4.1

Diusion Equation Based Models

. . . . . . . . . . . . . . .

25

4.4.2

Radiative Transfer Equation Based Models . . . . . . . . . .

26

4.5

Microwave . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

27

4.6

Radiation Therapy Planning . . . . . . . . . . . . . . . . . . . . . .

27

4.7

Proton CT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

29

4.8

Ultra Sound . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

30

5 Current and Future Work

32

5.1

Frequency Domain Upsampling

. . . . . . . . . . . . . . . . . . . .

33

5.2

Adaptive Renement . . . . . . . . . . . . . . . . . . . . . . . . . .

33

5.3

Gaussian Transfer Function

5.4

Lattice-based Fast Scattering

. . . . . . . . . . . . . . . . . . . . . .

34

. . . . . . . . . . . . . . . . . . . . .

35

6 Conclusion

36

iii

CONTENTS

CONTENTS

iv

1 INTRODUCTION

1

Introduction

A variety of imaging technologies have been extending human perception into the whole range of light spectrum. For example, X-ray image is widely used in medical area to aid diagnosis. Near-infrared is used in night vision. Near-ultraviolet is for forensic analysis. Usually, a dierent wavelength of light (electromagnetic wave) can lead us to discover hidden objects of great interest. Dierent as they seem, they all have a similar model. The physically based radiative transport equation (RTE) provides a solid and unied basis for those imaging techniques. It is widely used in computer graphics to simulate scene illuminated via visible light. RTE based method can generate photorealistic images, portraying every ne detail of the real world, outshine other drawing methods. Aiming at creating an exact copy of real scene, physically based method is proven as the only solution to generate high-quality images.

More importantly, when

applied to dierent wavelength, it facilitates various imaging technologies move from imaging to tomography, as shown in Table 1.

Table 1: Imaging technology development

2D Image technology X-ray radiography

gamma cameras diuse optical imaging

Tomographic technology X-ray computed tomography (CT) single photon emission computed tomography (SPECT) and positron emission tomography (PET) diuse optical tomography (DOT)

Unfortunately, sophisticated physics-based model can not be incorporated into existing analytical reconstruction method. Thus iterative reconstruction method comes into play. Iterative reconstruction involves a repeated sequence of forward and backward projections. It forms an estimation-correction loop until nal convergence. Since the forward projection is the simulation of rendering, the more accurate forward projection would give us better convergence rate and ner resolution. However, in practice, some physics eects, such as refraction and scattering, are usually simplied or even omitted to achieve interactive speed. Focusing on physically base rendering, we list photon-matter interaction cases in computer graphics, shown as Figure 1 [31]. While the ultimate model is class 9 (full global light transport). Mostly omitted parts such as scattering and refraction can be essential for medical imaging thus should not be comprised. Fortunately, with continuous growing computation power of

graphics processing unit

(GPU), the gap between physically

correct simulation and faster rendering speed is shrinking. We will witness various tomographic reconstruction techniques beneting from GPUs' growing power in the near future. This report is organized as follows. Section 2 describes the physical simulation including model 1, 2, 3 and 6 shown in Figure 1, as local illumination. Section 3 discusses the rest of the eect, as global illumination. Section 4 gives a survey of various of tomographic techniques. They all under the same physics rule of

radiative transfer equation 1

(RTE), the only dierence is

1 INTRODUCTION

Figure 1: Photon interaction model [31]

wavelength. This section is organized according to the electromagnet spectrum, shown in Figure 2.

Figure 2: Electromagnetic spectrum, courtesy to South Carolina Alga Ecology Lab Dierent rays, as electromagnet waves with dierent light frequency, are used to probe dierent part of scene. We illustrate dierent rays in according to their

Positron emission tomography (PET) and Single photon emission computed tomography (SPECT) are discussed in Section 4.1. X-ray's wavelength is between 0.01-10 nm. Computed Tomography (CT) is studied in Section 4.2. Visible light lies in 400-750nm wavelength interval. In Section 4.3, Optical tomography (OT) is discussed as a wavelength. Gamma-ray's wavelength is smaller than 0.01nm.

method to reconstruct volumetric phenomena from camera images. Near-infrared (NIR) is the ray with wavelength between 750-1100nm. Night vision system lies in this range.

Diuse optical tomography

(DOT) is an emerging technique where

visible and NIR light is used to probe the absorption and scattering properties of biological tissue, described in Section 4.4.

For larger wavelength, microwave

imaging is introduced in Section 4.5. Analogy to X ray CT, Proton CT is under development to aid simulation in proton radiation therapy in Section 4.7. The current and future work is in Section 5. Section 6 concludes the report.

2

2 LOCAL ILLUMINATION

2

Local Illumination

This section discusses the physics law of photon attenuation and emission in participating media. Based on this model, we continue to present previous research related to visualize illuminated volume with simplied scattering eects, known as local illumination. Ray tracing method with this simplied model from RTE can be compute interactively in modern GPU. Current research in this area mainly focuses on transfer-function design, for example spectra representation.

2.1 Attenuation and Emission In most cases, attenuation and emission are the most dominant eects of photon transport.

They serve as a simple case model without consider other more so-

phisticated photon-matter interactions. Since it provides a compromise between generality and eciency of computation, it is most widely used and studied model. If scattering is ignored, for any point on a ray with distance s to start point of the ray, the amount of light varies:

where

τ (s)

is the

dI = q(s) − τ (s)I(s) dS extinction coecient, I(s) is

(1)

s

light intensity at

is the transfer function representing light emitted by the material.

τ (s) is also called opacity. The opacity α of Rl α = 1 − exp(− 0 τ (t)dt) when it is projected parallel rendering,

a voxel of side

and

q(s)

In volume

l

is actually

to one voxel axis. Solving

this dierential equation, we get:

I(D) = I0 e

−

RD 0

τ (t)dt

+

Z D

−

q(s)e

RD 0

τ (t)dt

ds

(2)

0

is the distance from the viewer to theR start point of the ray and I0 s − τ (t)dt 0 is the intensity at s = 0. The quantity T (s) = e is the transparency of Where

D

the material between 0 and s. Thus it can be simplied to I(D) = I0 T (D) + RD 0 0 q(s)T (s)ds Finally, the light intensity I must be calculated through integration along the ray. The continuous integration along the ray

I=

Z D

c(s(x(t)))e−

Rt 0

x(λ)

can be computed through

τ (s(x(t0 )))dt0

dt

(3)

0 where

c

is the transfer function of color values. This emission-absorption vol-

ume rendering equation over a line segment is dened in 1988 [69].

Various of

technique to compute this volume rendering integral will be discussed in the Section 2.3.

2.2 Scattering Approximation Previous equation can be written as the dierential form

dI = w · ∇x I(x, w) = −τ (x, w)I(x, w) + q(x, w) dS

3

(4)

2.3 Ray Integration Method


w · ∇x I is the dot product between the light direction w and the gradient of radiance I with respect to position x. If a light ray is parametrized by dI arc length s, then w · ∇x I is actually . dS The term

Taking the scattering eect into consideration, the light transport equation is

dI = w · ∇x I(x, w) = −(τ (x, w) + σ(x, w))I(x, w) + q(x, w) dS Z σ(x, w0 )p(x, w0 , w)I(x, w0 )dΩ0

+

(5)

sphere Where

σ(x, w)

is the scattering coecient.

The last integration term is in-

scattering value, accumulating contributions of scattering from all possible direc0 0 tions w . p(x, w , w) is a phase function which describes the chance that light 0 is scattered from original direction w into w . With this probabilistic denition, p(x, w0 , w) ∈ [0, 1] and it is normalized according to

1 Z p(x, w0 , w)dΩ0 = 1 4π sphere For example, for uniform scattering,

(6)

p(x, w0 , w) =

ing phase function is so expensive to compute. functions are described in Section 3.2.3.

1 . The realistic scatter4π Details of physics based phase

Rendering with Blinn-Phong model is

one of the commonly used optical model, especially for surface shading. It is an approximation and is not physically correct, for example, do not conserve energy. It only account for the single scattering from external light, and assume received scattering light as constant ambient light.

w · ∇x I(x, w) =

dI = g(s) + gBlinnP hong (s) − τ (s)I(s) dS

gBlinnP hong (s) = Iambient + Idif f + Ispecular = kα Mα Iα + kd Md Id hl · ni + ks Ms Is hh · nin

(7)

(8)

Besides the Blinn-Phong model, Engels [23] listed a variety of other shading models, such as Lafortune's model, Banks' anisotropic mode, and the physically based model of Ashikhmin can be used. For physically based specular reection, a popular illumination model is the microfacet model introduced by Cook-Torrance.

2.3 Ray Integration Method There is a great amount of work to simulate the attenuation and emission in volume rendering, especially with GPU recently.

The volume rendering technique

can be classied as indirect method, such as iso-surface reconstruction and direct method that immediately display the voxel data.

Compared with indirect

rendering techniques (e.g. the Marching Cube algorithm [58]) using surface with geometric primitives,

Direct Volume Render

(DVR) is a more attractive approach.

It is more accurate since the rendering samples are given by direct interpolation without any intermediate representation.

4

2.3 Ray Integration Method


DVR as a large category, contain several methods. For example, ray casting, cell projection, shear-warp, splatting and texture-based methods.

The common

theme is an approximate evaluation of the volume rendering integral for each pixel. The continuous integral is represented by Riemann sum:

D/4t

I≈

X

(c(s(x(i4t)))4t ×

i=0

i−1 Y

e−τ (s(x(j4t))4t )

(9)

j=0

It is a zero-order approximation. By introducing the opacity value

e−τ (s(x(i4t))4t

and dene

Ci = c(s(x(i4t)))4t, I≈

n X

i−1 Y

Ci

i=0

αi = 1 −

the Eq. 9 can be simplied to

(1 − αi )

(10)

j=0

The computation can be formulated as front-to-back composition during the ray tracing.

Cf ront = Cf ront + (1 − αf ront )αback Cback

(11)

αf ront = αf ront + (1 − αf ront )αback

(12)

The most popular method to do this front-to-back composition is the texturebased method. An early texture-based method proposed by Westermann [87] was developed for 2D textures only. His slice-based algorithm contained XYZ 3 sets of object-aligned slices. More recent technique made the use of 3D texture supported in graphics hardware to produce viewport-aligned slices. This will reduce one stage of resampling and reduce some artifacts. Riemann sum method stated above is just an zero-order approximation of the continuous integral.

One disadvantage of zero-order ray casting method is

aliasing. Pre-integrated volume rendering [24] is a good approach to reduce the loss of discretized transfer function. It used rst-order approximation. As further stated by Kniss [46], this analytic integration is more accurate.

The zero-order

approximation (nearest neighbor lter) assumes that data values near the samples are constant. If we assume data value along the ray between parameter a and b vary linearly, the opacity term becomes:

α(v1 , v2 , l) = 1=e−τ l

Rl 0

ρ(v1 +t(v2 −v1 ))dt

= 1=e−τ lρ

0

(13)

v1 = v(a) is the data value at ray parameter a, v2 = v(b) is the value at 0 ray parameter b, l = b=a, and r is the density line integral along the segment. For where

arbitrary transfer function, the rst order (linear lter) integral can be expressed as

r (v1, v2) = 0

Z 1 0

=R(v1) r(v1 + t(v2=v1))dt = R(vv2)= v 2

(14)

1

where

R(v) =

Z 2 −∞

5

r(x)dx

(15)

2.4 Wavelength View


In general, the rst order integral

ρ0

has no analytic solution.

Assume the

transfer function is a sum of Gaussian, Kniss [47] developed an acceleration method similar to front-back composition. If we assume trilinear interpolation, a line segment across the volume will have an intensity prole. This prole is a non-equal-space piecewise polynomial of order 3.

More pleasant volume rendering result can be achieved by using second

order interpolation, as presented by Hajjar et al. [22] in 2008. Since there was 0 no analytical solution to ρ , they presented a 3D pre-integrated table to store the pre-computed result. Their 3D lookup table was computed in GPU using iterative method and achieved near-interactive updating rate. Algorithms supporting changing step size according to the piecewise boundary would result in a 4D lookup table. Since 4D lookup table would be too large to be computed interactively with transfer function, they still used equal-length step size with a 3D lookup table.

2.4 Wavelength View Each material has its unique ability to absorb and emit light. Human eyes may not able to distinguish some of them in visible light, because they have same response in visible light, or they are occluded by other material with large absorption in visible light. Therefore, various imaging devices come into play. Imaging techniques in various of wavelength use absorption and emission spectroscopy to extend our vision to full spectrum. The emission and absorption processes are traditionally simulated as transfer function, that is, for current lighting source, the material with dierent densities have dierent emission colors and dierent opacity (attenuation and scattering). In most cases, a dierent wavelength will reveal a dierent look of a given object. The property changes according to dierent wavelength are recognized as material's emission spectroscopy.

Noordmans et al. [61] described the lack of support for

emission spectrum in transfer function. Many researches incorporate spectra into volume rendering. The popular RGB model could be replaced by a more general and more physically-based spectra function [10] [11] [12]. Similar to emission spectroscopy, absorption spectroscopy is another useful concept. Attenuation coecient is the inverse term for penetration. Some opaque material in one wavelength will look transparent in another wavelength. Infrared imaging and ultraviolet imaging are developed to help night vision, thermal imaging and forensic analysis. The popular X-ray CT is based on the high penetration of tissue and of low penetration of bones. Dual Energy CT utilized the dierent spectra.

6

3 GLOBAL ILLUMINATION

3

Global Illumination

In real world scene, the straight line path is not always the case. According to refraction index, photon (electromagnetic wave) will bend its path through inhomogeneous medium. The local illumination models mentioned in previous section are based on straight path assumption, thus fail to generate realistic refraction. Furthermore, although Blinn-Phong shading model is popular for fast shading surfaces, it does not provide sucient lighting characteristics for translucent materials or materials where scattering dominates the visual appearance. Global illumination accounts for these two major eects. It can generate photorealistic eects such as caustic, soft shadow and color-bleeding eects. Section 3.1 focuses on the physics and path tracing method on refraction, which corresponding to model 5 in Figure 1. Section 3.2 describes the scattering model. Then we move to review various methods to simulate refraction and scattering eect in Section 3.3.

3.1 Refraction RTE and Eikonal equation are two physics models to simulate light transport. RTE is not very suitable to describe refraction, since it requires the refractive index to be constant in the medium.

Although extensions of RTE for spatially

varying refractive index can be derived, it is not favorable to Eikonal equation in eciency. Eikonal equation is commonly used instead of RTE, to simulate light propagation in inhomogeneous medium. One of the good points of Eikonal equation is that it represents wave-particle duality which is a physical characteristics of light. Reection and refraction are most easily described by thinking of light as particles. But refraction index is fundamentally described in terms of wavelength. A more complete understanding of illumination requires both models.

3.1.1 Wave Model The Eikonal equation is a non-linear partial dierential equation encountered in wave propagation. It explains light propagating wave fronts along arbitrary trajectory. It is derived from Maxwell's equations, and provides a link between wave optics and geometric optics. The physical meaning of the Eikonal equation is related to the formula

E = −∇Ω where

E

is the electric eld intensity and

(16)

W is the electric potential.

There

is a similar equation for velocity potential in uid ow and temperature in heat transfer. The physical meaning of this equation in the electromagnetic example is that any charge is pushed outward at a right angle from lines of constant potential. The same principle applies to thermodynamics and uid ow. For ray optics, the line of constant potential is replaced by a line of constant phase.

It represents

the edge of advancing light waves (wavefront). While the force lines is replaced by normal vectors coming out of the constant phase line at right angles. represent the light traveling direction.

7

They

3.1 Refraction


One fast computational algorithm to approximate the solution to the Eikonal equation is the fast marching method.

Figure 3: light wave propagation and viewing ray propagation [33]

3.1.2 Particle Model The most simple model for refraction is that photon transport across the boundary of two mediums. If we have the interface between two medium dened, the wellknown refraction equation to represent lights bending is Snell Law

ni sin θi = nr sin θr where

(17)

θi is the angle of incidence, θr is the angle of refraction, ni is the refraction nr is the refraction index of the refractive

index of the incident medium and medium.

Eq. 17 can be used multiple times for more generalized case, in which multiple media are dened in space. This simplication is useful in path tracing technique.

3.1.3 Parameterization In order to simulate the continuous wave propagation in computer, we need to perform a discretized parameterization. The propagation of viewing rays and light transport rays behave very similarly and the governing equations are derived from the same basic equation, the ray equation of geometric optics. However, we need to use dierent parameterizations to account for specics in the two processes. For light rays, we have to take the irradiance fall-o into account whereas viewing rays carry radiance. For viewing ray, a photon in a grid based volume keeping interacting with participating media can be expressed as the following equation

d dx (n ) = ∇n ds ds 8

(18)

3.1 Refraction


is the photon transport path, n is the scalar eld of refractive index. dx By dening v = n , we can rewrite Eq. 18 as a system of rst order dierential ds equations: Where

x(s)

v dx = ds n dv = ∇n ds

(19)

(20)

which can be discretized using a simple Euler forward scheme

vi n = vi + ∆s∇n

xi+1 = xi + ∆s

(21)

vi+1

(22)

The equations above have the property that the spatial step size is equal for all ray trajectories. This provides advantages for rendering, where the number of iterations for each particle trace should be approximately equal to ensure optimal performance. For light source wavefront propagation, we can discretize the wavefront into a set of inter-connected particles which are propagated independently. This way, the wavefront is subdivided into so-called wavefront patches whose corners are dened by light particles. The connectivity information is needed for the dierential irradiance computation. The propagation of the particles is similar to eye ray propagation. We reparameterize it to yield equitemporal discretization steps:

n

d 2 dx (n ) = ∇n dt dt v dx = 2 dt n dv ∇n = dt n

(23)

(24)

(25)

which can be discretized using Euler forward scheme

vi n2 ∇n = vi + ∆t n

xi+1 = xi + ∆t vi+1

(26)

(27)

This reparameterization is necessary to enable a simple formulation of the differential irradiance computation. It ensures that all particles stay on a common wavefront over time which is necessary to apply the simple intensity law of geometric optics instead of wavefront curvature tracking schemes. This formulation enables a fast GPU implementation of the wavefront propagation scheme as a particle tracer.

9

3.2 Scattering


3.2 Scattering This section describes the physics model behind scattering and the previous work in this area. We start by explaining the scattering as a physical concept and its computational models (Section 3.2.1 and 3.2.2). Light can be scattered by participating media, essentially changing the direction of light propagation. If the wave-length (or the energy of photons) is not changed by scattering, the process is called Conversely

inelastic scattering

elastic scattering (Rayleigh scattering),

(Compton scattering) aects the wavelength.

In visible light, the scattering eect is due to Rayleigh scattering . If the photon interaction extended to the energy range from 100 eV to 1 GeV (X-ray and Gammaray), Compton scattering will show up. These two interactions are usually referred as coherent scattering and incoherent scattering in physics. Scattering from larger spherical particles is explained by the Mie theory for an arbitrary size parameter

x.

The Mie theory reduces to the Rayleigh approximation.

Figure 4: Rayleigh and Compton scattering from PENELOPE [70]

3.2.1 Rayleigh Scattering As shown in Figure 4, photons in Rayleigh Scattering are scattered by bound atomic electrons without lose energy, i.e.

section

elastic.

The atomic

dierential cross

(DCS) per unit solid angle for coherent scatting is given approximately by

where

dσT (θ) dσR = [F (q, Z)]2 dΩ dΩ

(28)

dσT (θ) 1 + cos2 θ = re2 dΩ 2

(29)

θ is the polar scatterF (q, Z) is the atomic form factor for a specic material with atomic Z . The quantity re is the classical electron radius and q is the magnitude

is Thomson DCS for scattering by a free electron at rest, ing angle and number

of the momentum transfer given by 1 E θ E q = 2( ) sin( ) = [2(1 − cos θ)] 2 c 2 c

In Thomson law, the DCS Rayleigh scattering per scattering angle

(30)

θ

can be

expressed as

dσR (E, cos θ, Z) r02 = 2π (1 + cos2 θ)F (x(E, cos θ), Z)2 d cos θ 2 10

(31)

3.2 Scattering


3.2.2 Compton Scattering Photon in Compton scattering, as shown in Figure 4, will lose some of its energy. E is the energy of the incident photon. The energy of the scattered photon E 0 =

E − W,

for the scatter angle

θ

can be expressed as

E0 =

E 1+

( mE0 c2 )(1

(32)

− cos θ)

In Klein-Nishina law

KN re2 E 0 2 E 0 E dσCo = Z ( ) ( + 0 − sin θ2 ) dΩ 2 E E E

(33)

Similar to Rayleigh scattering, the scattering angle can be sampled from DCS given by Klein-Nishina law. It is modied by the incoherent molecular scatting function expressed as

dσC (E, cos θ, Z) r2 = 2π 0 (1 + cos2 θ)FKN · S(x(E, cos θ), Z) d cos θ 2 where

S(x, Z) is the molecular incoherent scattering function and FKN

(34)

is Klein-

Nishina coecient.

3.2.3 Phase Function After integrating the dierential cross section

dσ

over

∆(cos θ)j ,

Rayleigh scatter-

ing and Compton scattering can be expressed as normalized phase functions.

1 σR (E, ∆(cos θ)j , Z) · PR (E, ∆(cos θ)j ) = PN −1 j=0 σR (E, ∆(cos θ)j , Z) 2π

(35)

σC (E, ∆(cos θ)j , Z) 1 PC (E, ∆(cos θ)j ) = PN −1 · j=0 σC (E, ∆(cos θ)j , Z) 2π

(36)

The physically based model is often simplied due to their complexity. If the wavelength dependence is not an important characteristic of the phase function, Henyey-Greenstein (HG) phase function is often used. This phase function was specially designed to be easy to use for tting measured scattering data. A single parameter,

g

controls the distribution of scattered light.

pHG (cos θ) = The value of

g

1 1 − g2 2 (1 + g 2 − 2g(cos θ))3/2

must be in the range

(−1, 1).

Negative values of

(37)

g

correspond to

back-scattering, where light is mostly scattered back toward the incident direction, and positive values correspond to forward scattering. The greater the magnitude of

g,

the more scattering is scattered close to the

An alternative phase function was developed by Schlick as an ecient approximation to the Henyey-Greenstein function. It has been widely used in compute graphics due to its computational eciency since it does not call the power function.

11

3.2 Scattering


pSchlick (cos θ) =

1 − g2 1 2 (1 − g cos θ)2

(38)

Considering more sophisticated scattering eects like rainbow and glories, wavelength-dependent phase function is needed. Mieplot provides this function.

3.2.4 Monte Carlo Integration The Monte Carlo methods are probabilistic methods used to nd an approximate solution to integral equations that have dicult analytical solution. These methods are useful for computations in the matters of physics of light.

D. The Monte Carlo method allows to integrate the function g(x) over its domain D by generating a sequence of independent samples on D according to a probability density function (pdf ) p(x). The value of the integral can be seen as the expected value of the g(x) with pdf p(x), and this can be estimated by sampling the random variable p(x) variable on D using p(x) as pdf, obtaining the unbiased estimator We have a function

I=

g(x)

Z

to be integrated over a given domain

g(x)dx =

Z D

D

g(x) g(x) p(x)dx = Ef [ ] p(x) p(x)

I ≈ hIi =

N g(x) 1 X N i=1 p(x)

(39)

(40)

Importance sampling is a well-known technique for MC integration. The uniform pdf is a simple estimation but waste much computational cost. What's more, the resulting estimations are not stable, with large variation. If the ideal pdf is proportional to the integrand

p(x)

g(x),

it will makes the variance of the estimation

I

is unknown, density functions that mimic

zero. Since the value of the integral

importance functions. The sampling according to these importance functions is called importance sampling. In other words, importance sampling consists of sampling more points in the integrand have to be used. These functions are called

the regions where

|g(x)|

is greater. This technique is widely used in Monte Carlo

methods. The samples on

D

from the inverse of the cumulative distribution known as

xk

from

inversion method

P

−1

hξk i.

p(x) are usually obtained function P (x). This procedure is

according to the density function

, and consists in computing the sequence of samples

One simple discrete-domain example is Russian roulette to

termination photon. Where PDF, CDF and sampling is shown in Figure 5. Another frequently used case is exponential distribution

p(s) =

1 −s/λ e λ

(41)

Sampling formula will be

s = −λ ln(1 − ξ)

(42)

The expected value's variance depends on the number of sample points. Then the error in the Monte Carlo estimation (or convergence rate) is proportional to

12

3.3 Simulation Methods


Figure 5: PDF CDF sampling for Russian roulette

√

N,

where

N

is the number of samples taken. As an example, this means that

the number of samples has to be multiplied by 100 to reduce the error by one order of magnitude. This implies that pure Monte Carlo simulation is not able to be used as an interactive solution.

3.3 Simulation Methods Complex refractive photon path can be computed analytically by Eikonal Equation. While for the simple case (refraction between homogeneous media), refractive path can be simply using piece-wise line segments. For the complex case (photon path tracing in inhomogeneous medium), Ihrke proposed Eikonal rendering [33] to use Eikonal equation for fast casting of bent light rays and viewing rays.

They also propose an ecient light propagation

technique using adaptive wavefront tracing.

The light propagation method can

be used in path tracing together with Monte Carlo simulation of scattering. The limitation of ray tracing is that it can not compute indirect illumination eciently. For the latter case, the refractive path direction is depended on surface normal thus the use better lter to reconstruct surface will improve smoothness. In [52], they compared interpolation lters and found that the B-Spline family lters, a widely used non-interpolating basis function, estimate the function and its gradient much more accurately than CC lters.

In [51], they oer an elegant method

based on a ltered octree with pre-classied cells to identify a refractive iso-surface quickly, while still employing high-quality interpolation lters. The stochastic nature of involved processes such as photons interaction with matter and detection makes MC the ideal tool for modeling scattering . MC is discussed in Section 4.2.3.

X-ray

Monte Carlo methods are more versatile and

require less memory. However, pure Monte Carlo path tracing (including bidirectional path tracing) uses signicant amounts of computation time to render images without noise. It has slow convergence rate make it inecient. Photon mapping [34] is a method that uses biasing to reduce variance in MC simulation. possible.

The idea is aggressively storing and re-using information whenever

In their approach, ray tracing will rstly generate two photon maps,

then viewing ray will perform gathering. Global photon map and caustic photon map are generated as two separate photon maps. Global photon map need gather indirectly and require less photon density. It requires hemispherical nal gathering for good results, typically with 200 to 5000 rays per gather. Caustics are rendered directly since it has a high density of photons. As shown in Figure 6, during photon tracing, Russian roulette decides whether

13



absorb, refract or scatter. In the rendering stage, direct illumination is computed just like regular Ray-Tracing but the indirect illumination, which comes from the walls and other objects around, is computed from querying the stored photons in the photon map. The rst diuse surface seen either directly through a pixel or via a few specular reections is evaluated accurately using gathering. Several rays are shot from the rst hit point into the environment. Whenever a sample ray from the gathering step reaches another diuse surface the estimate from the global photon map is used shown in Figure 7. Photon map represents incoming ux in the scene, however, radiance (photon density) is used for rendering. There is an assumption that nearest N neighbor will represent the local photon density. Then a KNN search is performed to approximate the local photon density. sphere centered around

x

the radius of the sphere is

Lr (x, w) =

is extended until it contains

n

photons. At this point

r. Z Ω

fr (x, w0 , w)Li (x, w0 )(nx · w0 )dw0

d2 Φi (x, w0 ) (nx · w0 )dwi0 (nx · w0 )dwi0 dAi Ω Z d2 Φi (x, w0 ) = fr (x, w0 , w) dAi Ω n X 4Φp (x, wp ) fr (x, wp , w) ≈ πr2 p=1 =

A

Z

fr (x, w0 , w)

(43)

The nal component is caustic, which is can be rendered by simply render photon map since it contains high photon density.

Figure 6: photon tracing and gathering from Jensen [34, 35] Jensen later extended photon mapping from surface to participating media [35]. One dierence is interaction can happen anywhere in participating media. They rstly modied the photon interaction scheme accordingly. The

F (x)

expressing

the probability of a photon interacting with a participating medium at position

x

is

−

F (x) = 1 − τ (xs , x) = 1 − e

Rx xs

κ(s)ds

(44)

If photon interaction event happens, Russian roulette is used to decide whether absorb or scatter. The probability of a photon being scattered is the scattering

14



Figure 7: Radiance estimation for surface and volume [34, 35]

albedo

σ(x)/κ(x).

The new direction of a scattered photon is chosen using impor-

tance sampling based on the phase function at

x.

Using ray marching, the photon

maps can be generated similarly. Another modication happens in radiance estimation. The in-scattering ra0 diance Li depends on radiance L from all directions w over the sphere Ω. The in-scattered radiance is shown in Figure 7.

Li (x, w) =

Z Ω

fr (x, w0 , w)L(x, w0 )dw0

d2 Φi (x, w0 ) 0 dwi σ(x)dwi0 dV Ω d2 Φi (x, w0 ) 1 Z fr (x, w0 , w) = σ(x) Ω dV n 1 X 4Φp (x, wp ) ≈ fr (x, wp , w) 4 σ(x) P =1 πr3 3 =

Z

fr (x, w0 , w)

There are several methods to accelerate photon mapping. Irradiance caching is a technique that exploits the fact that the irradiance eld is often smooth, to accelerate the gathering step. This is done by computing the expected smoothness of the irradiance around a given sample location.

If the

irradiance is determined to be suciently smooth then the estimate is re-used for this region. The metric takes both distance and normal distance into account.

i =

||xi − x|| q + 1 − ~n · n~i R0

(45)

Given this estimate of the local variation in irradiance, Ward [85] developed a caching method where previously stored samples re-used whenever possible. All samples are stored in an octree. When a new sample is requested the octree is queried rst for previous samples near the new location. For these nearby samples the change in irradiance, is computed. If samples with a suciently low is found, these samples will be blended. If no previously computed sample has a suciently high weight then a new sample is computed. To further improve this estimate, the gradients of the irradiance can be added. The idea is looking not only at the distances to the nearest surfaces, but also in the relative change of the incoming radiance from dierent directions in order to more accurately estimate the gradient due to a change in position as well as orientation. It does not require any further samples, but simply uses a more sophisticated analysis of the samples in the irradiance estimate.

15

3.4 Wavelength View


While Global illumination traditionally is used for o-line image generation, GPU-based global illumination is developed recently. In 2008, Sun [77] proposed a GPU-based refraction pipeline more generally allowing for media whose index of refraction varies continuously throughout space.

Both the photons and the

viewing rays followed curved paths rather than the straight-line paths ordinarily used in photon mapping. It was built with dynamic voxelization refraction render and achieved real time rendering. Zhou presented a GPU-based kd-tree algorithm together with general photon mapping [98].

In 2009, Wang [84] implemented a

GPU-based adaptive sampling and light cut algorithm [81]. His optimized photon mapping approach achieved near-interactive frame rate for medium scene. Qiu [65] proposed a lattice-based LBM framework for volumetric global illumination where participating media and volumes were rendered with multiple physically-based scattering eects. The ray propagation used Monte Carlo simulation scheme based on material phase functions with spectral dependencies. They developed ecient particle storage and propagation schemes, in conjunction with an ecient lattice structure, which allowed diuse photon tracing algorithm to render high quality images at a fast speed. This work is mainly on CPU and it will be transferred it to GPU. Instant radiosity [39] is another two pass method. It was originally only for non-specular surface but later it was extended to handle refraction and reection. Their algorithm generates a particle approximation of the diuse radiance in the scene using the quasi-random walk based on the method of quasi-Monte Carlo integration. lights.

It approximates the indirect illumination using many virtual point

The resolvable detail is directly related to the number of virtual lights.

With this idea, Wald [80] implemented interactive global illumination using a cluster of PCs.

Light cut [81] is a scalable method that computes illumination

from a large number of point lights at sub-linear cost.

3.4 Wavelength View The eect refraction heavily depends on the wavelength. For visible light, a good example is prism. The refraction eects are dierent between red and blue, creating a rainbow.

For X-ray, refraction eects are negligible.

Compton scattering is not negligible.

16

For Gamma-ray,

4 TOMOGRAPHIC RECONSTRUCTION

4

Tomographic Reconstruction

Most tomographic application uses light as the probe to reconstruct 3D object. Some tomographic reconstructions use light intensity as the source of reconstruction. For example, X-ray and SPECT. Some of the application, only intensity is not enough. Time is added as another dimension. TOF-PET. Frequency can be taken into account in DOT. Considering transport time into the equation, the RTE can be written as

Z 1 ∂I + w · ∇I(x, t, w) + (τ + σ)I(x, t, w) = σ p(w, w0 )I(x, t, w)dΩ0 c ∂t sphere +q(x, t, w) (46) For example, if

σ

and

q

are zero, and the system is assumed to be steady-state,

then equation 46in the steady-state becomes the dierential form of the Radon transform for X-ray CT:

w · ∇I(x, w) + τ I(x, w) = 0 ⇒ I(b, w) = I(a, w)e− Similarly if

τ

and

σ

are zero and

q

R1 0

τ (a+λ(b−a))dλ

(47)

is non-zero and isotropic, we obtain the

Radon transform for SPECT:

w · ∇I(x, w) = q(x) ⇒ I(b, w) =

Z 1

q(a + λ(b − a))dλ

(48)

0 If both

τ

and

q

are non-zero, then the problem becomes the attenuated Radon

transform, with the exponential Radon transform representing the case whereτ is non-zero and constant. In optical tomography, scattering is by far the dominant process and Radon transform formulations of the forward problem are not appropriate, except for the tiny fraction of photons that are theoretically available, having undergone no scattering in transit.

4.1 Gamma-ray In

Single Photon Emission Computed Tomography

(SPECT) a single Gamma-ray

is emitted per nuclear decay. A gamma camera rotates around the patient and records 2D projections of the radioactivity.

A large number of such data sets

need be collected. Using back-projection method, it can reconstruct a 3D volume of the radiopharmaceutical distribution in the body. SPECT provides functional images with improved contrast but spatial resolution is limited. The resolution of a SPECT is poor (about 1 cm).

Positron emission tomography

(PET) detects pairs of Gamma-rays (511 keV

each) emitted indirectly by a positron-emitting radionuclide, which is introduced into the body on a biologically active molecule.

The use of detector to dene

a line along which the disintegration has taken place is inherently more precise for image formation than using a single-photon emitter and collimator.

Images

of event (tracer) concentration in 3-dimensional space within the body are then reconstructed.

The resolution of PET is about 57 mm.

preferred method of brain imaging.

17

PET scanning is the

4.1 Gamma-ray


Figure 8: standard FD and convolution-based FD from [21]

In 2007, Sitek [75]proposed a simple Compton scattering modeling in GPU. He used point clouds to represent data and deterministically modeled the scattering as a matrix multiplication. They are going to accelerate this method in CUDA and validation of their work in Monte Carlo simulation is not done yet. Traditionally, tomographic reconstruction is done Cartesian lattice.

As the

iterative reconstruction is commonly used, Sitek [76] shows 3D object can be reconstruct as adaptive tetrahedral in point cloud. The statistical noise variance in positron emission tomography (PET) can be reduced by an order of magnitude by using time-of-ight (TOF) information. Recent advances in technology (scintillators, photodetectors, and high-speed electronics) helped to practically develop Time-of-ight PET (TOF-PET) [60]. In 2009, Matej [59] proposed a view-grouped histo-images to accelerate the TOF-PET. His method can be used in both view-by-view update and ordered subset iterative reconstruction. He implemented his approach in conjunction with the row-action maximum-likelihood algorithm (RAMLA) with view-by-view update. His method achieves 10X speedup only in CPU. He is going to accelerate this further in GPU. Filtered back-projection for X-ray are applicable to PET reconstruction when data is processed in sinogram. However, list-mode data acquisitions provide extremely high temporal resolution with full spatial resolution. List-mode data can be binned into sinograms, allowing frame durations to be determined after acquisition. In [63] [64], they use GPU to accelerate list mode and histogram mode 3D OSEM. They get about 51X performance compared to CPU implementation. In order to get more accurate forward model, Monte Carlo simulation is used often as gold standard. A large variety of variance reduction techniques (VRT) have been introduced to speed up MC simulations. One example of a very eective and established VRT is known as forced detection (FD). Standard FD have been developed to minimize the number of photons that needs to be simulated. Photons that have scattered are not allowed to escape the object, but are always forced towards the detector. Convolution-based Forced Detection [21] involves replacing the sampling of the PDF by a convolution with a kernel which depends on the position of the scatter event, as shown in Figure 8.

They showed that a MC

simulator using CFD converges to noise-free projections up to one or two orders in magnitude faster than if standard FD is used. They experiment this on SPECT but the idea is not limited to Emission CT.

18

4.2 X-ray


4.2 X-ray The most developed and popular CT reconstruction technique is for X-ray. While analytical approaches can be traced back to the Radon transform, iterative algorithms seek to optimize some objective function, such as maximum likelihood or minimal error. In 1994, Cabral rstly proposed tomographic reconstruction in graphics pipeline utilizing texture mapping [17]. With the rapid development of GPU, recently many research and commercial groups reporting that the graphics pipeline can make a huge speed up compared to CPU. In GPU implementation, both exact algorithm and iterative algorithm can be formulated into a combination of projection and backprojection operators. Xu and Mueller [88] proposed GPU-based unied projection and backprojection in 2003. They implemented OSEM, SART and FDK within a unied framework. This framework in not only for transmission tomography but also valid for emission tomography. Later they [90] shows new oating point GPU result with projection and backprojection framework. We will explain exact algorithm and iterative algorithm separately in details. Then we will discuss some potential ways to improve the forward model (projection operator).

4.2.1 Filtered Back Projection The Radon transform is invertible in closed form by a variety of methods, such as resampling in the Fourier domain, ltered back-projection, or back-projection convolution. Filtered back-projection is most common one. It has a general form consist of two operator: projection and back projection. The projection is written as:

U ⊗F pi = where

t

Z L

µ(t)dt =

0

3 −1 NX

µj wij

(49)

j=0

is the parametric variable dened along the ray, and

between the source and the detector bin. voxels while the

p

The

L

is the distance

m are the attenuation factors of

are the pixel values recorded on the detector. The

weights with which the values of the

mj contribute to the pixels pi.

wij

are the

For FDK algorithm, wij is 0 further multiplied by a depth correction factor resulting wij . The back projection is written as Then a ramp lter is performed on projections.

vj =

X

0 p0i wij

(50)

p0i ∈Pϕ where

Pϕ are

ltered projection with viewing angle

ϕ.

This type of CT is called exact CT. GPU acceleration for popular FDK algorithm is done by Xu and Mueller [89]. FDK can be further accelerated in graphics pipeline.

Streaming and early-fragment-kill technique was exploited by Xu and

Mueller [94] later. Another exact conebeam algorithm is Katsevich Algorithm.

It is based on

function analysis and it is theoretical superior to FDK algorithm. It is a ltered

19

4.2 X-ray


backprojection type algorithm. This feature makes it a good candidate for future GPU acceleration. Other than traditionally reconstruct on Cartesian lattice, Xu and Mueller [92] used reconstruction on BCC lattice on GPU. BCC lattice rendering is fairly well supported by hardware. Much less sample can be used to achieve similar quality in CC lattice. Due to the Small number of samples in GPU-based projection, they used voxel driven other than pixel driven method to do interpolation.

4.2.2 Iterative Reconstruction Expectation Maximization (EM) and the Simultaneous Iterative Reconstruction Technique (SIRT) are two iterative computed tomography reconstruction algorithms often used when the data contain a high amount of statistical noise or have been acquired from a limited angular range.

A popular mechanism to increase

the rate of convergence of these types of algorithms has been by performing the correctional updates within subsets of the projection data.

This has given rise

to the method of Ordered Subsets EM (OS-EM) and the Simultaneous Algebraic Reconstruction Technique (SART). Most iterative CT techniques use a projection operator to model the under-

ϕ.

lying image generation process at a certain viewing angle

The result of this

projection simulation is then compared to the acquired image obtained at the same viewing conguration. If scattering or diraction eects are ignored, the modeling

ri

consists of tracing a straight ray

from each image pixel and summing the contri-

butions of the reconstruction voxels contribution of a

vj

to

ri

vj .

Here, the weight factor

wij determines

the

and is given by the interpolation kernel used for sampling

the volume. The projection operator is given as:

ri =

N X

i ∈ {1, 2, . . . , M }

vj wij

(51)

j=1 where

M

and

N

are the number of rays and voxels, respectively. Since GPUs

are heavily optimized for computing and less for memory bandwidth, computing the

wij

on the y, via bilinear interpolation, is by far more ecient than storing

the weights in memory. For SART, the correction update for projection-based algebraic methods is computed with the following equation:

(k+1)

vi

(k)

X

= vj + λ

pi ∈OSs

pi − ri N X

wij

ri =

N X

(k)

wil · vl

(52)

l=1

l=1 For OSEM, the algorithm is written as:

vj



N X



(k)

(k+1)

vi

=

wij

N X



di wij 

p0i ∈Pset

di =

pi N 3 −1

X

p0i ∈Pset

wil ·

(53)

(k) vl

l=0

About for further acceleration on iterative method, Xu and Mueller [95] studied the GPU constraint in OSEM by showing the optimal subset sizes. suggested similar pattern in SART.

20

They also

4.2 X-ray


4.2.3 Better Forward Model Traditional forward model is simple ray integration. It does not account for scattering, using models as simply as Radon transform. The error can be introduced by interpolation and resampling. Re-sampling error can be reduced by using better lters. Xu and Muller [91] reviewed dierent lters used in projection integration. For FDK reconstruction, Tu [78] performed noise simulation for cone-beam CT in CPU cluster. They validated their model with quantum noise, detector noise and system noise. To account for scattering, beam-harden eect on X-ray CT, scattering most be modeled. There are two kinds of approach, deterministic method and stochastic methods. Xu and Mueller [93] used Gaussian kernel simulate the scattering. Slice composition in volume rendering was used in GPU to perform projection and backprojection fast. The scattered (blurred) result was storing in FBO and added to next slice. Their scattering model was only over a hemisphere, without backward scattering. They reported that unmatched projector/back-projector pair will produce better result. In Section 3.2.4 we introduced some concepts and techniques of the Monte Carlo probabilistic methods for solving complex integration. Monte Carlo analysis in medical and biological engineering has been used for several decades. However with the ever-increasing power of computers, the utility of Monte Carlo simulation is increasing. The use of Monte Carlo (MC) techniques for radiation transport simulation has become the most popular method for modeling radiological imaging systems and particularly X-ray computed tomography (CT). MC method is considered as the accurate forward model, especially for scattering in X-ray. It is seen as the gold standard forward model in iterative method. Many of existing X-ray simulators by MC method is reviewed in [96]. There are public domain general purpose MC codes such as PENELOPE [70], GEANT4 [2], EGS [29] and MCNP [14].

PENELOPE and EGS are based on

FORTRAN 77. MCNP is in FORTRAN 90. GEANT4 is in C. A lot of publications employed above simulators to create their own application. Ay and Zaidi [9] simulate fan-and cone-beam CT based on the general purpose MCNP4C radiation transport computer code. The simulator includes the possibility of detailed simulation of the X-ray spectra [8]. Investigation of dierent microCT scanner congurations by GEANT4 simulations is made in [40]. These general purpose codes are physically widely validated and robust but slow for interactive application. Colijn et al. [20] developed a rapid MC-based micro-CT simulator, refereed as accelerated MC simulator (AMCS). The simulation process is divided in two main parts:

the projection data of primary X-ray photons are computed with

a ray-tracer and the scatter distribution is estimated using an accelerated MC simulation method, using point detector approach with denoising tting. In the point detector approach, photons from each interaction point are forced to generate a signal in each of the detector elements. In the case of cone-beam CT, forcing detection in each of the detector elements would increase the computing time prohibitively, since the number of detector pixels and thus the number of line integrations is extremely high. An eective way to reduce the number of line integrations is to force the photons towards a limited subset of the

21

Ndet

pixels

4.2 X-ray only.


FD Ndet =5

pixels or

FD Ndet = 100

pixels are chosen randomly if an interaction

is due to Compton or Rayleigh scatter, respectively. An extra weighting factor of FD is assigned to the photon to correct for the fact that not all pixels are Ndet /Ndet sampled. In the latter stage, Colijin [19] used Richardson-Lucy tting algorithm to obtain low-noise scatter estimates from the noisy FD projections. It uses ML algorithm to t Gaussian basis functions to the simulated data. Smooth estimates of scatter projections can therefore be obtained even from simulations with low number of photons. This allows for a reduction of the time needed for MC simulation by two orders of magnitude in comparison with MC with FD only.

Figure 9: Eect of dierent order of scattering [20]

Figure 10: Deterministic single scattering [48] Another method makes use of the fact that single scattering has the major eect. Multiply scattering, taking the most computation time, contains only low frequency. Kyriakou [48] proposed that the multiple-scatter distribution builds a low-frequency oset such that a fast and exact deterministic simulation of single scatter can be combined with a coarse estimate of multiple scatter in order to calculate the total-scatter intensity. as shown in Figure 11. To determine the next interaction point for a given but randomly selected number vector

s

ξ

, summing up of the photon's path must take place along the direction

with a given step size

ds

until

ln(1 − ξ) < −

X

− µ(E 0 , l · ds · → s )ds

(54)

l det det det Is = Is,1 + α((IsM C − Is,1 ) ∗ G) = Is,1 + α(4S M C ∗ G)

22

(55)

4.3 Optical


Figure 11: Primary, single scattering, total scattering and scattering-primary-ratio [48]

Recently, some simple MC simulators use CUDA to accelerate the process. Without consider attenuation (White Monte Carlo), in semi-innite geometry, photon transport get 1000x speedup [3]. However, that case is the simplest case. Based on MCML(Monte Carlo in Multi-Layered tissue) [83], they also developed CUDA MCML. It is for photon transport without Compton scattering in multiple layer media where HG phase function was used for approximate photon scattering. They reported about 1000x speed up in CUDA MCML. For more sophisticated models, some on going research reported that rewriting part of a realistic benchmark PENELOPE in CUDA would get 42x speedup.

4.3 Optical X-ray tomography can successful to reconstruct volume because some media looks transparent (or partially transparent) in X-ray. When similar idea applies to visible light, optical tomography is developed to reconstruct transparent and specular object. Glass is the typical object to reconstruct in this category. Irhke [33] studied wave-propagation in refractive object and provided a fast forward model for complex refractive object, based on Eikonal Equation.

He gave a review about

state of art in transparent and specular object reconstruction [31]. In some cases the object have high emission value but do not have much attenuation and scattering. Flames is the typical object to reconstruct in this case.

23

4.4 Diuse Optical


Figure 12: The three fundamental types of DOT instrument [71]

This special optical CT was developed by Ivo Irhke. Irhke [32] presented an optical tomography method using adaptive grid for reconstruction of a 3D density function from its projections. The proposed method is applied to reconstruct thin smoke and ames volumetrically from synchronized multi-video recording. Reconstruction of transparent object with simple geometry in dynamic scene is another interesting research topic. Fountain is the typical object to reconstruct in this area. Wang et al. [82] presented a reconstruction framework to model real water scenes captured by stereoscopic video camera. They introduced physicallybased constraints to rene reconstruction.

The combination of image-based re-

construction with physically-based simulation could model complex and dynamic scene. Their physically-guided modeling also improved robustness in tracking.

4.4 Diuse Optical Diuse optical tomography (DOT) is based on visible or near-infrared (NIR) light scattered across tissue (with thickness about several centimeters). Attenuation of NIR light in tissue is due to absorption eect and scattering eect. Absorption is caused by chromophores of both constant (water), and variable (HbO2 and Hb) concentration. The consequence of scattering is that the optical path length of individual photons does not equal the geometrical source-detector separation. The path length of photon is also wavelength dependent. It is a function of the absorption

µa , the scattering coecients µs , the scattering phase function f (cosθ),

and the tissue-measurement geometry. Thus attenuation measurements alone do not allow quantication of chromophore concentrations. The acquisition methods are shown below. Figure 12 shows three fundamental types of DOT instrumentation (continuous intensity, intensity-modulated and time-resolved). A streak camera with picosecond time resolution can be used to record signal.

Time-resolved and intensity-modulated systems can reconstruct

variations in both optical absorption and scattering, but that unmodulated, nontime-resolved systems are prone to artifact [7]. The expected resolution of the DOT is poor resolving up to 5 mm objects to a depth of 3

±4 cm, such resolution decreasing with increasing depth. Incremental

improvement may come from the use of auxiliary prior knowledge, but it is unlikely that the type of high-resolution images seen in CT or MRI can ever be obtained. One big dierence between NIR ray and x ray transport is scattering. While most of the information is contained in the absorption coecient, the scatter coecient in tissue is generally considerably larger, so that signals measured over

24

4.4 Diuse Optical


distances of a few millimeters or larger are dominated by diuse light. The Radon transform can not be directly applied. Due to the hardness to nd exact reconstruct algorithm for DOT, iterative reconstruction methods are often used. According to reviews of image reconstruction in diuse optical tomography [4] [73], these kind of methods are also called model-based iterative image reconstruction (MOBIIR) schemes.

These schemes

use a forward model to predict detector readings assuming a certain distribution of optical properties inside the medium. The predicted detector readings are compared to actual measurements by dening an appropriate objective function. The objective function is minimized by iteratively updating an initial guess of the distribution of optical properties. MOBIIR schemes in DOT mainly dier in their choice of forward model and in what way the spatial distribution of optical properties is updated. We will explain two popular forward models in details.

4.4.1 Diusion Equation Based Models diuse approximation

These group forward model is based on

(DA) equation. It

is simplied from RTE based on spherical harmonics. Three variable in the RTE depend on direction term

q.

w, the specic intensity I , the phase function p and the source

If these are expanded into spherical harmonics, we obtain an innite series

of equations which approximate to the RTE. The by taking the rst

N

dierential equations.

PN

approximation is obtained 2 spherical harmonics, which gives (N + 1) coupled partial As

N

increases, the

PN

approximation models the RTE

more accurately, but with increasing computational cost.

0 approximation and assume that the phase function p(w , w) 0 is independent of the absolute angle, i.e. p(w , w) = constant, that the photon If we take the

P1

ux changes slowly and that all sources are isotropic. We obtain DA equation

Φ(x, t)

1 ∂Φ − ∇ · Φ(x)∇κ(x, t) + τ (x)Φ(x, t) = q(x, t, w) c ∂t

(56)

Γ(x, t) = −κ(x)∇Φ(x, t)

(57)

is the photon density

Φ(x, t) =

Z

I(r, t, w)d2 w

(58)

wI(r, t, w)d2 w

(59)

4π

Γ(r, t)

is the photon current

Γ(x, t) =

Z 4π

κ(x) =

1 3(τ (x) + σ 0 (x))

(60)

σ 0 (x) = (1 − g)σ(x) is the reduced-scattering coecient. g is anisotropy (0 ≤ g ≤ 1) which is the mean cosine of the scattering phase function [6].

Where factor

The DA equation is applicable if the following criteria:

(i) the measurements

positions are not very close to the source, (ii) the measurement times are not very soon with respect to the input times, and (iii) the absorption coecient is much less than the scattering coecient.

25

4.4 Diuse Optical


The inverse problem can be solved by classical gradient-based solution methods. The form of the Jacobian will depend on three things: the form of the forward model and the nature of the perturbation. Forward models are deterministic based on the diusion approximation DA. The perturbation type will be determined by only two independent parameters: the absorption coecient and the diusion coecient; all other factors, such as the refractive index and scattering anisotropy, are pushed into these two parameters. The nite element method (FEM) can solve the steady-state, time-dependent, and frequency-domain versions of the diusion approximation DA. Arridge [7] developed TOAST(temporal optical absorption and scattering tomography) using FEM in 1997.

It was written in Matlab and available on web.

Time domain

method and frequency domain method can both be used to in FEM to solve DA. Nissilä et al. [30] compared time domain method and frequency domain method. They reported that frequency domain method was observed with higher contrast. It is a good candidate to accelerate in GPU in according to Ryoo [68]. His implementation showed 99% code of FEM can be parallelized from C in CUDA. This number was similar to LBM which they also parallelized in CUDA. They implemented GPU FEM in 2007 spring, using Geforce 8800. They get 10.1X speed up overall in CUDA version 0.8, compared with high optimized CPU library with SSE2 support.

They claimed that kernel speedup compared to a CPU binary

without SIMD support and optimized only for cache usage was on the order of 100X. Dierent from FEM approach, Hielscher [28] used Finite dierence method (FDM) to solve DA equation in 1999. After his initial work, he found DA approximation was limited and switched to the more general and accurate model (RTE) detailed in the following section. One disadvantage of DA is scattering-void area. Arridge et al. attempted to overcome the disadvantages of diusion-equation-based schemes [5].

They pre-

sented a hybrid radiosity-diusion approach for handling non-scattering regions with diusing domains. The method treated light propagation in highly scattering regions with the diusion approximation and used a ray-tracing method for the void areas. However, this approach did not address highly absorbing regions, and it requires a priori knowledge about the exact position of the void-like regions.

4.4.2 Radiative Transfer Equation Based Models Some researches pointed out that the DA approximation was limited. Ren, Bal and Hielscher [67] made a comparison between the DA-equation-based method RTEbased method. They showed RTE-based result was better, however, slower. Klose and Hielscher et al. [43, 45] introduced and experimentally tested an FDM based upwind-dierence discrete ordinates algorithm that computes numerical solutions of the time-independent RTE. Unlike the widely used diusion approximation, this model is capable of accurately describing light propagation in media that contain void-like regions with very low scattering and absorption coecients. The dierence between DA and RTE is the term accounting for the intervoxel scattering. DA uses a simplied to calculate the integration. In stead for RTE, many works in Hielscher's group [41] [44] [1] employed the widely used the Henyey-Greenstein phase function, which is commonly use in graphics.

26

4.5 Microwave


In the same year when Hielscher used FDM to solve DA, Klose and Hielscher et al. [42] rst proposed using FDM method to solve RTE. Later, they found QuasiNewton methods were often more reliable and converge faster than CG methods. Klose and Hielscher et al. [44] applied Quasi-Newton method to the image reconstruction in DOT. In particular, they implemented two known methods as the BroydenFletcherGoldfarbShanno (BFGS) method and the limited-memory BroydenFletcherGoldfarb Shanno (lm-BFGS) method. In conclusion, both the nite dierence method (FDM) and the nite element method (FEM) can be used to solve equations such as the RTE and DA. The nite element method (FEM) is better than FDM in regard to complex geometries and for modeling boundary eects. FEM suers from greater computational overhead and have limitation on scattering-void area.

4.5 Microwave Microwave tomography applies general application of basic CT principals utilizing microwave band. It stems from the high electrical contrasts reported between malignant tumors and normal breast tissue.

Scattering of EM waves in non-

homogeneous human body is however much more complicated than simple attenuation of ionizing radiation. Therefore development of microwave tomography is conditioned by new theoretical approach, optimization of evaluation algorithms and more ecient computer technique. However, this high contrast also increases the diculty of forming an accurate image because of increased multiple scattering [56]. They develop fast forward methods based on the combination of the extended Born approximation, conjugate- and bi-conjugate-gradient methods, and the FFM. They proposed two nonlinear MWI algorithms to improve the resolution for the high-contrast media encountered in microwave breast-tumor detection. In 2003 Zhang et al. [97] reported the full three-dimensional (3-D) forward scattering simulation in order to account for 3-D eects and to provide a fast solver in future 3-D nonlinear inverse scattering methods. The 3-D fast forward method was based on the stabilized bi-conjugate-gradient fast Fourier transform (BCGSFFT) algorithm. The method had been validated for various MWI measurement scenarios. In 2004 Bulyshev et al. [16] presented a new method of solving this inverse problem. Based on the known gradient approach the method had advantages to solve full scale 3D microwave tomographic problems using the vector equations. Their results showed the abilities of the method to reveal the internal structure of objects in the strong contrast case.

4.6 Radiation Therapy Planning Photon can be used in radiation therapy, so does electron.

Photon transport

models are discussed before. But electron transport especially scattering involving kinetic collision requires dierent algorithms. Multiple scattering simulation algorithms come into play. Multiple scattering simulation algorithms can be classied as either "detailed" or "condensed". In the detailed algorithms, all the collisions experienced by the

27

4.6 Radiation Therapy Planning 4 TOMOGRAPHIC RECONSTRUCTION particle are simulated. This simulation can be considered as exact. However, it can be used only if the number of collisions is not too large, a condition fullled only for special geometries (such as thin foils), or low kinetic energies. For larger kinetic energies the average number of collisions is very large and the detailed simulation becomes very inecient. High energy simulation codes use condensed simulation algorithms, in which the global eects of the collisions are simulated at the end of a track segment. The global eects generally computed in these codes are the net displacement, energy loss, and change of direction of the charged particle. These quantities are computed from the multiple scattering theories. Most particle physics simulation codes use the multiple scattering theories of Molière, Goudsmit-Saunderson and Lewis. The theories of Molière and Goudsmit-Saunderson give only the angular distribution after a step, while the Lewis theory computes the moments of the spatial distribution as well. Goudsmit-Sounderson scattering model for a multiple scattering is used in DPM [74], PENELOPE [70] and EGS [29]. Geant4 [2] uses Lewis' MSC theory. In practices, mixed scheme is widely used. The mixed algorithm simulates the "hard" collisions one by one and uses a MSC theory to treat the eects of the "soft" collisions at the end of a given step. PENELOPE , DPM and GEANT4 use a mixed algorithm to simulate electron transport. Weng et al. [86] implemented a SSE vectorized DPM (V-DPM). They reported 1.5 speedup compared to original DPM. Later, Tyagi [79] implemented DPM in CPU cluster and reported almost linear speed up in a 32-core CPU cluster. As for the fastest Monte Carlo simulation for radiotherapy treatment planning (electron or photon), Voxel Monte Carlo (VMC) [38] code was between 50 and 100 times faster than a corresponding EGS. DPM and MCDOSE had not obtained the same speed as the VMC++ code, although MCDOSE's speed was within a factor of 2, as shown in Table 2. 6 MV photon was used to estimate time and 18 MV photon was used to test accuracy [18].

Table 2: Comparison for dierent fast Monte Carlo code from [18]

Monte Carlo code

Time estimate (min)

% mean dierence relative to ESG4/PRESTA/DOSXYZ

EGS4/PRESTA /DOSXYZ VMC++ XVMC MCDOSE MCV DPM

43

0, benchmark caclulation

0.9 1.1 1.6 22 7.3

±1

MCNPX

60

PEREGRINE GEANT4

43 193

±1 ±1 ±1 ±1

Maximum dierence of 8% at Al/lung interface (on average ±1%agreement) ±1 ±1 for homogeneous water and water/air interface

VMC++ used Simultaneous Transport Of Particle Sets (STOPS), a newly

28

4.7 Proton CT


developed variance reduction technique. The motivation for the implementation of STOPS was the desire to achieve the eciency gain due to re-using electron tracks. Several particles that had the same energy but not position, direction, or statistical weight, form a particle set. Both electrons and photons were transported using particle sets. It further improves the same model in DPM by factor of 7.

4.7 Proton CT The need for Proton CT stems from proton radiation therapy. Proton radiation therapy more precise other radiation therapies using electron and photon.

It is

based on the well dened range of protons in material, with low entrance dose, a dose maximum ( Bragg peak ) and a rapid distal dose fall-o, providing better sparing of healthy tissue and allowing higher tumor doses than conventional radiation therapy with photons. At present, the potentials of proton therapy cannot be fully exploited because the conversion of attenuation (Hounseld) values, measured with X-ray computed tomography (CT), to relative electron density values is not always accurate. The resulting uncertainties can lead to range errors from several millimeters up to more than 1 cm depending on the anatomical region treated. Early in 2004, Shulte et al. [72] performed a design study was to dene the optimal approach to a pCT system. Conceptual and detailed design of a pCT system was presented consisting of a silicon-based particle tracking system and a crystal calorimeter to measure energy loss of individual protons. They discussed the formation of pCT images based on the reconstruction of volume electron density maps and the suitability of analytic and statistical algorithms for image reconstruction. In 2006, Petterson et al. [62] reported the results of a beam experiment to develop proton Computed Tomography (pCT). The energy loss permitted calculating the integrated proton stopping power along each proton path from which the electron density distribution could be reconstructed. They described the 2Dimage reconstruction of a low-contrast phantom, derived the relationship between contrast, pixel size, and dose, and study the spatial resolution achievable with this set-up. To test the feasibility of MLP model, some research [15] measured MLP of protons inside an absorber in a beam experiment with high position and angular resolution.

The locations of 200 MeV protons were measured at three dierent

absorber depth and binned in terms of the displacement and the exit angle measured behind the absorber. The observed position distributions were compared to theoretical predictions showing that the location of the protons can be predicted by MLP with an accuracy of better than 0.5 mm. The theoretical MLP prediction (MLP) and associated one

σ

and

2σ

envelopes use the well established Gaussian

approximation of multiple scattering theory, as shown in Figure 13. Li et al. [55] investigated the possible use of the ART with three dierent path-estimation methods for pCT reconstruction.

The rst method assumed a

straight-line path (SLP) connecting the proton entry and exit positions, the second method adapted the most-likely path (MLP) theoretically determined for a uniform medium, and the third method employed a cubic spline path (CSP). Their estimation methods are show in Figure 14. The two-dimensional pCT reconstruction of an elliptical phantom results were compared with GEANT4-based Monte

29

4.8 Ultra Sound


Figure 13: MLP from [15]

Carlo simulations. The ART reconstructions showed progressive improvement of spatial resolution when going from the SLP to the curved CSP and MLP path estimates. The MLP-based ART algorithm had the fastest convergence and smallest residual error of all three estimates. They suggested the tracking curved proton paths in conjunction with the ART algorithm. However, The fast convergence rate was according to per iteration, not for time. They reported 12h to reconstruct a 2D image.

Figure 14: SLP,CSP,MLP and Monte Carlo from [55]

4.8 Ultra Sound In diagnostic ultrasound imaging, high frequency pulses of acoustic energy are emitted into the patients' body where they experience reection at boundaries between tissues of dierent characteristic impedance. From the measurement of time delay and intensity of the reected pulses (echoes), an image indicating tissue interfaces can be reconstructed. Ultrasound imaging involves negligible risk, since the incident intensities are very small. The relatively simple technology employed makes it rather inexpensive compared to other clinical imaging modalities. The spatial resolution depends on many factors and is typically several millimeters. Acoustic frequencies typically range from 1 to 15 MHz. We can model this problem into matrix and solve system by conjugatedgradient method.

Essentially, sparse matrix solver-based method and Fourier-

space methods do not model the ray physics in a direct manner, but through some intermediate representation, which cannot easily capture the local interactions of

30

4.8 Ultra Sound


the ray with the medium. On the other hand, a complete simulation of the wave equation is prohibitively expensive. Based on Eikonal equation, Li [54] used fast marching method (FMM) to solve the refraction eect.

Fast marching method is a classical algorithm for

identifying the shortest path in a network of links.

The FMM is a single-pass,

upwind nite dierence scheme. It produces the correct viscosity solution to the Eikonal equation. Li's initial FFM-SART algorithm is based on CPU with sequential execution. In 2008, Li [53] implemented Fast Sweeping Method (FSM) and Fast Iterative Method (FIM) in order to better parallelized the FMM algorithm,. These Eikonal solvers' performance were compared on single CPU, CPU clusters and GPU. It achieved 100X speedup in FIM.

31

5 CURRENT AND FUTURE WORK

5

Current and Future Work

The current work mainly focuses on how to improve reconstruction quality in rather simplied forward models. Without consider scattering, assume straightline photon transport, the FBP algorithm is the fast solution to reconstruct 3D object from X-ray projection. In this simple model, we implemented several methods to achieve high quality rendering, to recover detail loss in the rendering procedure. Detail preservation in slice and volumetric data generated by medical and industrial Computed Tomography (CT) is a topic of great interest. These details can be hairline fractures, small pathological features such as tumors in medical scans, and textures of diagnostic value.

A recent trend has been to integrate

the visualization process more closely into the process that generated the data. In this way, precision losses incurred in the visualization procedure can be readily accounted for at the time the data is generated. In essence, the problem lies in the compression of the continuous representation maintained by the generating process into a discretized representation used in the subsequent, decoupled visualization step. The consequence can be simple blurring, but also the omission of ne object detail. Rautek [66] argued that it is the concatenation of interpolations that leads to artifacts in subsequent renderings of these volumes, which can be avoided by the proposed direct approach (termed Direct DVR (D2VR), where DVR stands for Direct Volume Rendering). However, D2VR approach suers from its large overhead. Typically computing an image of N2 pixels requires O(N4) interpolations, while DVR involves O(N3). While clever bounding box and ray termination schemes, coupled with GPU acceleration can accelerate D2VR to speeds of around 1 fps for moderate datasets, this is still not acceptable if the goal is to support more complex reconstruction algorithms, which will bring this frame rate down substantially. One simple solution would be to reconstruct the volume at a uniformly higher resolution.

It

would increase the required memory bandwidth of the rendering process, which on GPU is a precious resource. In order to overcome these constraints, we propose a hierarchical representation of data. The base resolution is volume grid people commonly use. Given a tolerance, some cells are classied as subdivision cells which contain ne details will be presented by more data sample points. the data generation process.

We incorporate this eort into

D2VR reconstructs the density values ad-hoc and

on-the-y, but it does not cache the results.

Instead, we construct the density

values as a pre-process and load it on demand. We derive a high-resolution data directly from the reconstruction process, and generate a compressed volume from acquired raw data within tight tolerances. This compressed data is an adaptive octree structure can be easily evaluated both for slice-based viewing as well as in 3D volume rendering, oering excellent detail preservation in zooming operations.

Our octree is optimized for the sparse nature of the ne details in CT

data. It is also constructed carefully to remove discontinuous transitions across the ne-coarse cells boundaries.

The octree traversal is accelerated in GPU. It

runs on interactive frame rate and shows great improvement on ne details. The framework is ready to support more complex reconstruction algorithms.

32

5.1 Frequency Domain Upsampling


5.1 Frequency Domain Upsampling The X-ray attenuation is currently simulated by the ray tracing method. It only support FBP reconstruction now but can be extended to iterative algorithm easily. A ramp lter is applied in frequency space. To reconstruct the details after upsampling, the order of upsampling and ramp-lter matters. Theoretically they are equal to each other but results show lter-upsampling procedure can get correct result but upsampling-lter procedure will have artifact. The failure of latter approach is because ideal upsampling is not possible.

Commonly used bilinear

or bi-cubic interpolation lter in upsampling will introduce some high frequency noise. This high frequency is severely magnied by later use of ramp-lter. Thus we choose to rst apply the ramp-lter then upsampling. For the same reason, we should also carefully choose the interpolation lter used in upsampling. Even applied after the ramp-lter, the upsampling using bilinear lter or bicubic lter can introduce sampling artifact. Frequency space upsampling is the necessary method to ensure the smoothness of upsampling. we apply the same method as proposed by Li and Mueller [50]. We move the signal band in Fourier-domain and recover it into high resolution space domain. Then the frequency leakage can be remedied by applying Welch lter.

5.2 Adaptive Renement Traditional bricking method is popular to represent hierarchical volume. It usually uses a bounding box to represent the region to rene. However, bricking's simple box-shape does not t to the CT reconstruction pipeline well. Compared to high resolution gold standard, the coarse volume can represent the data fairly good only need a few place need to rene.

The renement regions are usually near

high frequency which is the sharp edges.

First this high frequency has a line-

shape with is sparse in nature. Second, these edges occur often across the whole volume which make brick-clipping method very inecient. Then a ne granular hierarchical representation is better suiting CT reconstruction. A lot of research in octree-based multi-resolution framework [26, 49, 57]. Most multi-resolution approaches render each brick individually storing them in dierent textures. Their approach, however, does not support mixing of dierent resolution levels.

Recently, Beyer [13] introduce a blending scheme to render mixed

resolution.

The potential T junction problem was removed by shift the sam-

ple near high-low-resolution-boundary by half sampling distance. The appearing triangle or trapezoid regions then blended by distorted interpolation. proach does not give explicit error control.

This ap-

Since The renement cell position

is grouped in coarse granular octree, it is inecient to handle sparse renement regions. Kaehler [36, 37] developed adaptive mesh renement scheme for cosmological data. His method showed better compression than octree when was used to render very large and sparse scene. However, their renement cells are limited to convex shape, which is often not true for renement cell in medical data. We adopted more general ne granular cell into our adaptive renement data structure, with the cost of an 3D lookup texture. And l1 estimator is performed to determine the local sampling rate. The upsampled reconstruction data is made as gold standard. Given the trilinear lter and a coarse level sample, if the reconstructed signal error compared to gold standard is larger than threshold, then we

33

5.3 Gaussian Transfer Function


subdivide current cell and increase the sampling rate by two. The whole process is integrated with rendering in GPU, making it a unied framework for high quality rendering.

Figure 15: Subdivision Traditionally bricking method usually stores additional sample near the brick boundary, otherwise T junctions will appear at the ne-coarse brick boundaries. Renement cell face the same problem. The way to ensure C1-continuity across the cell boundary is downgrade ne cell's neighbor to a semi ne cell. Within this frame work, we can implement merge routine to save more space.

Figure 16: Merge With the research in hexagonal lattice [2527]. Mixed lattice structure could further increase the sampling eciency. There is no reason to prevent using our adaptive renement framework in lattice.

5.3 Gaussian Transfer Function In volume rendering, transfer function sampling could also introduce artifact. Transfer function is commonly stored as dependent texture.

Float point pre-

integration table can remove precision loss incurred from largely-changing transferfunction. However, the transfer function shape's resolution is still low (255x255)

34

5.4 Lattice-based Fast Scattering


and incapable to capture very shin iso-surface. We implemented Gaussian based transfer function [47]. Here is visualization UI.

Figure 17: A prototype of volume rendering interface

Figure 17 is a framework we developed to for volume rendering. It is going to be extended for mixed lattice rendering.

5.4 Lattice-based Fast Scattering We are planing to use a fast MC method to simulate scattering. to use lattices for rendering participating media.

We propose

First we sample the media in

hexagonal closest packing (HCP) lattices. Like photon mapping in participating media, our algorithm for rendering participating media has two passes. In the rst pass, photons are emitted from light sources and the photons energy is distributed in the scene. In the second pass, ray casting is used to collect the photon energy. When a photon is emitted from a light source, the nearest lattice site to the rst hit point on the lattice boundary will be calculated and the moving direction of the photon will be discretized to be one of the link directions. The photon will be traced on the links between lattice sites. When arriving at a lattice site

xi

from

wi, j it might be absorbed with probability sva or scattered with probability svs(xi, wi, j), otherwise it will continue moving through link wi,12=j . As discussed in Section 3.2.3, the scattering direction is decided by the phase function p(x, w´, w). link

In HCP lattice, each site only has 12 links. Thus the photon directions are discrete, the phase function

f (x, w´, w)

can be discretized as

f (xi , wi,j , wi , k),

where

wi,k is

the scattering link. The interaction between photons and lattice sites is stored in the volume photon map. Due to the regular pattern of HCP lattices, the photons can be stored in compact and ecient 3D arrays. The photon direction is simply the incoming link index and the photon location is implied by the lattice site index. We can develop ray casting algorithms for the second rendering pass.

Ray

casting casts rays into the HCP lattice and collects radiance values along rays. Compared with photon mapping, the most time-consuming step of the original photon mapping is the calculation of scattering, which is simply a table lookup operation in our method.

35

6 CONCLUSION

6

Conclusion

In this report, we systematically reviewed rendering methods in computer graphics. We analyzed both local illumination and global illumination in order to generalize simulation from visible light to other wavelength. We also gave an overview of the existing tomographic technique over electromagnetic wavelength together with Proton CT and ultra sound CT. To achieve high quality reconstruction, we built an X-ray projector and an adaptive reconstructor, decoupled with an octree framework supporting mixed resolution rendering. In addition, we are currently developing an accelerated Monte Carlo type scattering projector for photon in broad waveband. We also expect to have LBM included in framework.

36

REFERENCES

REFERENCES

References [1] GS Abdoulaev and AH Hielscher.

Three-dimensional optical tomography

with the equation of radiative transfer.

Journal of Electronic Imaging, 12:594,

2003. [2] S Agostinelli, J Allison, K Amako, J Apostolakis, H Araujo, P Arce, M Asai, D Axen, S Banerjee, and G Barrand. GEANT4a simulation toolkit.

Inst. and Methods in Physics Research, A, 506(3):250303, 2003.

Nuclear

[3] E Alerstam, T Svensson, and S Andersson-Engels. Parallel computing with graphics processing units for high-speed Monte Carlo simulation of photon migration.

Journal of Biomedical Optics, 13:060504, 2008.

[4] SR Arridge. Optical tomography in medical imaging.

Inverse problems, 15:41

41, 1999. [5] SR Arridge, H Dehghani, M Schweiger, and E Okada.

The nite element

model for the propagation of light in scattering media: a direct method for

Medical Physics, 27:252, 2000.

domains with nonscattering regions. [6] SR Arridge and M Schweiger.

Photon-Measturement density function. II:

nete-element-method calculations.

Applied Optics, 34(34):80268037, 1995.

[7] SR Arridge and M Schweiger. Image reconstruction in optical tomography.

Philosophical Transactions: Biological Sciences, 352(1354):717726, 1997.

[8] MR Ay, M Shahriari, S Sarkar, M Adib, and H Zaidi. Monte Carlo simulation of x-ray spectra in diagnostic radiology and mammography using MCNP4C.

Physics in Medicine and Biology, 49(21):48974918, 2004.

[9] MR Ay and H Zaidi. Development and validation of MCNP4C-based Monte Carlo simulator for fan-and cone-beam x-ray CT.

Biology, 50(20):48634886, 2005.

Physics in Medicine and

[10] S. Bergner, T. Möller, M. S. Drew, and G. D. Finlayson. Interactive spectral volume rendering. In

Visualization, 2002. VIS 2002. IEEE,

pages 101108,

2002. [11] S Bergner, T Möller, M Tory, and MS Drew. A practical approach to spec-

IEEE Transactions on Visualization and Computer Graphics, 11(2):207216, 2005.

tral volume rendering.

[12] Steven Bergner, Mark S. Drew, Torsten Möller246, and ller. A tool to create illuminant and reectance spectra for light-driven graphics and visualization.

ACM Trans. Graph., 28(1):111, 2009.

1477931.

[13] J Beyer, M Hadwiger, T Möller, and L Fritz. Smooth Mixed-Resolution GPU Volume Rendering.

2008.

IEEE/EG International Symposium on Volume

and Point-Based Graphics.

37

REFERENCES

REFERENCES

[14] JF Briesmeister. MCNP TM A General Monte Carlo N-Particle Transport

Version 4A, Los Alamos National Laboratory Manual LA-12625-M, Los Alamos National Laboratory, Los Alamos, New Mexico, USA, 1993. Code.

[15] M Bruzzi, N Blumenkrantz, J Feldt, J Heimann, FWS Hartmut, A Seiden, DC Williams, V Bashkirov, R Schulte, and D Menichelli. Prototype Tracking Studies for Proton CT.

IEEE Transactions on Nuclear Science,

54(1 Part

1):140145, 2007. [16] AE Bulyshev, AE Souvorov, SY Semenov, VG Posukh, and YE Sizov. Threedimensional vector microwave tomography: theory and computational experiments.

Inverse problems, 20(4):12391259, 2004.

[17] Brian Cabral, Nancy Cam, and Jim Foran. Accelerated volume rendering and tomographic reconstruction using texture mapping hardware, 1994. 197972 91-98. [18] IJ Chetty, B Curran, JE Cygler, JJ DeMarco, G Ezzell, BA Faddegon, I Kawrakow, PJ Keall, H Liu, and CMC Ma. Group No. 105:

Report of the AAPM Task

Issues associated with clinical implementation of Monte

Carlo-based photon and electron external beam treatment planning.

ical Physics, 34:4818, 2007.

[19] AP Colijn and FJ Beekman. scatter projections.

Med-

Accelerated simulation of cone beam X-ray

IEEE Transactions on medical imaging,

23(5):584590,

2004. [20] AP Colijn, W Zbijewski, A Sasov, and FJ Beekman. Experimental validation of a rapid Monte Carlo based micro-CT simulator.

Biology, 49(18):43214333, 2004.

[21] H De Jong, ETP Slijpen, and FJ Beekman.

Physics in Medicine and

Acceleration of Monte Carlo

SPECT simulation using convolution-based forced detection.

tions on Nuclear Science, 48(1):5864, 2001.

IEEE Transac-

[22] J. F. El Hajjar, S. Marchesin, J. M. Dischler, and C. Mongenet. order pre-integrated volume rendering.

PacicVIS '08., pages 916, 2008.

In

[23] K Engel, M Hadwiger, C Rezk-Salama, and JM Kniss.

graphics.

Second

Visualization Symposium, 2008.

Real-time volume

AK Peters Ltd, 2006.

[24] K Engel, M Kraus, and T Ertl. High-quality pre-integrated volume rendering using hardware-accelerated pixel shading. pages 916. ACM New York, NY, USA, 2001. Proceedings of the ACM SIGGRAPH/EUROGRAPHICS workshop on Graphics hardware. [25] A Entezari, R Dyer, and T Möller. Linear and cubic box splines for the body centered cubic lattice.

pages 1118. IEEE Computer Society Washington,

DC, USA, 2004. Proceedings of the conference on Visualization'04.

38

REFERENCES

REFERENCES

[26] A Entezari, T Meng, S Bergner, and T Möller. sional multiresolution transform.

A granular three dimen-

pages 267274, 2006.

Proceedings of

Eurographics/IEEE-VGTC Symposium on Visualization. [27] A Entezari, D Van De Ville, and T Möller. Practical box splines for reconstruction on the body centered cubic lattice.

IEEE Transactions on Visual-

ization and Computer Graphics, 14(2):313328, 2008.

[28] AH Hielscher, AD Klose, and KM Hanson. Gradient-based iterative image reconstruction scheme for time-resolved optical tomography.

tions on medical imaging, 18(3):262271, 1999.

IEEE Transac-

[29] H Hirayama, Y Namito, WR Nelson, AF Bielajew, SJ Wilderman, and U Michigan.

The EGS5 code system.

United States. Dept. of Energy, 2005.

[30] Nissilä I, JC Hebden, D Jennions, J Heino, M Schweiger, K Kotilahti, T Noponen, A Gibson, S Jävenpää, and L Lipiäinen. Comparison between a timedomain and a frequency-domain system for optical tomography.

Biomedical Optics, 11:064015, 2006.

Journal of

[31] I Ihrke, KN Kutulakos, HPA Lensch, M Magnor, and W Heidrich. Transparent and Specular Object Reconstruction. 2008. [32] I Ihrke and M Magnor. Adaptive grid optical tomography.

Graphical Models,

68:484495, 2006. [33] I Ihrke, G Ziegler, A Tevs, C Theobalt, M Magnor, and HP Seidel. Eikonal rendering: ecient light transport in refractive objects. ACM New York, NY, USA, 2007. International Conference on Computer Graphics and Interactive Techniques. [34] HW Jensen. Global illumination using photon maps.

Rendering Techniques,

96:2130, 1996. [35] HW Jensen and PH Christensen.

Ecient simulation of light transport in

scences with participating media using photon maps. pages 311320. ACM New York, NY, USA, 1998.

Proceedings of the 25th annual conference on

Computer graphics and interactive techniques. [36] R Kähler, M Simon, and HC Hege.

Interactive volume rendering of large

sparse data sets using adaptive mesh renement hierarchies.

IEEE Transac-

tions on Visualization and Computer Graphics, 9(3):341351, 2003.

[37] R Kähler, J Wise, T Abel, and HC Hege. GPU-assisted raycasting for cosmological adaptive mesh renement simulations. In

Volume Graphics, pages

103110, 2006. [38] I Kawrakow. VMC++, electron and photon Monte Carlo calculations optimized for Radiation Treatment Planning. 2000. Advanced Monte Carlo for Radiation Physics, Particle Transport Simulation and Applications. Proceedings of the Monte Carlo 2000 Conference. Vaz. Berlin Heidelberg: SpringerVerlag, 2001., p. 229.

39

REFERENCES

REFERENCES

[39] A Keller. Instant radiosity. Association for Computing Machinery, Inc, One Astor Plaza, 1515 Broadway, New York, NY, 10036-5701, USA, 1997.

In-

ternational Conference on Computer Graphics and Interactive Techniques: Proceedings of the 24 th annual conference on Computer graphics and interactive techniques. [40] M Khodaverdi, AF Chatziioannou, S Weber, K Ziemons, H Halling, and U Pietrzyk.

Investigation of dierent microCT scanner congurations by

GEANT4 simulations.

IEEE Transactions on Nuclear Science,

52(1 Part

1):188192, 2005. [41] HK Kim, UJ Netz, J Beuthan, and AH Hielscher. Optimal source-modulation frequencies for transport-theory-based optical tomography of small-tissue volumes.

Optics Express, 16(22):1808218101, 2008.

[42] AD Klose and AH Hielscher.

Iterative reconstruction scheme for optical

tomography based on the equation of radiative transfer.

Medical Physics,

26:1698, 1999. [43] AD Klose and AH Hielscher. Optical tomography using the time-independent equation of radiative transfer: Part 2: inverse model.

Journal of Quantitative

Spectroscopy and Radiative Transfer, 72(5):715732, 2002.

[44] AD Klose and AH Hielscher. Quasi-Newton methods in optical tomographic image reconstruction.

Inverse problems, 19(2):387409, 2003.

[45] AD Klose, U Netz, J Beuthan, and AH Hielscher. Optical tomography using the time-independent equation of radiative transfer. Part 1: Forward model.

Journal of Quantitative Spectroscopy and Radiative Transfer, 72(5):691713, 2002. [46] J Kniss, S Premoze, M Ikits, A Lefohn, and C Hansen. Closed-form approximations to the volume rendering integral with gaussian transfer functions. Technical report, Technical Report UUCS-03-013, School of Computing, University of Utah, 2003. [47] Joe Kniss, Simon Premoze, Milan Ikits, Aaron Lefohn, Charles Hansen, and Emil Praun. Gaussian transfer functions for multi-eld volume visualization, 2003. 1081509 65. [48] Y Kyriakou, T Riedel, and WA Kalender. Combining deterministic and Monte Carlo calculations for fast estimation of scatter intensities in CT.

Medicine and Biology, 51(18):45674586, 2006.

Physics in

[49] E LaMar, B Hamann, and KI Joy. Multiresolution techniques for interactive texture-based volume visualization.

pages 355361. IEEE Computer Soci-

ety Press Los Alamitos, CA, USA, 1999. Proceedings of the conference on Visualization'99: celebrating ten years. [50] A Li, K Mueller, and T Ernst.

Methods for ecient, high quality volume

resampling in the frequency domain. Washington, DC, USA, 2004.

pages 310. IEEE Computer Society

Proceedings of the conference on Visualiza-

tion'04.

40

REFERENCES

REFERENCES

[51] S Li and K Mueller. volume graphics. In

Accelerated, high-quality refraction computations for

Volume Graphics, pages 73229, 2005.

[52] S Li and K Mueller. Spline-based gradient lters for high-quality refraction computation in discrete datasets.

EuroVis 2005, 2005.

[53] S Li, K Mueller, M Jackowski, D Dione, and L Staib.

Physical-space

refraction-corrected transmission ultrasound computed tomography made computationally practical. berg, 2008.

pages 280288. Springer-Verlag Berlin, Heidel-

Proceedings of the 11th International Conference on Medical

Image Computing and Computer-Assisted Intervention, Part II. [54] S Li, K Mueller, M Jackowski, DP Dione, and LH Staib.

Fast marching

method to correct for refraction in ultrasound computed tomography. pages 896899. Citeseer, 2006. IEEE Symp. Biomed. Img. [55] T Li, Z Liang, JV Singanallur, TJ Satogata, DC Williams, and RW Schulte. Reconstruction for proton computed tomography by tracing proton trajectories: a Monte Carlo study.

Medical Physics, 33:699, 2006.

[56] QH Liu, ZQ Zhang, TT Wang, JA Bryan, GA Ybarra, LW Nolte, and WT Joines. Active microwave imaging. I. 2-D forward and inverse scatteringmethods.

IEEE Transactions on Microwave Theory and Techniques, 50(1

Part 1):123133, 2002. [57] P. Ljung, C. Lundstrom, A. Ynnerman, and K. Museth.

Transfer function

based adaptive decompression for volume rendering of large medical data sets. In

Volume Visualization and Graphics, 2004 IEEE Symposium on, pages 25

32, 2004. [58] WE Lorensen and HE Cline. Marching cubes: A high resolution 3D surface construction algorithm.

pages 163169. ACM New York, NY, USA, 1987.

Proceedings of the 14th annual conference on Computer graphics and interactive techniques. [59] S Matej, S Surti, S Jayanthi, M Daube-Witherspoon, R Lewitt, and J Karp. Ecient 3D TOF PET reconstruction using view-grouped histo-images: DIRECT-direct image reconstruction for TOF.

cal imaging, 2009.

[60] William W. Moses.

IEEE Transactions on medi-

Recent advances and future advances in time-of-ight

Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment, 580(2):919

PET.

924, 2007. [61] HJ Noordmans, HTM van der Voort, and AWM Smeulders. ume rendering.

Spectral vol-

IEEE Transactions on Visualization and Computer Graphics,

pages 196207, 2000. [62] M Petterson, N Blumenkrantz, J Feldt, J Heimann, D Lucia, A Seiden, DC Williams, HFW Sadrozinski, G Cuttone, and DL Presti. diography Studies for Proton CT. 2006.

41

Proton Ra-

REFERENCES

REFERENCES

[63] G. Pratx, G. Chinn, F. Habte, P. Olcott, and C. Levin. Fully 3-D list-mode

Nuclear Science Symposium Conference Record, 2006. IEEE, volume 4, pages 21962202, 2006. OSEM accelerated by graphics processing units. In

[64] G Pratx, G Chinn, PD Olcott, and CS Levin.

Fast, accurate and shift-

varying line projections for iterative reconstruction using the GPU.

Transactions on medical imaging, 28(3):435445, 2009.

[65] F Qiu, F Xu, Z Fan, N Neophytos, A Kaufman, and K Mueller. based volumetric global illumination.

IEEE

Lattice-

IEEE Transactions on Visualization

and Computer Graphics, 13(6):15761583, 2007.

[66] P Rautek, B Cséfalvi, S Grimm, S Bruckner, and E Groller. D 2 VR: Highquality volume rendering of projection-based volumetric data.

2006.

Joint

Eurographics-IEEE TCVG Symposium on Visualization. [67] K Ren, G Bal, and AH Hielscher. cal tomography in small domains:

Transport-and diusion-based opti-

Applied Optics,

a comparative study.

46(27):66696679, 2007. [68] S Ryoo.

sors.

Program optimization strategies for data-parallel many-core proces-

PhD thesis, 2008.

[69] P Sabella.

A rendering algorithm for visualizing 3D scalar elds.

SIGGRAPH Computer Graphics, 22(4):5158, 1988.

ACM

[70] F Salvat, JM Fernández-Varea, and J Sempau. PENELOPE, a code system for Monte Carlo simulation of electron and photon transport. pages 9264, 2003. Workshop Proceedings, Issy-les-Moulineaux, France, ISBN.

Development of a time-resolved optical tomography system for neonatal brain imaging. PhD thesis, 2000.

[71] FEW Schmidt.

[72] R Schulte, V Bashkirov, T Li, Z Liang, K Mueller, J Heimann, LR Johnson, B Keeney, HFW Sadrozinski, and A Seiden. Conceptual design of a proton computed tomography system for applications in proton radiation therapy.

IEEE Transactions on Nuclear Science, 51(3 Part 3):866872, 2004.

[73] M Schweiger, A Gibson, and SR Arridge. Computational aspects of diuse optical tomography.

Computing in Science and Engineering, 5(6):3341, 2003.

[74] J Sempau, SJ Wilderman, and AF Bielajew. DPM, a fast, accurate Monte Carlo code optimized for photon and electron radiotherapy treatment planning dose calculations.

Physics in Medicine and Biology,

45(8):22632292,

2000. [75] A. Sitek, G. El Fakhri, Ouyang Jinsong, and J. S. Maltz.

Fast analytical

modeling of compton scatter using point clouds and graphics processing unit (GPU). In

Nuclear Science Symposium Conference Record, 2007. NSS '07.

IEEE, volume 6, pages 45464548, 2007.

42

REFERENCES

REFERENCES

[76] A Sitek, RH Huesman, and GT Gullberg. Tomographic reconstruction using an adaptive tetrahedral mesh dened by a point cloud.

on medical imaging, 25(9):11721179, 2006.

[77] X Sun, K Zhou, E Stollnitz, J Shi, and B Guo.

IEEE Transactions

Interactive relighting of

dynamic refractive objects. ACM New York, NY, USA, 2008. International Conference on Computer Graphics and Interactive Techniques. [78] SJ Tu, CC Shaw, and L Chen. Noise simulation in cone beam CT imaging with parallel computing.

Physics in Medicine and Biology, 51(5):12831298,

2006. [79] N Tyagi, A Bose, and IJ Chetty. Implementation of the DPM Monte Carlo code on a parallel architecture for treatment planning applications.

Physics, 31:2721, 2004.

Medical

[80] I Wald, T Kollig, C Benthin, A Keller, and P Slusallek. Interactive global illumination using fast ray tracing.

pages 1524. Eurographics Association

Aire-la-Ville, Switzerland, Switzerland, 2002. Proceedings of the 13th Eurographics workshop on Rendering. [81] Bruce Walter, Sebastian Fernandez, Adam Arbree, Kavita Bala, Michael Donikian, and Donald P. Greenberg. lumination.

Lightcuts: a scalable approach to il-

ACM Trans. Graph., 24(3):10981107, 2005.

1073318.

[82] H Wang, M Liao, Q Zhang, R Yang, and G Turk. Physically Guided Liquid Surface Modeling from Videos. 2009. [83] L Wang, SL Jacques, and L Zheng. light transport in multi-layered tissues.

in Biomedicine, 47(2):131146, 1995.

MCML Monte Carlo modeling of

Computer Methods and Programs

[84] R Wang, K Zhou, M Pan, and H Bao. An ecient GPU-based approach for interactive global illumination. 2009. [85] GJ Ward, FM Rubinstein, and RD Clear. A ray tracing solution for diuse interreection. pages 8592. ACM New York, NY, USA, 1988. Proceedings of the 15th annual conference on Computer graphics and interactive techniques. [86] X Weng, Y Yan, H Shu, J Wang, SB Jiang, and L Luo. A vectorized Monte Carlo code for radiotherapy treatment planning dose calculation.

Medicine and Biology, 48(7):111120, 2003.

Physics in

[87] R Westermann and B Sevenich. Accelerated volume ray-casting using texture mapping.

pages 271278. IEEE Computer Society Washington, DC, USA,

2001. Proceedings of the conference on Visualization'01. [88] F Xu and K Mueller. Towards a unied framework for rapid 3D computed tomography on commodity GPUs. volume 4, 2003. 2003 IEEE Nuclear Science Symposium Conference Record.

43

REFERENCES

REFERENCES

[89] F Xu and K Mueller.

Ultra-fast 3D ltered backprojection on commodity

graphics hardware. pages 571574, 2004. IEEE International Symposium on Biomedical Imaging: Nano to Macro, 2004. [90] F Xu and K Mueller. Accelerating popular tomographic reconstruction algorithms on commodity PC graphics hardware.

Science, 52(3 Part 1):654663, 2005. [91] F Xu and K Mueller.

IEEE Transactions on Nuclear

A comparative study of popular interpolation and

integration methods for use in computed tomography.

2006.

IEEE 2006

International Symposium on Biomedical Imaging (ISBI06). [92] F Xu and K Mueller. Applications of optimal sampling lattices for volume acquisition via 3D computed tomography. In

Volume Graphics,

page 57?3,

2007. Volume Graphics Symposium. [93] F Xu and K Mueller. GPU-Acceleration of Attenuation and Scattering Compensation in Emission Computed Tomography, 2007. [94] F Xu and K Mueller.

Real-time 3D computed tomographic reconstruc-

tion using commodity graphics hardware.

Physics in Medicine and Biology,

52(12):34053420, 2007. [95] F Xu, K Mueller, M Jones, B Keszthelyi, J Sedat, and D Agard. On the eciency of Iiterative ordered subset reconstruction algorithms for acceleration on GPUs, 2008. [96] H Zaidi and MR Ay. Current status and new horizons in Monte Carlo simulation of X-ray CT scanners.

puting, 45(9):809817, 2007.

Medical and Biological Engineering and Com-

[97] ZQ Zhang, QH Liu, C Xiao, E Ward, G Ybarra, and WT Joines. Microwave breast imaging: 3-D forward scattering simulation.

Biomedical Engineering, 50(10):11801189, 2003. [98] K Zhou, Q Hou, R Wang, and B Guo. graphics hardware.

IEEE Transactions on

Real-time kd-tree construction on

ACM Trans. Graph, 27, 5, 126:111, 2008.

44