Composed scattering model for direct volume rendering

Vo1.11 No.5 J. of C o m p u t . Sci. & Technol. S e p t e m b e r 1996

Composed Scattering Model for Direct Volume Rendering

~ t Cai Wenli ( ~ . ~ ) and Shi J i aoy ing ( ; H ~ )

State Key Lab of CAD & CG, Zhejiang University, Hangzhou 310027 t Email: [email protected]

Received October 23, 1995; revised March 20, 1996.

A b s t r a c t

Based on the equation of transfer in transport theory of optical physics, a new volume rendering model, called composed scattering model (CSM), is presented. In calculating the scattering term of the equation, it is decomposed into volume scattering intensity and surface scattering intensity, and they are composed with the boundary detection operator as the weight function. This proposed model differs from the most current volume rendering models in the aspect that in CSM segmentation and illumination intensity calculation are taken as two coherent parts while in existing models they are regarded as two separate ones. This model has been applied to the direct volume, rendering of 3D data sets obtained by CT and MRI. The resultant images show not only rich details but also clear boundary surfaces. CSM is demonstrated to be an accurate volume rendering model suitable for CT and MRI data sets.

Keywords : Volume rendering, volume illumination model, ray casting, voxel, equation of transfer.

1 I n t r o d u c t i o n

Currently, the display of 3D data sets, especially 3D scalar da ta set from CT and MRI, is a main research direction in scientific visualization. The methods in- clude contour extraction or reconstruction [1-3], voxel-based rendering under surface i l lumination model[4,5], and direct volume rendering under volume il lumination model [6-12]. Due to its high fidelity, direct volume rendering at t racts more research attentions and becomes the highlight of the research direction in scientific visualization.

Most existing volume rendering techniques imply the fact tha t segmentat ion and illumination intensity calculation are two separate processes. The interesting parts in the da ta set are first segmented using image segmentation operators [4'5] and then illumination intensity is calculated using illumination model. The all-or-none segmentat ion me thod is a kind of alternative method which not only loses the rich

This research was supported by the National Natural Science Foundation of China. This paper was reported at the International Workshop on Virtual Reality and Scientific Visualization in Hangzhou, China, in April 1995.

434 J. of Comput. Sci. & Technol_ Vol. 11

details in data sets but also is the main error source of artifacts. A lot of boundary surfaces are exaggerated or ignored. In order to avoid this kind of segmentation, DrebinF] proposed the material classification and mixture model based on proba- bility theory. The percentage of each material in one voxel is calculated according to Bayesian estimation equation. Different materials are assigned to different colors and opacities. The color and opacity of each voxel are mixed up by its material percentages. However, this kind of maximum .likelihood segmentation requires that no more than two material distributions overlap. For example, according to the X-ray absorption of each material, the material distribution order in CT intensity data is air, fat, soft tissue and bone. Drebin's model can only calculate the linear combination of two successively distributed materials, such as the mixture of air and fat, or the mixture of fat and soft tissue, or the mixture of soft tissue and bone. It cannot calculate the percentages of two discrete materials, such as air and bone.

With the all-or-none criterion, alternative segmentation supposes that there is a strict boundary surface which divides the volume into two parts. It means that after segmentation, each voxel is determined to belong to the inside or the outside of the boundary surface. But in most realistic cases, the boundary surface occupies a certain thickness and the density varies gradually within the boundary surface. The strength of boundary surface is the absolute value of density gradient at different points within the boundary surface and equivalent to the coefficient of surface illumination intensity. Volume rendering in CT, MRI and other data sets requires accurate display of the boundary surface, including its strength and position, as well as the details within the boundary surface.

Neither alternative segmentation nor Drebin's model can accurately render this gradually changing effects. Drebin's model uses different materials' percentages to produce the mixture effect of different materials. The boundary surface is positioned by the combination of different materials' percentages in a-voxel. But in a well-

distributed mixture, i.e. homogeneous mixture, Drebin's mode[ fails. For example, in the mixture of red ink and water no boundary surface exists although each voxel is a mixture of two materials. Thus, it is very difficult to satisfy the requirement of volume rendering only by using the segmentation. Prom this point of view, our model introduces the value of boundary detection operator as the strength of boundary surface and takes it into the account of illumination intensity calculation of direct volume rendering. This leads to our composed scattering model (CSM).

The early work on direct volume rendering is the rendering of random medium, especially on gas objects, such as cloud, fog, mist and vapor. Blinn [tal, Kajiya N, Max [141 and Sakas [121 studied the attenuation of rays crossing through the random medium. Their major focuses are on single scattering, multiple scattering, and different scattering phase functions. Their researches resulted in a series of random medium rendering models and algorithms aiming at blur boundaries like the images of cloud and fog. This kind of model is called source attenuation model.

In fact, the data field obtained by the no-destructive evaluation (NDE) devices such as CT and MRI is a kind of random medium density field. But it requires a relatively clear boundary compared with the cloud and fog images. The equation of

No. 5 Composed Scattering Model for Direct Volume Rendering 435

transfer in the transport theory of optical physics is a precise equation to calculate the radiative intensity difference during the period when the ray passes through the continuous random medium. Krueger [161 reported the earliest work in which he presented an accurate direct volume rendering algorithm based on the equation of transfer. Monte Carlo random simulation method, the traditional method used to solve the equation of transfer, was employed to convert the equation into the Neumann series. Different forms of the integral term K in the Neumann series were discussed to show their effects on displaying the features of the data field. In theory, Krueger introduced the volume term, surface term, point term and their usual forms. In addition, Krueger's work is considered as the basis of accurate direct volume rendering which is different from the early work on volume rendering. Unfortunately, Krueger only discussed the solution of the equation but not reached the volume rendering model from the point view of showing the data features, especially showing boundary surfaces. His attentions lie in a general direct volume rendering algorithm and the filtering of random noise in image space. Moreover, the computat ion cost will be rather high if his method is directly employed without any simplifying. Motivated by Krueger's work, we studied the accurate direct volume rendering model for CT and MRI data set, especially the display of boundary surface. As a result, we derived a specific model, called CSM, which is very suitable for the realistic rendering of CT and MRI data sets.

The major attentions of this paper lie in the volume rendering model. Based on the equation of transfer, we decompose the scattering intensity into volume scattering intensity and surface scattering intensity which represent two different scattering features of volume data. The detail of the volume data is preserved in the volume scattering intensity and the information of boundary surface is preserved in the surface scattering intensity. The value of segmentation is used to represent the strength of boundary surface. The image segmentation operator is directly integrated into the composition of scattering intensity as the weight of surface scattering intensity. We call it cgmposed scattering model (CSM). CSM takes the segmentation and illumination intensity calculation as two coherent processes. Therefore, CSM not only renders the realistic boundary surfaces in data set, but also maintains most of the details within it, i.e. aiming at the high fidelity of direct volume rendering.

We first introduce the equation of transfer under some simplifications such as single scattering. We deduce this equation and present its simplified discrete forms. The scattering term and its decomposition form are discussed. At last, we will propose CSM and its ray casting implementation. CSM has been adopted to display a lot of volume data sets such as ultrasonic wave, CT and MRI, and the results are encouraging.

2 E q u a t i o n of Transfer The equation of transfer in transport theory is applied to calculate the gain and

loss of ray intensity while the ray crosses through a continuous random medium. This equation can be used to interpret many physical phenomena. Also, it is employed

436 J. of Comput. Sci. & Technol. Vol. 11

to calculate the intensity of light ray or microwave in a tmosphere and biological medium.

ro S

e s /

Fig.1. The intensity at r along ~'~ in V.

d,

g,'

e s

Fig.2. The intensity of differential volume in scattering equation.

In a random m e d i u m field V, while ray S crosses th rough V along direction e's, due to the particles ' mutua l action, absorpt ion and scat tering will take place in its propagat ion way. As Fig.1 shows, the intensity I(r, ~) at point r in field V along direction ~'~ is composed of coll imated intensity and diffuse intensity.

z(r, = g,) + e(r, g,) (1)

where Ii(r, ~) is the coll imated intensity, which is the intensi ty of incident ray S at r along direction ~'~; and Id(r, ~) is the diffuse intensity, which is the scat ter ing intensity of all particles in V at point r along direction ~',.

According to the equat ion of transfer in optical physics [15], I(r, ~) can be calculated with Eqs.(2) and (3).

F I ( r , g,) = I(ro, g , ) - e - '(~) + (e -( ' (~)- ' (r ' ) ) �9 Q(r,, g,)). ds (2) J r 0

Q ( r , , c , ) : + (3)

where, r0 is the ray incident point , I(ro, g~) is its intensity of incident ray along direction g~;

7-(r) : f<o ~(r~)ds is the optical depth; o t is the ext inct ion coefficient, which is made up of absorpt ion te rm and scat ter ing

term, i.e. r : a~(r~) + a~(r~), where a~ is absorpt ion coefficient, ~s is scat ter ing coefficient, ~-~ is called albedo; o- t

P(gs, g's) is the phase function, where e s'' is the scattering direction of particles in V excluding point r , , and this function defines the scat ter ing contr ibut ions along direction g, of rays from all directions scat tered by the particles at point r~ (see Fig.2);

q(r~, es) is the source function, which defines the intensity of radia t ion of particles at point r~ along direction g~.

The first t e r m of Eq.(2) is the coll imated incident radiative intensity, which is the radiative intensi ty of incident ray along direction ~'~ at point r t h rough a t t enua t ion of optical dep th ~-(r). The second te rm of Eq.(2) is the diffuse radiat ive intensity, which includes the sum of scat ter ing radiative intensity scat tered by the particles from r0 to r along direction ~'~, and the sum of source radiative intensi ty diffused


by the particles from r0 to r along direction ~'~. They are the first and the second terms of Eq.(3) respectively.

The equation of transfer is a precise equation to calculate the radiative intensity of ray propagation through continuous random medium field. We build our composed scattering model (CSM) for direct volume rendering based on this physical theory.

TTt /

Irl!], X 1

rO /e~

~ v

-2- Io * _

e$

"'-'Y" I Fig.3. A ray crossing through V. Fig.4. The scattering radiative

calculation at point rs.

3 Composed Scattering Model (CSM)

It is very difficult to have analytic solution for the equation of transfer under general random medium field. From the point of view of obtaining clear boundary surfaces, it is reasonable to suppose that the field considered is a low albedo field, ra ther than a high albedo field such as cloud or fog with blur boundary. The specific simplifying conditions are:

(j) single scattering;

(~) each particle is directly illuminated by the light source; and

(~) all particles in one voxel own the same physical properties. Then the 4~ solid angle integral Q(r~, ~) in Eq.(2) is simplified to calculate the

direct light source contributions. As Fig.3 shows, a ray ~'~ crosses through the random medium field V and in-

tersects with the voxels in V at n + 1 points, r0, r l , . . . , r,~. Consider two intersect points with voxel i at ri-1 and ri.

Let li =1 ri - r i - 1 I, and I(ri, g~) is the radiative intensity at point ri along direction ~'~. According to Eq.(1),

I(ri, ~ ) = Ii(ri, ~) + Id(ri, ~) (4)

From the first te rm of Eq.(2), we know that the contribution of I(ri-1, gs) reaching at rl makes up the collimated radiative intensity Ii(ri, ~) at point r~.

Ii(ri, ~) = I ( r i - l , ~ ) . e- f~' ,,(~)d~ = I(ri-1, ~ ) . ti (5)

ti = r f~i at(s)ds : e-~rt(rs).ll (6)

where ti is the t ransparency of interval l~, and crt(r~) is the extinction coefficient at a sample point in interval li.


From Eq.(3), we decompose the second term in Eq.(2) into two parts:

Id(Ti, ~s) = L(Ti, ~s) + Iq(r~, ~ ) (7)

where Is (ri, gs) is the radiative contributions along direction e's scattered by particles in interval from ri-1 to ri directly illuminated by light sources, called scattering intensity; and Iq(ri, gs) is the radiative contributions along direction gs diffused by particles in interval from ri-1 to ri, called source intensity.

We will show how to calculate Is and Iq as follows. In Fig.4, consider a sample point rs in interval from ri-1 to ri, then the radiative intensity at point rs along direction gs is

Is(~s, ~s) = ~s(~s). ;(~s, ~'~)- i0 (8)

where the constructions of scattering term crs(rs) and phase function P(~'s, ~'s) are the key to obtain clear boundary surface and rich details.

Under the isotropic condition, i.e.

p(e~,e s ) = 1, aV(r~) = F(p(rs)) (9)

where p(rs) is the density value at point rs, F(x) is a function which maps density to radiative intensity, the scatter intensity at point rs is

g = I0. ~y(~s)- ;(~s, ~'s) = Iv. F(p(rs)) (10)

We call 1~ as the volume scattering intensity, cr~(rs) is a volume coding function, which indicates the isotropic scattering radiative intensity at sample point under different i l lumination situations.

In order to clearly display the boundary surfaces in volume data field, it is nec- essary to introduce surface effects in the scattering intensity calculation. Therefore, we need the light source direct illumination, which is coincident with our simplifying condition.

Surface scattering is a kind of anisotropic scattering. We may define the phase functions as

ps(es,es) =1 ( f ' ~ ' D I + N ' I ~ ,

where N is the gradient vector at point rs, i.e. the normal vector of boundary surface at point rs. Considering the front and back surfaces of boundary surfaces, we use the absolute value of ( N . ~';) in Eq.(10).

Therefore, we get surface scattering intensity I~(r~, ~s) as Eq.(12).

g(~s, es) I0 os .ps(s ,~'s) 02)

We have already decomposed the scattering intensity into the volume scattering intensity and surface scattering intensity. Then we use weighted summat ion to compose the scattering intensity as Eq.(13).

Is(rs, ~ ) = W : . I~(rs, g,) + W v-/~V(rs, gs) (13)


where W 2 and W~ are the weight functions of I~ and I~ respectively. Allocating weight functions gives us a way to adjust the influences of surface scat-

tering intensity and volume scattering intensity. Physically, the closer the sample point approximates the boundary surface, the greater the influence of the surface scattering intensity is. Therefore, in fact these two weight functions are the image boundary detector operators, which may be used as a threshold value of the fuzzy segmentation of boundary surfaces. The value of segmentation is employed to calculate the scattering intensity in order to clearly display the boundary surfaces.

Many image boundary detectors can be chosen as weight functions, such as gradient operator, Laplacian operator and Marr-Hildreth operator. Among them, the gradient operator is a simple one.

Let the gradient at r~ be (g~,gy,g~), and the gradient boundary operator is

defined as g = I g ~ + gy2 + g~. It is obvious that

w: g, w : (p + p. o) (14)

It means that the weight of surface scattering intensity is in direct proportion to g, and the weight of volume scattering intensity is also in direct proportion to (p+p.g). It indicates that the closer the sample point approximates the boundary surface, the more the scattering light condenses.

Also, second order differential operator, Laplacian operator, V2f( i , j ,k ) and Marr-Hildreth operator V2[G(i, j, k).f(i , j, k)] can be used as weight functions. Com- pared with gradient operator, the values of these operators are scalar and can be directly used for scattering intensity calculation. The advantage of Marr-Hildreth operator is that it can eliminate the noise with Gaussian filter. We will compare the effects of different boundary detection operators as weight functions in the next section.

These two kinds of scattering intensity make up the scattering intensity Is at point rs. In fact, Is is a sphere function defined by the angle between ~'~ and ~'15, which is maximum at two polars +N . The kernel of this sphere is the isotropic volume scattering intensity.

The source intensity plays a role like the ambient intensity in local illumination model. But to a certain extent, it is more important than ambient intensity. The source intensity is the radiative intensity contributed to I(ri, ~ emitted by particles from ri-1 to ri, whose value is determined by the density of the sample points as shown by Eq.(15) and can be considered as a complement to the scattering intensity of interval li.

q( s) = Fq(p(rs)) (15)

Thus, the equations from (4) to (15) make up our composed scattering model. For the transparency ti = e -zJ~ of interval li in Eq.(6), where at = ors +cra, it is

obvious that as and aa are the scattering and absorption coefficients of collimated intensity respectively. In order to simplify the calculation and enhance the boundary


effect, we suppose that at at point rs is dependent on the density value and the gradient value at this point.

Let at = at,p,g �9 (p + gp), then,

~t v'g + a~, (16) 5 r t , p , g - - _ _ ~ ( :Ts ,p ,g ~ - O a , p , g p + gP P + gP

where ffs,p,g and ~,,p,g are constants. Thus, we can directly calculate the collimation incident ray t ransparency t~ according to the value of p(r,) and g(r,) at point %.

By the above analysis, we conclude the diffuse radiative intensity in interval li as follows.

I~(,-. 6) = d [to- ~r.(rs, g . ) . p(e~, g's) + q(%, g*)]" e - f l ,.td. -

= [,to. o,(,-,, g,)-p(g~, g'~) + q(,-,, 4 ) ] . e- '"'d~

= ~o - ~ , (~ , g~) ; ( < , r + q("-, g~) e-~,-, I{ ~ - - crt

= w : . I : (~ , g,) + w : . I;(~, ~,) + q(~, g , ) . (1 - tO crt,o,g(P + gP)

Q(,- , ,~ , ) - (~ - td

( i t )

Thus, the intensity at point ri is

I(ri , ~s) = I(r ,-1, ~ ) . t, + I~ (ri, e , ) . (1 -- t~) (18)

The total intensity reaching the eye is:

= I~(~1, ~',) + t l . (Ij(~2, g~) + t2- (~(~a, #2 + " ")

+ t~_l. (I~(r~, g~) + Ibackg~,,~)---) = rd(T~, #2 + t~- ~rdO'2, ~;) + t~- t~- Ie(~a, #2 + "

n i - -1

= Z ( I I t ,) . i~(r, ,~) i = 1 j = l

(19)

Therefore, the summat ion order of intensity composition is from front to back along the viewing direction. When the accumulated transparency is less than a prede- fined threshold r i.e. i-1 1-Ij=l tj < r the summation process may be stopped. This summation order is very convenient for ray casting of direct volume rendering.

4 I m p l e m e n t a t i o n a n d R e s u l t D i s c u s s i o n

Based on CSM, we have implemented a ray casting Mgorithm for direct volume rendering. The images generated by using this algorithm are based on the following data sets:


(~) CT data set, 128 x 128 • 197 • 8 bit (sloth)

(~) MRI data set, 150 • • 192 x 8 bit (head) The sampling steps are evenly spaced along ray. The step distance is so selected

that each intersected voxel should be sampled once at least. The sample value and its gradient are calculated by using 3D linear interpolation, where the difference is the center difference. We use the HLS coding [17] to convert the density of volume into color. The cutting is the even distance cutting along ray toward the inner, which seems as a kind of peeling. The rendering images are shown in Figs.5 and 6 (please refer to color page 1 for the figures.)

Fig.5 uses the sloth chest CT data set. These images show the chest bone and the inner of chest. Due to the fact that the breastbone belongs to cartilage, it is very difficult to display its precise structure with the general threshold method for CT bone segmentation, as image (b) in Fig.5 shows. The CSM rendering generates image (c) which clearly displays the continuous boundary surface of the breastbone including its position and structure. In image (d), we display the inner of the chest with cutting.

Fig.6 shows the images of the head MRI data set. The thickness and the strength of the boundary surface ensure the display of the semi-transparent head skin with subcutaneous blood vessel (see image (a)). If there is no thickness, the subcutaneous blood vessel cannot be included. On the other hand, if there is no strength, the subcutaneous blood vessel cannot be displayed so clearly although it is included. With segmentation, we clearly display the inner Structure of brain, cerebrum, cerebellum, eyeball and medulla oblongata in image (b). In images (c) and (d), we display the results of cutting which show the cross section of head and brain. In the cross section of the head in image (c), we can see the skin and inner details clearly. In image (d), we can see the inner of the brain by the cutting of the brain into two parts.

These head MRI images and the sloth breastbone images are full of realistic details and cannot be obtained using the current volume rendering models. It is obvious that if the resolution of data set is higher, the CSM highlights its features more: rich details and clear boundary surfaces.

5 C o n c l u s i o n

It is known that the current volume rendering commonly adopts the alternative segmentation methods. Segmentation and illumination intensity calculation are considered as two separated processes in volume rendering. We think that this kind of alternative segmentation based on the threshold and extreme value is the source of errors. Therefore, we proposed the method to introduce the fuzzy segmentation value into the calculation of illumination intensity.

Beginning from the equation of transfer, we simplified the scattering conditions and decomposed the scattering intensity into surface scattering intensity and volume scattering intensity. The segmentation value was regarded as the weight function in the summation of scattering intensity. Thus, we built up our CSM. The CSM


which in t roduced the segmenta t ion into the i l luminat ion in tens i ty ca lcula t ion shows greater fidelity t h a n the popular me thods which take segmenta t ion and i l luminat ion in tens i ty calculat ion as two separa ted processes. Rich detai ls and clear b o u n d a r y surfaces are the m a j o r character is t ics of CSM. The render ing results also demon- s t ra te the fact t h a t it is very sui table for the accura te volume render ing of C T and MRI d a t a sets.

We th ink t h a t surface render ing is a special case of volume rendering, i.e. m e d i u m part icles are d i s t r ibu ted according to cer ta in geometr ic form. Therefore , to cer ta in extent , volume render ing is a more general render ing model which should conta in the surface model . F rom this point of view, we proposed the composed sca t te r ing model (CSM) based on the equa t ion of t ransfer in optical physics. We hope CSM will grow up to a more general model . This will be the fu ture research work of our project .

R e f e r e n c e s

[1] Lorensen W, Cline H. Marching cubes: A high resolution 3D surface construction algorithm. In SIGGRAPH'87.

[2] Nielson G, Hamann B. The asymptotic decider: Resolving the ambiguity in marching cubes. In Visualization'91.

[3] Muller H, Klingert A. Surface interpolation from cross sections. In Focus on Scientific Visual- ization (Hagen H, Muller H, Nielson G eds.), Springer-Verlag, 1993.

[4] Levoy M. Display of surface from volume data. [EEE CG & A, May 1988.

[5] HoehneK et al. 3D-visualization of tomographic volume data using the generalized voxel model. Visual Computer, 1990, 6(1).

[6] Kajiya J, von Herzen ]3. Ray tracing volume rendering. In SIGGRAPH'84.

[7] Drebin R, Carpenter L, Haurahan P. Volume Rendering, In SIGGRAPH'88.

[8] Upson C, Keeler M. V-Buffer: Visible volume rendering. In SIGGRAPH'88.

[9] Sabellap. A rendering algorithm for visualization 3D scalar fields. In SIGGRAPH'88.

[10] Laur D, Hanrahan P. Hierarchical splatting: A progressive refinement algorithm for volume rendering. In SIGGRAPH'91.

[11] Westover L. Footprint evaluation for volume rendering. In SIGGRAPH'90.

[12] Sakas G, Gerth M. Sampling and anti-aliasing of discrete 3D volume density textures. Computer and Graphics, 1992, 16(1).

[13] Blinn J. Light reflection functions for simulation of clouds and dusty surfaces. In SIG- GRAPH'82.

[14] Max N. Light diffusion through clouds and haze. Computer Vision, Graphics and Image Pro- cessing, 1986, 33.

[15] Ishimaru A. Wave Propagation and Scattering in Random Media. Academic Press, 1978.

[16] Krueger W. Volume rendering and data feature enhancement. Computer Graphics, 1990, 24(5).

[17] Farrell E. Color display and interactive interpretation of three dimensional data. IBM J. of Research and Development, 1983, 27(4).

For the biographies of Cai Wenl i and Shi J i aoy ing , please refer to p.488 of this issue.

Cas Wenli e~ at: Composed Scattering Model :~or Direct Volume Rendering

7" - .......

~i t . ;

(a) Outlook~ng

of sloth chest

(b) Sloth chest bone (e) Sloth chest bone (d) Inner parts

ws breastbone ws breastbone of sloth chest re)r.dered by threshold rendered by CSM

Fig.5. CSM rendering results of sloth CT data set.

(a) llead sks v.dth subcutaneous blood vessel (b) Inner structure of head, inc[udlng cerebrum,

cerebellum, eyeball and medulla oblougata

(c) Top brah~ and its headskin (d) Ila!f brain rendc:ed by cutting

Fig.6. CSM zenderh~g results of head MILl data set.

Cai Wenl~ c~ ~.: Display/ng of De~ails in Subvoxal Accuracy

(a) Before segmentation (b) A~er segmentation

Fig.lO. Top view of head before and aIler detail segmentation.

Fig.!l. Rendering of skin, skull,

meninx and brain.

Documents

Composed scattering model for direct volume rendering