Ray-Patch Intersection for Improving Rendering Quality of Per-pixelDisplacement Mapping

Ravish Mehra 1 Subodh Kumar11 IIT Delhi

Abstract

Recent GPU advances have popularized pixel-shaderbased techniques for per-pixel displacement mapping. Animportant step in the process is to compute intersection ofray with a bilinear patch approximating the displacementmap. Accurate ray-patch intersection can significantly en-hance the quality of rendered images. However, the increasein quality comes at a significant computation cost resultingin a substantial drop in performance.

This paper explores this trade-off between computationand quality. We propose several techniques for ray-patch in-tersection and compare it to exact ray-quadric/ray-bilinearintersection proposed previously. We show that our quadricbased local-gradient-plane approximation achieves a com-parable visual quality with a significant performance im-provement.

1. Introduction

Ray-patch intersection has traditionally been used invarious ray-tracing approaches for rendering complex ge-ometry. This geometry can be made up of either simpleprimitives like points and triangles or high order primitiveslike second order patches (bilinear, quadric, SLIM, Bezier).These patches are not onlyC0 continuous but may also havesmooth derivatives as well. This results in visibly bettershading than is possible with simple primitives. These rep-resentations have numerous applications in scientific visu-alizations, CAD systems for modeling geometry, physicalsimulations of particles and biological applications for vi-sualizing DNA’s & molecules.

Recently, ray-patch intersection has found a new applica-tion – per-pixel displacement mapping (also called inversedisplacement mapping). Based on programmability of mod-ern GPU’s, per-pixel displacement mapping is a ray-castingapproach that tries to achieve the same visual effects as tra-ditional displacement mapping [2] but without ever gener-ating or drawing the detailed geometry. Incorporating ray-patch intersection in the last steps of per-pixel displacementmapping algorithm (i.e. near the surface boundary) signifi-

cantly improves the visual quality of the rendered results.However, since it requires expensive computation, it im-pacts the performance as well becoming the main bottle-neck.

This paper is a detailed study of ray-surface patch in-tersection. We propose a sequence of approximation tech-niques for this and test the validity of these approximationswith chosen patch sizes. We provide a comparison betweenthe exact and approximate techniques on the basis of qual-ity and performance achieved by them. Finally, we proposean approximate ray-quadric intersection technique in whichthe quadric surface is approximated with a tangent planewhose normal is in direction of surface gradient. We showthat the quality achieved by our method is comparable toexact ray-quadric intersection but is much faster. We alsogive a simple conversion formula for converting height fieldbased bilinear patch into quadric. The main contributions ofthis paper are as follows:

1. A novel quadric based local-gradient-plane approx-imation & intersection technique that achieves highrendering quality and is much faster than numericallysolving ray-quadric intersection.

2. A quality and speed based comparison of different ex-act and approximate intersection techniques.

Figure 1. ICVGIP box: res. 5122, 45 fps

2. Related work

In the last few years, there has been increasing interestin graphics community for using non-triangular primitivesfor displaying complex models. Several high order sur-face representations such as bilinear, quadric, SLIM, iso-surfaces have been used providing superior quality shad-ing as compared to flat triangles and points. Several pointsplatting approaches [1, 6, 9] have been proposed for ren-dering these primitives. Sigg et al [10] render each quadricprimitives separately by representing them as single vertexand computing their bounding volume in vertex shader. Forfragments generated via this bounding box, ray-quadric in-tersection is computed in fragment shader. Finally, depthbuffer is used for finding closest quadric’s fragment andblending the result.

Per-pixel displacement mapping on the other hand, ren-ders a single bounding volume for the entire geometry,which could be highly complex and detailed. Actual ge-ometry is stored offline in the pre-processing stage eitheras height-map, cone-map [4] or relief-map [7, 8] in 2D tex-ture or VDM [12], GDM [13] or distance-map [3] in 3Dtexture. This dataset is passed onto the fragment shaderthat computes the actual intersection of viewing ray withthe detailed geometry. This intersection point is used forfinal shading. These techniques score over traditional dis-placement mapping [2] by not tessellating the surface andhence not rendering a large number of micro polygons gen-erated, thus achieving real time performance. Since implicitrepresentation of detailed geometry in all these techniquesis still triangular, shading achieved is of the similar qualityas rasterization.

Recently, several per-pixel displacement mapping tech-niques changed the implicit surface representation to higherorder primitives(surface patches) and performed ray-patchintersections as the last step of their algorithm for signifi-cantly improving their rendering results. Oh et al [5] rep-resented the discrete height field as bilinearly interpolatedheight-map and sampled it linearly along the viewing ray tofind the ray-bilinear intersection, thus removing blockinesscaused by discrete height-map. But as pointed out in [11],this linear approximation of bilinear height-map is not al-ways correct and fails at places where the height-map is notapproximately linear.

Tevs [11] create a hierarchical maximum mipmappedheight-map with bilinear patch residing at the lowest hier-archy level. Ray-bilinear patch intersection is performed atthe lowest level either analytically using the method pro-posed in [9] or approximately using linear & binary searchsteps to converge to the final intersection point. They drew acomparison between these two approaches and favored thesecond, approximate, technique due to the significant per-formance cost of the first.

3. Overview

Per-pixel displacement mapping techniques for heightfield rendering store the detailed complex geometry as aheight-map. This height-map is a uniformly sampled 2Dgrid that stores at each sample point the height of the sur-face at that point above a reference surface. Different tech-niques process this height-map for generating specializeddatasets such as cone-map [4], relief-map [7, 8] or maxi-mum mipmapped height-map [11]. These datasets are usedby their respective algorithms in the rendering stage forfinding point of intersection of the ray with the detailed ge-ometry.

For piecewise patch approximation of height mappedsurface we do the following - Within each 2D grid cell weperform an approximation for the height-mapped surfacepassing ’above’ that cell. This results in a piece-wise ap-proximation for the entire geometry. This cell approxima-tion could be in the form of a triangular tessellation, bilinearor quadric, or simply planar. For the triangular tessellation,we first find the height of the surface patch at the center ofthe cell. The triangles are generated by connecting the sur-face points at the corners of the cell with that at the centeryielding four triangles. Approximation by a planar patch isa plane fitting problem and can be solved by standard al-gorithms such as Levenberg-Marquardt algorithm (LMA).Conversion to a bilinear patch can be done in a straightfor-ward manner as given in [9]. This bilinear expression canbe converted to a quadric function directly(Section 4.1). Wecompute the patch parameters offline and store them in a 2Dtexture that is passed onto the fragment shader. Triangular,bilinear and quadric patch parameters can also be computedonline in the fragment shader by passing the heightmap.

For ray-patch intersection, we will first discuss quadricbased exact numerical roots technique and then proposethree approximate techniques – triangular patch, planarpatch and local-gradient-plane approximation.

Figure 2. Overview of algorithm

A generic overview of the rendering pass employed byper-pixel displacement mapping techniques incorporatingray-patch intersection is as follows:

Algorithm 1 Per-pixel displacement mapping with ray-patch intersection

1. Instead of generating and drawing the actual detailedgeometry, we draw its bounding box.

2. For each fragment f generated by the bounding box,we trace a ray starting at its texture location r in thedirection d (from eye to r) until it reaches close to de-tailed geometry.

3. We access the parameters of the patch correspondingto the cell reached in step 2 and compute ray-patch in-tersection exactly or approximately.

4. If the intersection lies within the cell, we have success-fully found the intersection point and exit. Otherwisethe ray hits the boundary of the cell and we must goback and continue step 2.

The ray-geometry intersection reduces to first locatingthe intersection in a cell and then computing the exact inter-section of the ray with the patch corresponding to that cell.Ray-tracing step and the exact definition of close to detailedgeometry varies based on the actual algorithm being used.For maximum mipmapped heightmap [11] and pyramidaldisplacement mapping [5], we switch to ray-patch intersec-tion when we reach the lowest level of the mipmap hierar-chy. Sphere tracing [3] uses distance to the nearest pointon the geometry as the safety distance for tracing the ray.When this safety distance becomes zero(or less than someε), we perform ray-patch intersection.

Ray-patch intersection, due to its high computationalcost, drastically impacts the performance of per-pixel dis-placement mapping and becomes the main bottleneck. Ta-ble 2 compares the performance of a per-pixel displace-ment mapping algorithm – sphere tracing, with and withoutray-patch intersection. Figure 3 shows the visual qualityachieved by both the methods.

(a) sphere tracing (b) sphere tracing + ray-patch

Figure 3. Quality comparison

We now discuss in detail several approaches of doingsurface patch approximation and ray-patch intersection.

4. Quadric surface & ray-quadric intersection

Each height-map cell defined by its four corners can beexactly represented as a bilinear patch that maintains C0

continuity. This bilinear patch is a second degree functionand can be directly converted into a quadric. We use the fullsecond degree quadric defined by following equation-

Ax2 +By2 + Cz2 +Dxy + Eyz

+ Fzx+Gx+Hy + Iz + J = 0 (1)

4.1 Bilinear patch to quadric conversion

This section presents the conversion of bilinear surfacepatch into corresponding quadric. The results are statedhere while the derivation is given in Appendix. Bilinearpatch formed by four points ( ~VA, ~VB , ~VC , ~VD) described byweighing parameters (u, v) is :

V (u, v) = (1− u)(1− v) ~VA + (1− u)v ~VB+ (1− v)u ~VC + uv ~VD

(2)

Equation 34 gives us the quadric coefficients:

A = B = C = E = F = 0 (3)

D =

(a3b2c1

)G =

(c3b2 − d2a3

b2c1

)(4)

H =

(b3c1 − a3d1

b2c1

)I = −1 (5)

J =

(d1d2a3 + d3b2c1

b2c1

)−(d2b3c1 + d1c3b2

b2c1

)(6)

We store coefficients D, G, H, J as RGBα components ina 2D texture(shown in Figure 4).

Figure 4. Quadric parameters

4.2 Ray-quadric Intersection

For computing the intersection of ray r+td with quadricwe will first discuss numerically solving ray-quadric inter-section.

(a) 642 (b) 1282 (c) 2562

Figure 5. Statue surface: Rendering result for ray-quadric at different resolutions

4.2.1 Numerical roots

We solve the equation of the ray with quadric as follows:Ray equation : r+ td : (x0, y0, z0) + t(dx, dy, dz)Solving with quadric,

A(x0 + tdx)2 +B(y0 + tdy)

2 + C(z0 + tdz)2

+D(x0 + tdx)(y0 + tdy) + E(y0 + tdy)(z0 + tdz)

+ F (z0 + tdz)(x0 + tdx) +G(x0 + tdx)

+H(y0 + tdy) + I(z0 + tdz) + J = 0

(7)

Substituting,

P =Ad2x +Bd2y + Cd2z +Ddxdy + Edydz + Fdzdx

Q =2x0dxA+ 2y0dyB + 2z0dzC + x0dyD + y0dxD

+ y0dzE + z0dyE + z0dxF + x0dzF +Gdx

+Hdy + Idz

R =Ax20 +By20 + Cz20 +Dx0y0 + Ey0z0 + Fz0x0

+Gx0 +Hy0 + Iz0 + J

(8)

Finally,Pt2 +Qt+R = 0 (9)

Algorithm 2 Ray-quadric intersectionif P = 0 then

t = −RQ

elseif Q2 − 4PR < 0 then

t = HIT CELL BOUNDARY(x0, y0, z0)else

t =−Q±√

Q2−4PR

2P

Quadric patch defined in this manner truly represents thesurface formed by the four corners of the height-map cell.Figure 5 shows the quality achieved at various resolutionsof coefficient texture(Figure 4). Note the superior quality

Figure 7. Ray-quadric intersection

even at low resolutions. Due to a large number of stepsinvolved in the ray-quadric intersection step, this method ismuch slower.

5 Approximate intersection techniques

In this section, we describe in detail the three approxi-mate ray-patch intersection techniques.

5.1. Ray-triangular patch intersection

In this approach, we approximate the surface patch in aheight-map cell with four triangular sub-patches formed bythe four corners and the midpoint of that cell. This com-putation can be done online eliminating the need for pre-processing. For finding ray-patch intersection, we computeray-triangle intersection for the four triangular sub-patchesand test the validity of the intersection point. For a validintersection, the point should lie within the boundaries ofthe cell and the intersected triangle. Also the normal of thistriangle(n) should be facing the ray(r + td) i.e. d.n < 0.We stop the algorithm if the intersection point is valid. Oth-erwise, we hit the wall and continue tracing the ray. Figure 8illustrates the steps of the algorithm.

Figure 6 shows the rendering result achieved by thismethod for various resolutions of height-map. Rendering

(a) 642 (b) 1282 (c) 2562

Figure 6. Rocky surface: Rendering result for triangular patch at different resolutions

quality increases with resolution as approximation of sur-face with triangular patch becomes more accurate at higherresolution.

Figure 8. Triangular patch intersection

5.2. Planar approximation and ray-plane in-tersection

We approximate the surface of the height-map cell withthe planeAx+By+Cz+D = 0 by performing plane fittingand store the coefficientsA,B,C,D in RGBα channels of2D texture. This texture is passed to the fragment shader.

Figure 9. Planar approximation parameters

For finding ray-plane intersection, we numerically solvefor intersection of the plane equation with the ray r+ td asfollows: Putting r = (x0, y0, z0),d = (dx, dy, dz) in theplane equation

A(x0 + tdx)+B(y0 + tdy)+C(z0 + tdz)+D = 0 (10)

t = − (Ax0 +By0 + Cz0 +D)

(Adx +Bdy + Cdz)(11)

Here (A,B,C) is the direction of normal(N ) to the plane.Small shader code resulting in increased performance is

the main advantage of this technique. But this planar ap-proximation is valid only at high resolutions. At lower res-olutions, C0 continuity is not maintained causing missed in-tersections and obvious cracks at cell boundaries. Figure 11shows the quality achieved by this method for different res-olutions of coefficient texture(Figure 9). Planar approxi-mation becomes more & more accurate as we increase theresolution.

Figure 10. Ray-plane intersection

5.3 Local-gradient-plane approximation

We now present an approximate ray-quadric intersectionapproach that achieves superior rendering quality compa-rable to exact ray-quadric intersection discussed before butat the same performance cost as ray-plane intersection. Forfinding the ray-quadric intersection of the ray r+td startingat point r(x0, y0, z0) going in the direction d(dx, dy, dz),we first find the corresponding point r′(x0, y0, z′) locatedbelow r lying on the quadric(see Figure 12). We find the

(a) 642 (b) 1282 (c) 2562

Figure 11. ICVGIP surface: Rendering result for planar approximation at different resolutions

gradient vector∇ of the quadric surfaceAx2+By2+Cz2+Dxy + Eyz + Fzx+Gx+Hy + Iz + J = 0 at r′.

∇ = (∇x,∇y,∇z) (12)∇x = (2x0A+ y0D + z′F +G) (13)∇y = (2y0B + x0D + z′E +H) (14)∇z = (2z′C + y0E + x0F + I) (15)

Figure 12. Local-gradient-plane intersection

We then construct a plane tangential to the surface pass-ing through r’ such that the normal vector N of the plane isin the direction of the ∇ i.e N = ∇. Equation of plane canbe written as N.(x, y, z) +M = 0 for constant M

∇xx+∇yy +∇zz +M = 0 (16)

Since the plane passes through r′,

M = −(∇xx0 +∇yy0 +∇zz′) (17)

Using equation 11, the solution becomes

t = −(∇x(x0) +∇y(y0) +∇z(z0) +M

∇xdx +∇ydy +∇zdz

)(18)

=−∇z(z0 − z′)

(∇xdx +∇ydy +∇zdz)(Using eq. 17) (19)

Figure 13 shows the rendered result of local-gradient-plane approximation at various resolutions of quadric coef-ficient texture(Figure 4). It can be seen that this methodachieves high quality similar to numerically solving ray-quadric.

(a) 642

(b) 1282

Figure 13. Glass surface: Gradient-plane

6. Comparison and Results

We compare our proposed approximation techniques -triangular, planar & gradient plane, with exact bilinearand quadric patch techniques in terms of performance andspeed.

Table 1 shows a comparison between their renderingperformance for different test cases at varying resolutions.

Statues surface Rocky surface Glass surface ICVGIP surfaceResolution 642 1282 2562 5122 642 1282 2562 5122 642 1282 2562 5122 642 1282 2562 5122

Triangular 67 39 21 11 53 29 16 8 88 52 28 15 43 23 12 7Bilinear 99 55 30 17 78 42 22 13 123 73 40 22 71 40 21 13Planar 348 218 127 70 219 151 89 50 352 244 153 88 159 107 63 40

Quadric 225 147 84 46 139 98 59 32 235 160 98 56 102 66 43 28Gradient 273 191 115 62 174 129 81 45 288 208 134 78 127 83 54 35

Table 1. Performance comparison of various techniques (in fps). Screen resolution 1024x768

In this test, we incorporate only steps 1,3,4 of the algo-rithm(Section 3) and assume that we are always close tothe surface. This gives us a direct comparison betweenthese techniques as we only perform ray-patch intersectionfor tracing the ray. Among these, triangular, bilinear andquadric are the slowest as they require complicated rootsolving that requires longer shader code. Planar intersec-tion is the fastest but the quality of rendering suffers at lowresolution(Figure 11a,b). On the other hand, gradient planeachieves faster frames per second with better quality evenat low resolution.

In Table 2, we show the rendering performance of aper-pixel displacement mapping algorithm - sphere tracing,and augment it with different ray-patch intersection tech-niques. We define close to surface when the safety distanceof sphere tracing becomes zero(or less than ε) and performall the four steps of the algorithm. We can also see the im-pact of ray-patch intersection on performance of per-pixeldisplacement mapping. Simple sphere tracing(ε=0.01) withno ray-patch intersection is obviously the fastest but yieldsthe least quality. With ε = 0, the quality improves butbecomes extremely slow. Sphere tracing(ε=0.01) with ray-triangular, ray-quadric and ray-bilinear is slow but yield thehighest quality. Ray-planar is fast but gives good renderingquality only at higher resolutions. Ray-gradient-plane, onthe other hand, is fast and gives rendering quality compara-ble to triangular, bilinear & quadric even at low resolutions.Figures 1 & 14 show the demos rendered using sphere trac-ing augmented with ray-gradient-plane intersection.

We have implemented the proposed techniques as ShaderModel 4.0 fragment programs. Timings reported in this pa-per were produced on Intel PentiumD 3.4GHz machine withGeforce 8800GTS graphics card and 2GB RAM. All im-ages given in the paper were rendered at a screen resolutionof 1280x968.

7. Conclusion and Future Work

We have presented several exact and approximate tech-niques of performing ray-patch intersection and explored its

applicability in improving the rendering quality of per-pixeldisplacement mapping. We have provided a detailed quality& performance based comparison between these techniquesand their impact on per-pixel displacement mapping. Wehave also tested the validity of the proposed approximationsat different resolution.

For future work, we would like to explore other surfacepatch approximations ( Bezier, SLIM ) along with efficient& accurate methods for their ray-patch intersection.

(a) Tiled cylinder: texture 5122, 88fps

(b) Earth: Coeff. texture 5122, 135fps

Figure 14. Ray-gradient-plane intersection

Statues surface Rocky surface Glass surface ICVGIP surfaceResolution 642 1282 2562 5122 642 1282 2562 5122 642 1282 2562 5122 642 1282 2562 5122

ST(ε = 0.0) 142 145 149 101 120 118 115 47 160 156 149 61 124 140 145 75ST(ε = 0.01) 630 605 576 490 492 455 412 309 720 629 555 450 465 447 413 275+ Triangular 73 62 54 40 41 33 25 19 71 57 49 39 45 39 30 24+ Bilinear 93 78 64 49 54 43 34 24 93 74 62 47 60 52 42 31+ Planar 292 260 229 193 161 152 132 88 278 225 194 160 146 137 116 88

+ Quadric 216 204 182 157 118 111 101 72 200 167 145 124 107 94 85 69+ Gradient 250 241 223 185 143 136 129 88 233 204 179 147 129 116 107 79

Table 2. Performance comparison of various techniques (in fps). Screen resolution 1024x768

References

[1] M. Botsch, A. Hornung, M. Zwicker, and L. Kobbelt. High-quality surface splatting on today’s gpus. Point-BasedGraphics, 2005. Eurographics/IEEE VGTC Symposium Pro-ceedings, pages 17–141, June 2005.

[2] R. L. Cook. Shade trees. SIGGRAPH Comput. Graph.,18(3):223–231, 1984.

[3] W. Donnelly. Per-pixel displacement mapping with distancefunctions. GPU Gems 2, 22(3):123–136, 2005.

[4] Dummer. Cone step mapping: An iterative ray-heightfieldintersection algorithm. 2006.

[5] K. Oh, H. Ki, and C.-H. Lee. Pyramidal displacement map-ping: a gpu based artifacts-free ray tracing through an imagepyramid. In VRST ’06: Proceedings of the ACM sympo-sium on Virtual reality software and technology, pages 75–82, 2006.

[6] M. Pauly, R. Keiser, L. P. Kobbelt, and M. Gross. Shapemodeling with point-sampled geometry. ACM Trans.Graph., 22(3):641–650, 2003.

[7] F. Policarpo and M. M. Oliveira. Relief mapping of non-height-field surface details. In I3D ’06: Proceedings ofthe 2006 symposium on Interactive 3D graphics and games,pages 55–62, New York, NY, USA, 2006. ACM.

[8] F. Policarpo, M. M. Oliveira, and a. L. D. C. Jo˙ Real-time relief mapping on arbitrary polygonal surfaces. In I3D’05: Proceedings of the 2005 symposium on Interactive 3Dgraphics and games, pages 155–162, New York, NY, USA,2005. ACM.

[9] S. D. Ramsey, K. Potter, and C. Hansen. Ray bilinear patchintersections. journal of graphics tools, 9(3):41–47, 2004.

[10] C. Sigg, T.Weyrich, M.Botsch, M. GrossRamsey, K. Potter,and C. Hansen. Gpu-based ray-casting of quadratic surfaces.Eurographics symposium of point based rendering, 2006.

[11] A. Tevs, I. Ihrke, and H.-P. Seidel. Maximum mipmaps forfast, accurate, and scalable dynamic height field rendering.In SI3D ’08: Proceedings of the 2008 symposium on Inter-active 3D graphics and games, pages 183–190, 2008.

[12] L. Wang, X. Wang, X. Tong, S. Lin, S. Hu, B. Guo, andH.-Y. Shum. View-dependent displacement mapping. ACMTrans. Graph., 22(3):334–339, 2003.

[13] X. Wang, X. Tong, S. Lin, S. Hu, B. Guo, and H.-Y. Shum.Generalized displacement maps. Computer Graphics Fo-rum, 22(3):334–339, 2004.

A Bilinear patch to quadric conversion

Bilinear patch formed by four adjacent cell points~VA(x1, y1, z1), ~VB(x2, y1, z2), ~VC(x1, y2, z3), ~VD(x2, y2, z4)

described by weighing parameters (u, v) is equation 2.Substituting

~a = (a1, a2, a3) = ~VA + ~VD − ~VB − ~VC (20)= (0, 0, z1 + z4 − z2 − z3) (21)

~b = (b1, b2, b3) = ~VC − ~VA (22)= (0, y2 − y1, z3 − z1) (23)

~c = (c1, c2, c3) = ~VB − ~VA (24)= (x2 − x1, 0, z2 − z1) (25)

~d = (d1, d2, d3) = ~VA (26)= (x1, y1, z1) (27)

And rearranging,

V (u, v) = uv~a+ u~b+ v~c+ ~d (28)(X,Y, Z) = uv(a1, a2, a3) + u(b1, b2, b3) (29)

+ v(c1, c2, c3) + (d1, d2, d3) (30)

Comparing both sides,

X = 0 + 0 + vc1 + d1 (31)Y = 0 + ub2 + 0 + d2 (32)Z = uva3 + ub3 + vc3 + d3 (33)

Substituting value of u and v from Equations 31& 32 inEquation 33, we get

Z =

(a3b2c1

)XY +

(c3b2 − d2a3

b2c1

)X +

(b3c1 − a3d1

b2c1

)Y

= +

(d1d2a3 + d3b2c1 − d2b3c1 − d1c3b2

b2c1

)(34)