Upload
basisspace
View
366
Download
0
Embed Size (px)
DESCRIPTION
This presentation shows the original method for implementing Per-Face Texture Mapping (PTEX) in real-time on commodity hardware. PTEX is used throughout the film industry to handle texture seams robustly while simultaneously easing artist workflow.
Citation preview
Per-Face Texture Mapping for Per-Face Texture Mapping for Realtime RenderingRealtime Rendering
““Realtime Ptex”Realtime Ptex”
Quick NoteQuick Note
• For CUDA/Compute folks:For CUDA/Compute folks:
– Ptex != PTXPtex != PTX
GoalsGoals
Render native Ptex datasets in real-time on commodity hardware
Remove texture seams from textured modelsRemove texture seams from textured models Remove expensive, manual model unwrap step from art Remove expensive, manual model unwrap step from art
pipelinepipeline Support arbitrary resolutions on a per-face basisSupport arbitrary resolutions on a per-face basis
Video Time!Video Time!
© Disney. Used with permission.
Video RecapVideo Recap
Stored as 8Kx8KRendered as 2Kx2K
© Disney. Used with permission.
Model StatisticsModel Statistics
• ~800 FPS on GTX 460 with no optimization~800 FPS on GTX 460 with no optimization
• 278M of Color Texture Data278M of Color Texture Data
• 5812 Patches5812 Patches
© Disney. Used with permission.
General StepsGeneral Steps
LoadModelLoadModel
RenderRender
Preprocess
Draw Time
Bucket and Sort
Bucket and Sort
Generate Mipmaps
Generate Mipmaps
Fill Borders
Fill Borders
Pack Texture Arrays
Pack Texture Arrays
Reorder Index Buffer
Reorder Index Buffer
Pack Patch
Constants
Pack Patch
Constants
Red: Vertex and Index dataGreen: Patch Constant informationBlue: Texel dataOrange: Adjacency information
Load ModelLoad Model
LoadModelLoadModel
RenderRender
Preprocess
Draw Time
Bucket and Sort
Bucket and Sort
Generate Mipmaps
Generate Mipmaps
Fill Borders
Fill Borders
Pack Texture Arrays
Pack Texture Arrays
Reorder Index Buffer
Reorder Index Buffer
Pack Patch
Constants
Pack Patch
Constants
Red: Vertex and Index dataGreen: Patch Constant informationBlue: Texel dataOrange: Adjacency information
Load ModelLoad Model• Vertex DataVertex Data
– Any geometry arranged as a quad-based meshAny geometry arranged as a quad-based mesh
– Example: Wavefront OBJExample: Wavefront OBJ
• Patch TexturePatch Texture
– Power-of-two texture imagesPower-of-two texture images
• Adjacency InformationAdjacency Information
– 4 Neighbors of each quad patch4 Neighbors of each quad patch
• Easily load with library available from Easily load with library available from http://ptex.us/http://ptex.us/
Load Model (cont’d)Load Model (cont’d)
• Texel DataTexel Data
– Per face, load the largest available mipmap level from the Per face, load the largest available mipmap level from the source filessource files
– In memory, place the loaded texel data into a memory In memory, place the loaded texel data into a memory buffer that has a fixed-size texel border regionbuffer that has a fixed-size texel border region
– The borders must be big enough for the largest filter kernel The borders must be big enough for the largest filter kernel you will support (Border size = ½ filter kernel size)you will support (Border size = ½ filter kernel size)
• Bilinear (2x2 kernel): 1 pixel borderBilinear (2x2 kernel): 1 pixel border
• 16x Aniso (16x16 kernel): 8 pixel border16x Aniso (16x16 kernel): 8 pixel border
Load ModelLoad ModelVB: … IB:
Load ModelLoad ModelVB: … IB:
Bucket and SortBucket and Sort
LoadModelLoadModel
RenderRender
Preprocess
Draw Time
Bucket and Sort
Bucket and Sort
Generate Mipmaps
Generate Mipmaps
Fill Borders
Fill Borders
Pack Texture Arrays
Pack Texture Arrays
Reorder Index Buffer
Reorder Index Buffer
Pack Patch
Constants
Pack Patch
Constants
Red: Vertex and Index dataGreen: Patch Constant informationBlue: Texel dataOrange: Adjacency information
Bucket and SortBucket and Sort
• Bucket ptex surfaces into groups by Aspect RatioBucket ptex surfaces into groups by Aspect Ratio
– Each Aspect Ratio will be one bucketEach Aspect Ratio will be one bucket
– 1:1 bucket, 2:1 bucket, 4:1 bucket, etc1:1 bucket, 2:1 bucket, 4:1 bucket, etc
• Then, within each bucket, sort by decreasing surface Then, within each bucket, sort by decreasing surface size and assign IDs.size and assign IDs.
– This allows us to densely pack texture arrays, avoiding “empty” This allows us to densely pack texture arrays, avoiding “empty” surfaces.surfaces.
Bucket and SortBucket and Sort
Bucket 4:1Bucket 1:1
0
Reorder Index BufferReorder Index Buffer
LoadModelLoadModel
RenderRender
Preprocess
Draw Time
Bucket and Sort
Bucket and Sort
Generate Mipmaps
Generate Mipmaps
Fill Borders
Fill Borders
Pack Texture Arrays
Pack Texture Arrays
Reorder Index Buffer
Reorder Index Buffer
Pack Patch
Constants
Pack Patch
Constants
Red: Vertex and Index dataGreen: Patch Constant informationBlue: Texel dataOrange: Adjacency information
Reorder Index BufferReorder Index Buffer
• OriginalOriginal • Post Sort, we have 2!Post Sort, we have 2!
IB:IB(1:1):
IB(4:1):
Generate MipmapsGenerate Mipmaps
LoadModelLoadModel
RenderRender
Preprocess
Draw Time
Bucket and Sort
Bucket and Sort
Generate Mipmaps
Generate Mipmaps
Fill Borders
Fill Borders
Pack Texture Arrays
Pack Texture Arrays
Reorder Index Buffer
Reorder Index Buffer
Pack Patch
Constants
Pack Patch
Constants
Red: Vertex and Index dataGreen: Patch Constant informationBlue: Texel dataOrange: Adjacency information
Generate MipmapsGenerate Mipmaps
• Walk through surfaces, and generate a mipmap chain Walk through surfaces, and generate a mipmap chain from native size to (MinFilterSize x MinFilterSize) for from native size to (MinFilterSize x MinFilterSize) for each surface. each surface.
– Bilinear stops at 2x2, 8xAniso stops at 8x8Bilinear stops at 2x2, 8xAniso stops at 8x8
• Ptex data files do not guarantee complete mipmap Ptex data files do not guarantee complete mipmap chains—although the library can generate all levels for chains—although the library can generate all levels for you—with pre-multiplied alpha.you—with pre-multiplied alpha.
• Mipmap chains stop to allow for unique pinning values Mipmap chains stop to allow for unique pinning values in the cornersin the corners
Generate MipmapsGenerate Mipmaps
• Done for every surface, but only inside the surface—the Done for every surface, but only inside the surface—the border is not touched.border is not touched.
Fill BordersFill Borders
LoadModelLoadModel
RenderRender
Preprocess
Draw Time
Bucket and Sort
Bucket and Sort
Generate Mipmaps
Generate Mipmaps
Fill Borders
Fill Borders
Pack Texture Arrays
Pack Texture Arrays
Reorder Index Buffer
Reorder Index Buffer
Pack Patch
Constants
Pack Patch
Constants
Red: Vertex and Index dataGreen: Patch Constant informationBlue: Texel dataOrange: Adjacency information
Fill BordersFill Borders
• Copy neighbor texels into border area of this surface’s Copy neighbor texels into border area of this surface’s mip levelmip level
– Match source and destination number of pixels when Match source and destination number of pixels when possiblepossible
• Bordered textures are the heart of the logical realtime Bordered textures are the heart of the logical realtime ptex solutionptex solution
• Allows 1-2 texture lookups per ptex sample requestAllows 1-2 texture lookups per ptex sample request
– 1 if not performing tween-mip-level interpolation, 2 otherwise1 if not performing tween-mip-level interpolation, 2 otherwise
Fill BordersFill Borders
Fill BordersFill Borders
Fill BordersFill Borders
Fill BordersFill Borders
Fill BordersFill Borders
Pack Texture ArraysPack Texture Arrays
LoadModelLoadModel
RenderRender
Preprocess
Draw Time
Bucket and Sort
Bucket and Sort
Generate Mipmaps
Generate Mipmaps
Fill Borders
Fill Borders
Pack Texture Arrays
Pack Texture Arrays
Reorder Index Buffer
Reorder Index Buffer
Pack Patch
Constants
Pack Patch
Constants
Red: Vertex and Index dataGreen: Patch Constant informationBlue: Texel dataOrange: Adjacency information
Texture ArraysTexture Arrays
• Like 3D / Volume Textures, except:Like 3D / Volume Textures, except:
– No filtering between 2D slicesNo filtering between 2D slices
– Only X and Y decrease with mipmap level (Z doesn’t)Only X and Y decrease with mipmap level (Z doesn’t)
– Z indexed by integer index, not [0,1]Z indexed by integer index, not [0,1]
• E.g. (0.5, 0.5, 4) would be (0.5, 0.5) from the 5E.g. (0.5, 0.5, 4) would be (0.5, 0.5) from the 5 thth slice slice
• API SupportAPI Support
– Direct3D 10+: Texture2DArrayDirect3D 10+: Texture2DArray
– OpenGL 3.0+: GL_TEXTURE_2D_ARRAYOpenGL 3.0+: GL_TEXTURE_2D_ARRAY
Pack Texture ArraysPack Texture Arrays
• Copy all generated data into Texture2DArrayCopy all generated data into Texture2DArray
• Each Texture2DArray represents a single mipmap levelEach Texture2DArray represents a single mipmap level
– Texture2DArrays present a view of the data that is efficient Texture2DArrays present a view of the data that is efficient for GPU layoutfor GPU layout
– Logical Textures cut across the same page index of every Logical Textures cut across the same page index of every Texture2DArrayTexture2DArray
Pack Texture ArraysPack Texture Arrays
0 1 2
3
Logical Texture Layout
Pack Texture ArraysPack Texture Arrays
10x10x3(1+8+1)x(1+8+1)x3
6x6x4(1+4+1)x(1+4+1)x4
4x4x4(1+2+1)x(1+2+1)x4
GPU Layout Texture2DArray
gColor[0]
gColor[1]
gColor[2]
Pack Patch ConstantsPack Patch Constants
LoadModelLoadModel
RenderRender
Preprocess
Draw Time
Bucket and Sort
Bucket and Sort
Generate Mipmaps
Generate Mipmaps
Fill Borders
Fill Borders
Pack Texture Arrays
Pack Texture Arrays
Reorder Index Buffer
Reorder Index Buffer
Pack Patch
Constants
Pack Patch
Constants
Red: Vertex and Index dataGreen: Patch Constant informationBlue: Texel dataOrange: Adjacency information
Pack Patch ConstantsPack Patch Constants
• Each primitive has a “PatchInfo” struct:Each primitive has a “PatchInfo” struct:
– TextureId – which array slice contains our dataTextureId – which array slice contains our data
– TopMipLevel – the index of the top-most mipmap level for this TopMipLevel – the index of the top-most mipmap level for this texturetexture
– FlipUVs – whether or not to flip UVs, allows 1:2 and 2:1 to be FlipUVs – whether or not to flip UVs, allows 1:2 and 2:1 to be grouped into same bucketgrouped into same bucket
– MaxMipLevels – Maximum mipmap level for each edgeMaxMipLevels – Maximum mipmap level for each edge
Fill BordersFill Borders
LoadModelLoadModel
RenderRender
Preprocess
Draw Time
Bucket and Sort
Bucket and Sort
Generate Mipmaps
Generate Mipmaps
Fill Borders
Fill Borders
Pack Texture Arrays
Pack Texture Arrays
Reorder Index Buffer
Reorder Index Buffer
Pack Patch
Constants
Pack Patch
Constants
Red: Vertex and Index dataGreen: Patch Constant informationBlue: Texel dataOrange: Adjacency information
RenderRender
• Brief Recap of D3D11 Pipe stagesBrief Recap of D3D11 Pipe stages
New Direct3D 11 StagesNew Direct3D 11 Stages
IAIA VS HS TSTS DS GS RASRAS PS OMOM
Programmable (Shader)
Fixed Function
RenderRender
• In the Hull ShaderIn the Hull Shader
– Store pre-expansion PrimitiveID to output control pointsStore pre-expansion PrimitiveID to output control points
– This is used everywhere to determine which set of Patch This is used everywhere to determine which set of Patch Constants are owned by the currently running thread (in Constants are owned by the currently running thread (in Domain, Geometry or Pixel Shaders)Domain, Geometry or Pixel Shaders)
New Direct3D 11 StagesNew Direct3D 11 Stages
IAIA VS HS TSTS DS GS RASRAS PS OMOM
RenderRender
• In the Domain ShaderIn the Domain Shader
– Vertices belonging to a quad meshes are evaluated with a Vertices belonging to a quad meshes are evaluated with a domain location, which is (0,0)-(1,1) for each patchdomain location, which is (0,0)-(1,1) for each patch
– Use this value to store our UV locationUse this value to store our UV location
New Direct3D 11 StagesNew Direct3D 11 Stages
IAIA VS HS TSTS DS GS RASRAS PS OMOM
RenderRender
• Texture lookups in Domain or Pixel Shader are replaced Texture lookups in Domain or Pixel Shader are replaced with a “ptex” sample function.with a “ptex” sample function.
– Determines which logical texture to work fromDetermines which logical texture to work from
– Compute mipmap level(s) to accessCompute mipmap level(s) to access
– Scale and bias computed (u,v) by mipmap sizeScale and bias computed (u,v) by mipmap size
– Lookup texels, return weighted averageLookup texels, return weighted average
Texture Lookup Shader CodeTexture Lookup Shader Code
• Traditional (D3D11)Traditional (D3D11)
return return
gTxDiffuse.Sample(gTxDiffuse.Sample(
gSampler,gSampler,
I.fTextureUV );I.fTextureUV );
• PtexPtex
return return
ptex( gTxDiffuse,ptex( gTxDiffuse,
gSampler, gSampler,
I.uPrimitiveId,I.uPrimitiveId,
I.fTextureUV );I.fTextureUV );
Complex logic hiddenin single function call
Questions?Questions?
• jmcdonald at nvidia dot comjmcdonald at nvidia dot com
• Brent dot Burley at disneyanimation dot comBrent dot Burley at disneyanimation dot com
• http://ptex.us/http://ptex.us/
• http://groups.google.com/group/ptexhttp://groups.google.com/group/ptex
• Or visit our studio sessionOr visit our studio session
– Monday, 8 August @ 4:30 PM – 5:00 PMMonday, 8 August @ 4:30 PM – 5:00 PM
– The Studio / West Building, Ballroom AThe Studio / West Building, Ballroom A
Additional ConsiderationsAdditional Considerations
• Filtering across edges with differing numbers of pixelsFiltering across edges with differing numbers of pixels
• Filtering across cornersFiltering across corners
Filtering across edgesFiltering across edges
Filtering across edgesFiltering across edges
Filtering across edgesFiltering across edges
Filtering across cornersFiltering across corners
• Corners with valence > 4 Corners with valence > 4 cannot be exactly cannot be exactly matched with a matched with a bilerp (4 samples)bilerp (4 samples)
© Disney. Used with permission.
Filtering across cornersFiltering across corners
• The solution is a bit more involved.The solution is a bit more involved.
– First, walk the mesh and determine which corners are sharedFirst, walk the mesh and determine which corners are shared
– For each shared group, determine the correct value for when For each shared group, determine the correct value for when we’re exactly at that corner. E.g. Simple Average.we’re exactly at that corner. E.g. Simple Average.
– Then, modify every mipmap level of every surface of that Then, modify every mipmap level of every surface of that group s.t. the shared corner has the same valuegroup s.t. the shared corner has the same value
– When you’re in the corner, everyone will perform the same When you’re in the corner, everyone will perform the same lookup—regardless of mipmap level—and continuity prevailslookup—regardless of mipmap level—and continuity prevails
Filtering across cornersFiltering across corners
• For Realtime Ptex, we apply pinning to all corners, For Realtime Ptex, we apply pinning to all corners, regardless of valence.regardless of valence.
Pinned CornersPinned Corners
Questions redux?Questions redux?
• jmcdonald at nvidia dot comjmcdonald at nvidia dot com
• Brent dot Burley at disneyanimation dot comBrent dot Burley at disneyanimation dot com
• http://ptex.us/http://ptex.us/
• http://groups.google.com/group/ptexhttp://groups.google.com/group/ptex
• Or visit our studio sessionOr visit our studio session
– Monday, 8 August @ 4:30 PM – 5:00 PMMonday, 8 August @ 4:30 PM – 5:00 PM
– The Studio / West Building, Ballroom AThe Studio / West Building, Ballroom A