“MDL materials to GLSL shaders: theory and...

April 4-7, 2016 | Silicon Valley

www.esi-group.com

Andreas Süßenbach, NVIDIA

Andreas Mank, ESI Group

04/04/2016

MDL MATERIALS TO GLSL SHADERS THEORY AND PRACTICE

2

“a pioneer and world-leading provider in Virtual Prototyping”

— www.esi-group.com/

3

INTERACTION

FIDELITY

BEHAVIOR APPEARANCE

4

MDL IN COMMERCIAL PRODUCTS

4/3/2016

MATERIAL CONSTRUCTION

Substance Designer

MATERIAL TWEAKING

Cinema 4D Maya mental ray others

MATERIAL SHARING (LIBRARY)

5

MDL IN CUSTOM RENDERERS

4/3/2016 03.04.2016

MATERIAL TWEAKING MATERIAL

CONSTRUCTION

MATERIAL SHARING (LIBRARY) N

VID

IA

IRAY

DEFINITION

MD

L S

DK

IMPLEMENTATION

GLSL (Rasterizer)

CUDA (Ray Tracer)

6

AGENDA

What is MDL, what is the MDL SDK ?

How do I use the MDL SDK ?

How do I map MDL materials to GLSL shaders ?

What is the issue with material parameters?

7

WHAT IS MDL, WHAT IS THE MDL SDK ?

8

WHAT IS MDL ?

Declarative language to specify visual material properties

No shader language !

Renderer agnostic

export material tintedStuff( color parTint = color(.6,.2,.2) ) = material ( surface: material_surface( scattering: df::specular_bsdf( tint: parTint ) ) );

9

MDL MATERIAL FIELDS

material_surface:

surface

material_surface:

backface

material_geometry material_volume

bool:

thin_walled

color:

ior

bsdf: scattering

material_emission: emission

bsdf: scattering

material_emission: emission

float3: displacement

float: cutout_opacity

float3: normal

vdf: scattering

absorption_coefficient

scattering_coefficient

10

WHAT IS THE MDL SDK ?

A dynamic library providing a C++ API

Used to load and compile MDL materials

Can be used to create and modify materials as well

Used to get detailed information out of the compiled material

11

HOW DO I USE THE MDL SDK ?

12

USING MDL SDK 1/5

Initializing

Material Compiling

Material Parsing

Expression Parsing

Call Parsing

HINSTANCE dll = LoadLibrary("libmdl_sdk.dll");

INeuray_factory* factory = (INeuray_factory*)GetProcAddress(dll, "mi_neuray_factory");

mi::base::Handle<mi::neuraylib::INeuray> neuray = factory(0,MI_NEURAYLIB_API_VERSION );

mi::base::Handle<mi::neuraylib::IMdl_compiler> mdlCompiler = neuray->get_api_component<mi::neuraylib::IMdl_compiler>();

13

USING MDL SDK 2/5

Initializing

Material Compiling

Material Parsing

Expression Parsing

Call Parsing

mi::Sint32 reason = mdlCompiler->load_module ("::nvidia::vMaterials::AEC::Concrete::concrete_blocks");

mi::base::Handle<const mi::neuraylib::IModule> module( mdlCompiler->access<mi::neuraylib::IModule>( "mdl::nvidia::vMaterials::AEC::Concrete::concrete_blocks"));

for ( mi::Size i=0 ; i<module->get_material_count() ; i++ )

{

mi::base::Handle<mi::neuraylib::IMaterial_definition const> materialDefinition(mdlCompiler->access <mi::neuraylib::IMaterial_definition>(module->get_material(i));

mi::base::Handle<mi::neuraylib::IMaterial_instance> materialInstance

(materialDefinition->create_material_instance(0,&result));

mi::base::Handle<mi::neuraylib::ICompiled_material> compiledMaterial

(materialInstance->create_compiled_material (mi::neuraylib::IMaterial_instance::CLASS_COMPILATION,1.0f ,&result);

}

14

USING MDL SDK 3/5

Initializing

Material Compiling

Material Parsing

Expression Parsing

Call Parsing

for (mi::Size i=0; i<compiledMaterial->get_parameter_count(); i++)

{

char const* parameterName = compiledMaterial-> get_parameter_name(i);

mi::base::Handle<mi::neuraylib::IValue const> value (compiledMaterial->get_argument(i));

}

for (mi::Size i=0; i<compiledMaterial->get_temporary_count(); i++)

{

mi::base::Handle<mi::neuraylib::IExpression const> expression

(compiledMaterial->get_temporary(i));

}

mi::base::Handle<mi::neuraylib::IExpression const> surfaceExpression

(compiledMaterial->get_field("surface"));

mi::base::Handle<mi::neuraylib::IExpression const> backfaceExpression

(compiledMaterial->get_field(„backface"));

15

USING MDL SDK 4/5

Initializing

Material Compiling

Material Parsing

Expression Parsing

Call Parsing

switch( expression->get_kind() )

{

case mi::neuraylib::IExpression::EK_CONSTANT: ...

break;

case mi::neuraylib::IExpression::EK_PARAMETER: ...

break;

case mi::neuraylib::IExpression::EK_TEMPORARY: ...

break;

case mi::neuraylib::IExpression::EK_DIRECT_CALL: ...

break;

}

16

USING MDL SDK 5/5

Initializing

Material Compiling

Material Parsing

Expression Parsing

Call Parsing

mi::base::Handle<mi::neuraylib::IType const> type(call-> get_type());

mi::base::Handle<mi::neuraylib::IExpression_list const> arguments (call->get_arguments());

for (mi::Size i=0; i<arguments->get_size(); i++)

{

char const* name = arguments->get_name(i);

mi::base::Handle<mi::neuraylib::IExpression const> argument

(arguments->get_expression(i));

}

17

MDL SDK: MAP MDL TO AN EXPRESSION TREE

MDL Material Expression Tree

export material tintedStuff( color parTint = color(.6,.2,.2) ) = material ( surface: material_surface( scattering: df::specular_bsdf( tint: parTint ) ) );

tintedStuff

surface

scattering emission

backface

scattering emission

18

SIMPLIFY YOUR WORK: MDLParser

Simple C++ class to derive from

About 40 pure virtual functions (visitor pattern)

Simple string and value based interface

“Guides” you through the Expression tree of a material

Hides all the intricacies of the MDL SDK

If you just want to convert MDL Materials into something

19

HOW DO I MAP MDL MATERIALS TO GLSL SHADERS ?

20

MAPPING MDL TO GLSL

MDL

Parameters

Temporaries

Constants

Operators (+,-,…)

Function calls

Expression tree

GLSL

uniforms

global variables

42;

temporary3 + n * color;

snippets !!

stitched snippets !!

21

float mdl_math_minValue( in vec3 a ) { return( min( min( a.x, a.y ), a.z ) ); }

GLSL SNIPPETS: FUNCTIONS

vec4 mdl_df_weightedLayer( in float weight, in vec4 layer, in vec4 base ) { return( mix( base, layer, weight ) ); }

22

vec4 specularReflectionBSDF( in vec3 N, in vec3 L, in vec3 lightSpecular , in vec3 materialSpecular ) { const float shininess = 256.0f; vec4 rgba = vec4( 0.0f, 0.0f, 0.0f, 1.0f ); float cosTheta = dot( N, L ); if ( 0.0f < cosTheta ) { vec3 R = reflect( -L, N ); float cosAlpha = max( 0.0f, dot( R, viewDir ) ); float shine = pow( cosAlpha, shininess ); rgba.rgb = shine * lightSpecular * materialSpecular; } return( rgba ); }

GLSL SNIPPETS: BSDF

23

GLSL SNIPPETS

A snippet per BSDF (currently, there are 6 different)

A snippet for each function (currently, there are about 50)

A snippet for some commonly used helper functions

-> about 80 snippets

24

mdl_base_fileBumpTexture(bump_texture, bump_strength * 4, mono_average, temporary14, vec2(0,1), vec2(0,1), wrap_repeat, wrap_repeat, temporary15, false )

GLSL SNIPPETS: STITCHED TOGETHER

::mdl::base::file_bump_texture( texture: bump_texture, bump_source: ::base::mono_average, scaling: float2((1.f/texture_scale.x) * 0.5f, (1.f/texture_scale.y) * 0.5f), translation: texture_translate, clip: false, factor: bump_strength * 4.f, texture_space: uv_space_index)

25

GLSL SNIPPETS: WHAT TO DO WITH THEM ?

Fixed shader skeletons !!

Call some functions in those skeletons

Fill the bodies of those functions with some stitched snippets

-> resembles kind of “virtual function calls”

26

void main(void) { stateNormal = normalize(varNormal); if (! gl_FrontFacing) { stateNormal = - stateNormal; } texCoord0 = varTexCoord0; tangent = normalize(varTangent); binormal = normalize(varBinormal); viewDir = normalize(varEyePos - varWorldPos); vec4 rgba = vec4(0.0f, 0.0f, 0.0f, 0.0f); if (0.0f < evalCutoutOpacity(stateNormal)) { vec3 normal = evalNormal(normal); materialIOR = evalIOR(normal); vec3 materialEmissive = vec3(0.0f, 0.0f, 0.0f); bool useFront = gl_FrontFacing; if (useFront) { materialEmissive = evalMaterialEmissiveFront(normal); } else { // there's no emission on the back-side, unless thinWalled is true useFront = !evalThinWalled(); if (!useFront) { materialEmissive = evalMaterialEmissiveBack(normal);

} materialIOR = 1.0f / materialIOR; } rgba = vec4(materialEmissive, 0.0f); if (0 < sys_NumLights) { vec3 lightAmbient; for (int i=0; i<sys_NumLights; i++) { sampleLight(sys_Lights[i], varWorldPos, lightDir, lightAmbient, lightDiffuse, lightSpecular); rgba += useFront ? evalColorFront(normal) : evalColorBack(normal); } rgba.a /= sys_NumLights; } else { rgba.a = 1.0f; } rgba.a *= alphaCutout; if (0.0f < rgba.a) { lightDir = reflect(-viewDir, normal); rgba.rgb += (useFront ? evalEnvironmentFront(normal) : evalEnvironmentBack(normal)).rgb; } } emitColor(rgba); }

MASTER FRAGMENT SHADER

27

float cutoutOpacity = evalCutoutOpacity(stateNormal);

if (0.0f < cutoutOpacity)

{

vec3 normal = evalNormal(stateNormal);

materialIOR = evalIOR(normal);

FRAGMENT SHADER DETAIL

28

“VIRTUAL” FUNCTIONS USED

evalCutoutOpacity

evalNormal

evalIOR

evalMaterialEmissiveFront

evalMaterialEmissiveBack

evalThinWalled

evalColorFront

evalColorBack

evalEnironmentFront

evalEnvironmentBack

29

YOUR WORK

Create a set of shader skeletons that fit your needs

Specify a set of “virtual” functions to be filled

Implement the snippets to be used for the materials

Stitch the snippets for those functions according to the materials

30

WHAT IS THE ISSUE WITH MATERIAL PARAMETERS ?

31

MATERIAL PARAMETERS

Compiled material just has simple parameters!

MDL provides great flexibility on material parameters

Parameter types can range from simple values to call graphs

The MDL compiler does some mapping from complex to simple

-> some parameters of a compiled material might not have a name !

32

if ( hasParameter( compiledMaterial, name ) ) { // real-time case: just feed changed parameter into your shader } else { // recompile material ! if ( previousHash == currentHash ) { // interactive case: feed all (unnamed) parameters into your shader } else { // recreate your shader !! // feed all parameters into your shader } }

PSEUDO-CODE TO HANDLE PARAMETER CHANGES

33

CONCLUSION

Material Designer: pay attention to exposed material parameters

Application Developer: be prepared to also support the complex parameters

34

SUMMARY

Have a set of fixed shader skeletons calling specified functions

Fill the bodies of those functions with some stitched-in GLSL snippets

Pay attention to complex material parameters

35

DEMO: MATERIAL CREATION

36

DEMO: MATERIAL CREATION

37

DEMO: MATERIAL CREATION

38

DEMO: MATERIAL CREATION

39

DEMO: MATERIAL CREATION

40

mdl 1.0; import df::*; import state::*; import base::*; import tex::*; import anno::*; import math::*; export material scratched_silver ( color base_color = color(0.9, 0.9, 0.9) [[ anno::display_name("Color"), anno::description("The color of the material") ]], uniform texture_2d roughness_texture = texture_2d("resources/scratch_damage.png", tex::gamma_srgb) [[ anno::display_name("Roughness texture"), anno::description("Roughness of the surface finish") ]] ) = let { base::texture_coordinate_info coordinate = base::transform_coordinate(

transform: base::rotation_translation_scale(0.0,0.0,float3(1, 1, 1)) ); float roughness = base::file_texture( texture: roughness_texture, mono_source: base::mono_average, uvw: coordinate ).mono; bsdf glossy_bsdf = df::simple_glossy_bsdf ( mode: df::scatter_reflect, tint: base_color, roughness_u: (roughness - 0.1) / 0.9 ); } in material ( surface: material_surface( scattering: glossy_bsdf ) );

SCRATCHED_SILVER.MDL

simple_glossy_bsdf

0.9

roughness_texture

.mdl .cu .ptx

41

SCRATCHED_SILVER.CU rtTextureSampler<float4, 2> roughness_texture;

rtDeclareVariable(float, roughness_texture_gamma, , "Roughness texture gamma: Gamma value for texture Roughness texture");

rtDeclareVariable(float3, base_color, , "Color: The color of the material");

class MaterialGetter_0 …

class GlossyGetter_1 … // simple_glossy_bsdf // mdl::df::simple_glossy_bsdf

class EmissionGetter_2 … // material emission

class EmissionGetter_3 … // material emission

class GeometryGetter_5 … // material geometry

typedef Glossy<GlossyGetter_1> Glossy_0;

typedef MaterialEmission<EDF, EmissionGetter_2> Emission_1;

typedef MaterialSurface<Glossy_0, Emission_1> Surface_2;

typedef MaterialEmission<EDF, EmissionGetter_3> Emission_3;

typedef MaterialSurface<BSDFs::Empty, Emission_3> Surface_4;

typedef MaterialVolume<VDF, VolumeGetter_4> Volume_5;

typedef MaterialGeometry<GeometryGetter_5> Geometry_6;

typedef Material<Surface_2, Surface_4, Volume_5, Geometry_6, MaterialGetter_0> TheMaterial;

RT_PROGRAM void monteCarloClosestHit() {

const TheMaterial mat = TheMaterial(

Surface_2(

Glossy_0(),

Emission_1( EDF() )

),

Surface_4(

BSDFs::Empty(),

Emission_3( EDF() )

),

Volume_5( VDF() ),

Geometry_6()

);

monteCarloEyeClosestHitDispatcher<TheMaterial>(mat);

}

Material parameters

42

DEMO: RASTERIZER VS. RAY TRACER

RAY TRACER RASTERIZER

Shadows

Reflections

Walnut

Silver

Car paint

43

DEMO: RASTERIZER VS. RAY TRACER

RAY TRACER RASTERIZER

Shadows

Reflections

Walnut

Silver

Car paint

44

DEMO: RASTERIZER VS. RAY TRACER

RAY TRACER RASTERIZER

Shadows

Reflections

Walnut

Silver

Car paint

45

DEMO: RASTERIZER VS. RAY TRACER

RAY TRACER RASTERIZER

Shadows

Reflections

Walnut

Silver

Car paint

46

DEMO: RASTERIZER VS. RAY TRACER

RAY TRACER RASTERIZER

Shadows

Reflections

Walnut

Silver

Car paint

47

DEMO: RASTERIZER VS. RAY TRACER

RAY TRACER RASTERIZER

Shadows

Reflections

Walnut

Silver

Car paint

48

CONCLUSION

Support MDL materials in your application

Share MDL materials across applications

Maintain material appearance across applications

“a single unified material library that can be used in multiple applications”

April 4-7, 2016 | Silicon Valley

www.esi-group.com

THANK YOU