Goals of Canvas Render Target● Real time generation of textures with an easy to use interface (UCanvas).

● Modifiable in both Blueprints and C++.

● Good integration with Unreal Editor and its tool sets.

● Ability to render materials within a texture.

Research● Unreal Engine 3 and 4 have exposed render targets which allow arbitrary

things to be rendered to them. Most commonly this is used to set up scene captures.

● Unreal Engine 3 had a scripted texture which created a UCanvas interface to render to the render target.

Research● UCanvas is a straightforward and easy to use 2D rendering library. It can

render:○ Textures in a variety of ways and supports rotation.○ Materials and supports rotation.○ Text and bitmap fonts.

● Thus, the easiest way to achieve this was to duplicate something very similar to Unreal Engine 3 and modernize it for Unreal Engine 4.

UCanvasRenderTarget.h● DECLARE_DYNAMIC_MULTICAST_DELEGATE_ThreeParams

(FOnCanvasRenderTargetUpdate, UCanvas*, Canvas, int32, Width, int32, Height)○ This declares a type of delegate which can have multiple functions

bound to it acting like a mailman system. This delegate forwards the active Canvas and the working width and height.

● virtual void UpdateResource()○ This function is used to update the render target. Depending on the

contents of the render target, this could be called just once per lifetime or once per frame to get real time results.

UCanvasRenderTarget.h● static UCanvasRenderTarget2D* CreateCanvasRenderTarget2D

(TSubclassOf<UCanvasRenderTarget2D> CanvasRenderTarget2DClass, int32 Width, int32 Height)○ This is a static function which is instances a type of

UCanvasRenderTarget with dimensions.

● void Render(class UCanvas* Canvas, int32 Width, int32 Height)○ This is a Blueprint native event. Thus, if a Blueprint implementation of

this function isn’t found then it will execute the native code path.

● void GetSize(int32& Width, int32& Height)○ Returns the size of the canvas render target.

● FOnCanvasRenderTargetUpdate OnCanvasRenderTargetUpdate○ Delegate reference which is defined above. This holds weak

references to functions within objects.

UCanvasRenderTarget.cpp● UCanvasRenderTarget2D::UpdateResource()

○ if (IsTemplate()) - This check is needed to ensure that we don’t execute UpdateResource() when compiling in the Blueprint Editor.

○ StaticFindObjectFast(...) - This finds an instance of a UCanvas that has been created with a name.

○ ConstructObject<UCanvas>(...) - If we did not find an instance of the UCanvas then we need to construct one for all UCanvasRenderTargets to use.

UCanvasRenderTarget.cpp● UCanvasRenderTarget2D::UpdateResource()

○ Canvas->AddToRoot() - This prevents the UCanvas from being garbage collected as we want this instance of UCanvas to be a singleton which is used by all UCanvasRenderTargets.

○ UpdateResourceImmediate() - Implemented by UTextureRenderTarget2D. Called to prevent UCanvasRenderTarget from being cleared after a render update.

○ CanvasRenderTargetMakeCurrentCommand(...) - Enqueues a rendering command to set this render target as the current RHI render target for the Canvas to draw on.

UCanvasRenderTarget.cpp● UCanvasRenderTarget2D::UpdateResource()

○ OnCanvasRenderTargetUpdate.Broadcast(...) - All bound rendering delegates are called to update the canvas render target.

○ Render(...) - Calls either the Blueprint implemented event or the native function if one hasn’t been defined. This allows Blueprint to override C++ when required.

○ CanvasRenderTargetResolveCommand(...) - Enqueues a rendering command to resolve this render target after Canvas has drawn onto it, so that it is available for rendering.

UCanvasRenderTarget.cpp● UCanvasRenderTarget2D::CreateCanvasRenderTarget2D(...)

○ ConstructObject<UCanvasRenderTarget2D>(...) - This performs the actual instancing of the UCanvasRenderTarget. We accept different classes to allow a generic static function to instance all Blueprint or C++ subclasses of UCanvasRenderTarget.

● UCanvasRenderTarget2D::Render_Implementation(...)○ This function is automatically created by the Unreal Header Tool since

the UFUNCTION definition of the Render() function is declared with BlueprintNativeEvent.

○ This function is virtualized, so it can be redefined by a C++ subclass which is executed if the Render function does not have a Blueprint implementation.

Blueprint Example● Use Unreal Editor to create a new Blueprint subclass of

UCanvasRenderTarget.

● Implement the Render() event and use Blueprint Canvas functions to draw to the render target.

● These Blueprint Canvas functions were also created by me. They are just Blueprint mappings of C++ functions.

● You can now use the Level Blueprint to instance the Blueprint UCanvasRenderTarget and assign it to a Material Instance Constant which is then used in the world.

● This method is useful when you want to generate textures on the fly and use them in several places in your game.

Level Blueprint Example● In the Level Blueprint we first add the Blueprint node which will instance a

UCanvasRenderTarget to use.

● From here we can use the delegate binding to assign a new Blueprint event which is called when the Update() function is called.

● The Blueprint event set up has the necessary references to Canvas and the UCanvasRenderTarget width and height for us to use the Blueprint Canvas nodes to draw on the render target.

Level Blueprint Example● We’ll need to call the Update() function manually. We could call it once, or

call it once per Level Tick if we need the updates to be very frequent.

● You can now use the Level Blueprint to instance the Blueprint UCanvasRenderTarget and assign it to a Material Instance Constant which is then used in the world.

● This method is handy for creating one off textures that would help make the level more unique or allow you to show level specific information.

C++ Example● Subclassing UCanvasRenderTarget in C++ allows you to create more

elaborate rendering algorithms.

● Render_Implementation(...) is the function that you want to override to update the UCanvasRenderTarget.

● You could also create a subclass which can then be subclassed in Blueprints, thus the C++ subclass would act more like a helper class.

C++ Example● From here you can either instance it in Blueprints and use it like before or

you can instance it in C++ and use it in C++ as well.

● This method is really handy for creating quite complex compositions to use for HUD elements or player skins.

Goals of Texture2DDynamic● Efficient method to upload data from Blueprint or C++.

● Principally used for generating data for materials to use.

● Fast enough for real time usage.

Extending Texture2DDynamic● This sort of functionality already exists in Unreal Engine 4 and it is

commonly used for generating textures from other sources such as Steam avatars.

● Extending Texture2DDynamic would allow us to do everything we would need.

Texture2DDynamic.h● UCLASS(..., Blueprintable, BlueprintType)

○ This exposes Texture2DDynamic as a Blueprintable class which allows developers to create new Blueprint classes. BlueprintType declares that this class can be used as a variable.

● bool PostInit(...)○ Called after the Texture2DDynamic has been intiailized. This is

declared as a BlueprintNativeEvent so it can either be implemented in Blueprints or subclassed in C++. It has an meaningless return parameter so that in the Blueprint Editor it appears in the side panel rather than being hidden in the Events menus.

● class UTexture2DDynamic* Create(...)○ This static function creates a UTexture2DDynamic for use in the

game. This takes a class to allow the developer to instance a Blueprint subclass or a C++ subclass.

Texture2DDynamic.h● void UpdateMipFromVectorData(...)

○ Helper function which uploads an array of FVectors as RGB bytes. Alpha is assumed as 1.0f.

● void UpdateMipFromLinearColorData(...)○ Helper function which uploads an array of FLinearColor as RGBA

bytes.

● void UpdateMipFromColorData(...)○ Helper function which uploads an array of FColor as RGBA bytes.

● void UpdateMipFromByteData(...)○ Function which validates the incoming byte data and updates the mip

map data.

Texture2DDynamic.cpp● UTexture2DDynamic::UpdateMipFromVectorData(...)

○ int32 ExpectedSize = GetExpectedDataSize() - Handy helper function which just returns the expected data size based on the texture parameters it was created with. We often want to check that the data we’re sending off is correctly sized,

○ ByteData.Add(255 * V.Z) - Since we want the data as byte, we need to convert it. You could also do some post processing of the data here to ensure the data is correct.

○ ByteData.AddZeroed(ExpectedSize - ByteData.Num()) - We don’t really want to force non programmers to a lot of arbitrary rules, thus if the developer did not provide enough data, we can just fill the rest of the array with zeros.

Texture2DDynamic.cpp● UTexture2DDynamic::UpdateMipFromByteData(...)

○ FMipUpdateParams - This is a struct which wraps all of the data we want to send to the render thread. It is entirely possible to send more than one parameter but this makes it easy to expand in the future.

○ ENQUEUE_UNIQUE_RENDER_COMMAND_ONEPARAMETER - This macro allows us to send the new mip map data to the rendering thread to update the texture.

○ FMemory::Memcpy(...) - Since we’re just writing the data, getting the texture’s mip map array pointer and then copying the new mip map data array handles that nicely and efficiently for us.

○ RHIUnlockTexture2D(...) - Grabbing the mip map data array pointer requires us to lock the Texture2DDynamic since it is running on a separate thread (we don’t want threading issues here). Since we’re now done with copying the data, we need to unlock the Texture2DDynamic.

Blueprint Example● As UTexture2DDynamic is a Blueprint type, we can create a new Blueprint

subclass directly in Unreal Editor.

● PostInit() is the event that we would want to implement in Blueprint to create and the texture data.

● The Blueprint can then be instanced in Blueprint or C++ and used with materials and so forth just like UCanvasRenderTarget.

● This method is handy if levels need some unique gradients to use for creating special areas or randomizing things … perhaps associated with when the player decides to play the game?

C++ Example● If you need more complex gradients or want to mathematically calculate

the texture data, it is probably best done in C++ for speed reasons. Generating perlin noise or multiple gradients (when generating a 256 color palette texture for example) is much easier in C++.

● PostInit_Implementation() is the function you will want to override to generate the data and then call UpdateMipFromByteData to send it away.

Gradient Mapping Example● Gradient mapping is a method which allows you to use a color palette to

colorize the end result in a shader.

● Think of this method as duplicating the games of the past where we could use different palette indexes to create different variations of the same enemy, weapon or other game entities.

● This method works in Unreal by using a texture as a UV coordinate lookup for a gradient map (or palette map).

● Where Texture2DDynamic would come in handy is allowing you to dynamically generate new gradient maps based on user data which would allow you to expose player driven customization options.

Conclusion● Unreal Engine 4’s Texture2D classes are very flexible and using them in

conjunction with other subsystems such as the Material subsystem, Particle subsystem or even the Post Processing subsystem.

● Theoretically, since you can upload any kind of data into a Texture2D … you could get the shader system to perform some calculations which you then read the results of at a later time. So GPU processing with CPU read back could lead to interesting results.

● You could also read image data from a variety of formats such as JPG, PNG and so forth which you could then use in game.

● You could also read video data and stream it at run time.

● If other libraries output to a RGBA buffer, then you could use that data as well.