Unreal Engine 5.1 Update: Ready for Virtual Production?

Introduction

The initial release of Unreal Engine 5.0 was aimed at game developers. Features such as Nanite, Lumen, MetaSounds and World Partition improve the look and performance of games. However, some of these features did not support in-camera VFX (ICVFX), and other virtual production specific features from 4.27 were not brought over to 5.0. Well, the recent update to Unreal Engine 5.1 changes that, while bringing even more features to this workflow. Those in the ICVFX and Virtual Production field that remained on UE 4.27 will now want to take a look at what this new version has to offer. The release notes are quite extensive (if you tried to print them, it would be nearly 600 pages), so in this article, we will cover some of the top features for those in the VP and ICVFX fields. 

Lumen

Lumen is Unreal Engine’s new real-time lighting system. This is a software base raytracing system that can leverage RT-specific hardware for better and faster results. This update improves how this system handles foliage, meaning more natural-looking plants and outdoor environments. Also improved is how reflections on transparent materials are handled. Windows and other glass materials will now look much closer to traditionally ray-traced materials. 

Most importantly, Lumen now supports multiple viewports in nDisplay for ICVFX. That means those shooting on an LED volume will finally be able to use Lumen in full. It is worth noting that Epic is still calling this “experimental”, so users will want to do some testing with their own projects and volumes to be sure they are getting the results they are after. 

Nanite

Nanite is Unreal’s virtualized geometry system that allows for the use of highly detailed meshes without the need for several LODs. Creating LODs is a very laborious process, but was necessary for real-time performance optimization. Nanite will reduce the geometry of a mesh as polygons get small enough on screen, thereby taking the strain off the GPU. This is an excellent feature for filmmaking where many assets are created with very high fidelity, but now there isn’t the extra overhead of high polygon assets in the background taking unnecessary resources. With 5.1 nanite now supports foliage, such as trees and shrubs. This even extends to landscape grass meshes. This will provide much greater flexibility with large outdoor environments. 

On-Demand Shader Compilation

One area of frustration for any user of Unreal Engine is compiling shaders. Depending on the size of your project, and the capabilities of your processor, this could take anywhere from 5 minutes to an hour or more. Yes, you can do some work while the shaders are compiling, but most of your CPU power will be dedicated to compiling these shaders, not to mention many of the materials in the viewport will not show up until they are compiled. This feature will delay the compiling of shaders until they are actually needed to be displayed on the screen. For larger projects, this will speed up the initial loading of the project, allowing users to get to work faster. It should be noted that this only applies to the project specific shaders, and not the global shaders that are compiled during the initial startup of the engine. Artists, especially those working on very large projects without direct access to the DDC will find this most useful. On the other hand, those filming on an LED volume will likely not want to rely on this at the time of filming as it will cause performance issues when new shaders need to be compiled. In this situation, it would be safer to compile all the shaders ahead of time. 

Path Tracer

Unreal Engine added Path Tracer to the rendering options in 4.27, so it is still a very new feature. Initially, this was created to provide a “ground truth” to compare with real-time ray tracing results. The latest release added support for more engine features such as exponential height fog, Sky Atmosphere, planar decals, and water geometry. This brings it more in line with the standard rendering pipeline. Also added is Multi-GPU support, as long as the two cards are connected with an NVLink bridge and SLI is enabled. Unfortunately, the new RTX 4000 series does not support this technology, so hopefully in future updates, Epic will add a way to use multiple GPUs without NVLink as other offline renderers do. 

Another addition to the Path Tracer is the option for Optix denoiser. This runs on the GPU and should be faster than the current CPU-based denoiser. The Optix denoiser is labeled as “experimental” so it might not be completely ready for production work, but it is worth investigating to see if the quality difference is acceptable for your project. If nothing else, it will help give faster previews while iterating on your project. 

Light Mixer

Light Mixer is a new editor window that will display all lights in a scene. This gives you quick access to be able to edit all the lights in a scene. It’s not uncommon to have dozens of lights in any given scene, so this will be beneficial to any Unreal user, but especially Virtual Production users who film on an LED volume. When paired with the improved DMX capabilities, Light Mixer will make editing the lighting on a set very efficient. It is still technically in Beta, so be prepared for bugs, but it is something that any Unreal user will want to use. 

In-Camera VFX Editor

The In-Camera VFX Editor is another new window within Unreal that is a collection of the most used controls in one interface. Controls such as color corrections, nDisplay adjustments, and light cards are all presented in an easy-to-use interface, making it much more simple to get a stage set up and ready for filming. Previously this could have been done with UI scripting but would have taken considerable knowledge of UE scripting language and UI creator. Now it is built right into the editor. 

SMPTE-2110 Support

One of the major updates in Unreal 5.1 is the addition of SMPTE-2110 and a new nDisplay Media I/O Mapping System. These updates will allow for greater configuration when using an LED volume. When using LED wall processors that also support SMPTE-2110, a variety of video sources can be mapped to nDisplay viewports. There are so many uses for this, it will be exciting to see what studios come up with. 

One of the biggest uses will be to dedicate a single system as the inner frustum, while the other nodes will render the rest of the volume. Currently, the inner frustum is rendered by whichever node or nodes, the camera is currently facing. This requires quite a bit of coordination between the nodes as the rendering is handed off from one node to the next as the camera pans across the volume. Some uses in this space use two GPUs per node to get extra performance as one GPU can be dedicated to the inner frustum, while the other is rendering the outer. The downside to this is that each node needs to have 2 GPUs, however, when a particular node is not on camera, that second GPU sits idle. With SMPTE-2110, a single node could be designated as the inner frustum with a high-end GPU, while the outer frustum nodes could be using less expensive GPUs if they don’t need to be achieving the same level of fidelity. This feature is still experimental, so much testing will still need to be done to find the best configuration, and it may be unstable. Also, this does require the LED wall processors to support SMPTE-2110, so if your particular hardware does not support it, you may have to use the existing workflows. 

Conclusion

These are just some quick highlights of the new features in Unreal 5.1 that many of our customers will be interested in. We highly recommend checking out the release notes to see more about the new animation and general improvements here; https://docs.unrealengine.com/5.1/en-US/unreal-engine-5.1-release-notes/. Many in the ICVFX and Virtual Production fields have held off on moving from 4.27 to 5, however, this update shows Epic’s commitment to these fields and provides enough new features that most users will begin their new projects in Unreal 5.1