HYPE Platform is still being developed in a stealth mode, and I cannot disclose all the details of the product yet. Generally speaking, we are trying to solve a very BIG problem in video marketing/advertising industry, and we do that with the help of Unreal Engine. UE is used by AAA video game developers as well as other industries to create stunning, high-fidelity interactive graphics experiences. One may wonder, why anyone would want to customize the already state of the art and “all - included “game engine such as Unreal. Trust me, no game engine is perfect and unique use cases require very specific features which normally are not considered standard in game development industry.
Slug Text in HYPE media editor (WIP)
HYPE platform targets video advertising sector, where graphic quality of text is extremely important. Unfortunately, text rendering component (not GUI but one you can drop into the map) in UE4 is a naïve implementation of SDF text originally pioneered by Valve , with all the downsides the technique brings with it, such as quality degradation on zoom-ins, rounded corners of the glyph make this approach unfit for use with professional grade advertising content. Therefore, we had to bring a third-party solution, whereas the natural choice was Eric Lengyel’s SlugLibrary. Slug is a state of the art,self-contained, high performance text rendering library, which is compatible with all the industry standard low level graphics APIs. It also comes with built-in text shaping engine, which is normally required to layout glyphs and lines of text according to the calligraphic rules and full Unicode support. One would have to use complex third-party solutions just for that part, such as Harfbuzz . Additionally, Slug allows rendering of SVG graphics, per-glyph transforms, range styling (like html text style tags) etc. All these make it a Swiss-knife solution for text rendering feature in any possible scenario.
The challenges.
We had to figure out if it's possible to use the existing mesh setup infrastructure and abstain from needless modification of the engine’s code. Slug vertex layout interleaves several vertex attributes in one stream of data. It also expects some uniforms to be passed into vertex shader and a couple of data texture samplers bound to the fragment shader. I checked FLocalVertexFactory which uses FDynamicMeshVertex, and found that it didn't fit into the data layout of Slug.
Perspective transform issues due to coordinate system difference
On the vertex shader side, we could use the existing vertex structure by adding Slug related macro define and put all Slug specific attributes in it. However .cpp interface of the FLocalVertexFactory presented a problem since it couldn’t use an arbitrary vertex buffer layout. Trying to adjust the existing interface to our needs would mean substantial modifications to the source code of the engine in the areas such as FLocalVertexFactory , DynamicMeshBuilder and probably some more regions which are sensitive to changes and lack proper documentation. Therefore, we decided to implement our own vertex factory (VF). The good news were that custom VF didn't require messing with the engine sources. The API was designed to be extensible and the whole thing could be implemented in UE4 plugin.
The learning resources.
No official documentation exists on how to extend VFs. All I could find on Epic’s doc pages was a list of classes and properties without any explanation on how to use those. I was able to find a blogpost written by Matt Hoffman, and another one – by Khammassi Ayoub. Both are pouring an overwhelming amount of information about UE’s rendering system and vertex factories. However, Hoffman’s article is outdated for 4.27 as many APIs underwent significant changes since it was written. Also, both articles give an example of a partial customization of the VFs, mostly stuff like adding a custom shading model and custom code to the built-in vertex and pixel shaders. After reading both postings, I still had no clue how to approach the task technically as the amount of the information was overwhelming. I started looking into UE4 sources for examples of custom vertex factories such as LocalVertexFactory and TextRenderComponent. I also received some help and guidance on how to add a custom shading model from Epic’s technical support staff on UDN.
The result.
This features had been in development in parallel with other important tasks. The first iteration included integration of Slug as a post process 2D overlay.
As this approach didn't require messing with VFs and UE's built-in shaders, it was a relatively easy start. Overall, it took me around 6 month to accomplish full integration of Slug Library into Unreal, fix bugs and bring the plugin to a production ready state. If I had to work only on this task full time, it would probably take me a month and half. It wouldn't be exaggeration to say that 70% of that time had been invested into learning of UE internal APIs and get those to work as expected in this case. Actual Slug Library porting process to UE had been the easy part as the code is rendering API agnostic. Slug shaders are HLSL compatible so it was just a matter of replacing the input uniforms with those in UE base vertex and pixel pass shaders.
Final tests of text features in UE editor
By Michael Ivanov. 12.06.22