Messenger application development in Unreal Engine 5 for Android.

So, we started developing a mobile app using Unreal Engine 5 for the Android platform. As often happens in our company, the need for custom functionality arose quite quickly. The app contains elements similar to those of chat applications like Facebook Messenger or WhatsApp. Therefore, one of the central components we had to deal with was the Android native virtual keyboard (VK) and its related functionality. Below is a summary of the challenges we have encountered in this area over the last four months. Some of them we were able to overcome by modifying the engine, while other issues remain unsolved.

Catch Virtual Keyboard ‘show’ and ‘hide’ events.

UE does expose these events, but not in blueprints. You need to do it in C++. The solution is to bind these events in code and then propagate them up to the blueprint system (BPs) via custom delegates.

Then fire those to Blueprints:

At the time of this writing, in UE 5.2.1, only the OnVirtualKeyboardShown() event fires.

Trigger Virtual Keyboard opening in code.

By default, UE pops up the virtual keyboard (VK) once a user clicks inside the text input field. However, there may be cases where there is no text input, as in our situation. Our app sends typed text to the server immediately after the user commits the text. All we want to do is instruct UE to open the VK. UE5 has an interface for that, but it requires quite a non-trivial amount of work.

The ShowVirtualKeyboard() function requires an object that implements IVirtualKeyboardEntry. The idea is to create a connection between the VK’s input field and a user-defined widget to transfer the text string. You can refer to SVirtualKeyboardEntry as an example of how a potential implementation may look. It would be beneficial to have an optional interface that requires no more than a pointer to FString to populate with the text from VK, and a couple of callback functions to catch events such as 'text committed/changed'. This would be sufficient for use cases like the one I described above.

Customizing Virtual Keyboard text input UI.

Let’s say you want to develop a messenger app in Unreal Engine. It may sound like a crazy idea, but trust me, it is a quite realistic scenario.

The goal is to set up a chat UI, similar to WhatsApp, where there is a button to the right of the text input field. In fact, I am surprised that this problem hasn’t been addressed until now by Epic, because at least on my phone (Samsung Galaxy A53), the only way to commit text from VK is to click outside the widget, which, from a UX perspective, is an ugly workaround. No high-level API is available to customize VK layout in Blueprints or C++. The only way to add a custom view (UI widget in Android terminology) into VK’s text input line is to modify the following file.

Engine\Build\Android\Java\src\com\epicgames\unreal\GameActivity.java.template

UE uses this template to render the Android native UI, handle Android events, and dispatch them to the engine. The modified file can be found here. It should appear like this on an Android device:

Virtual keyboard text input bug in portrait mode.

In 5.2.1, there is a bug where the VK text input field only opens to half its height. This occurs only in portrait mode. To fix this in portrait, you should modify the aforementioned Java template file:

The fix is to add the following line:

in GameActivity.java.template file. The above fix is a courtesy of UE dev community.

Overall, the experience of developing for Android is not that bad. The most annoying part so far is testing the game on the target device. Packaging and deploying a game on an Android device feels slow. While I don’t have concrete metrics to compare with, overall, the debugging process feels wearisome. It would be nice to have VK working in the UE mobile emulator window for native UI debugging on the PC.