Qt Input Method – In Depth

18.06.2015 Tobias Koenig 8 comments

What is an Input Method and what do we need it for?

To answer that question, let’s see what Wikipedia says:

“An input method (or input method editor, commonly abbreviated IME) is an operating system component or program that allows any data, such as keyboard strokes or mouse movements, to be received as input. In this way users can enter characters and symbols not found on their input devices.”

So an input method allows you for example to input Chinese, Japanese, Korean or Indian characters into a text input field of an application, even though there is only a Latin keyboard attached to the computer. That is done by analyzing the text, which is typed in as Latin, and e.g. opening a popup menu with a pre-selection of Chinese characters, which are associated with that Latin input. The user can now select one of these Chinese characters, which will then replace the Latin input in the text field.

Since input method support is nowadays an important requirement for UI toolkits, it doesn’t come as a surprise that our favorite one (aka Qt) provides it as well. But since Qt is cross-platform, it doesn’t support only a single input method, but various ones, depending on the target platform.

In this blog post, we’ll have a closer look at how that is implemented, what classes are involved and where it can be extended or adapted.

Input methods in Qt

To get an overview of the input method handling in Qt, let’s have a closer look at the involved components first:

QPA plugin: Since version 4.8, Qt contains an abstraction layer (Qt Platform Abstraction), to simplify the porting of the UI parts of Qt to new windowing systems. This layer consists of a set of abstract interface classes (QPlatform*), which are reimplemented for the target windowing system and are bundled as a plugin that can be loaded at runtime. The central interface is QPlatformIntegration, which is instantiated from the plugin by QGuiApplication on startup, and provides access to all the other QPA interfaces.

QPlatformInputContext: The QPlatformInputContext is an abstract interface for the various input method systems that are available on the different windowing systems. For every supported system, there is a sub class of QPlatformInputContext, which implements the communication with the actual input method backend (e.g. QIBusPlatformInputContext for IBus). These subclasses are either available as standalone plugins, which can be loaded at runtime (Qt 5.4 ships two of them, ‘ibus’ and ‘compose’), or are compiled into the QPA plugin (e.g. QQnxInputContext for QNX and QWindowsInputContext for Windows). QPlatformInputContext is part of Qt’s private API, so as an application developer, you won’t access it directly, instead the global instance of the public class QInputMethod is used. This global instance is returned by QGuiApplication::inputMethod() and it simply forwards the calls to its methods to the global QPlatformInputContext instance. The latter is provided by the loaded QPA plugin (see QPlatformIntegration::inputContext()), which returns one of the above mentioned subclasses.

QGuiApplication: The QGuiApplication inherits QCoreApplication and therefore contains the main event loop of the application, which is responsible for forwarding incoming events from the native windowing system to the appropriated objects in the application. While QCoreApplication only knows about the UI-independent events, the QGuiApplication has knowledge about UI-related states, like which widget has the active keyboard focus, what the current input method is etc.

Text input widgets: The purpose of the text input widgets (e.g. QLineEdit or QTextEdit) is to visualize the keyboard input of the user in a textual representation. Since the key events, which come in from the native windowing system, cannot always be mapped to output characters one-to-one, the text input widgets need support from the input method system to do the conversion.

Follow the stream

While we now know the major players in the game and their roles, in the next step we’ll see how they interact with each other. For that, we follow a key stroke on the keyboard throughout the complete system until it shows up as character in a QLineEdit.

The first thing we need in our application is an input widget, which is sensitive to keyboard input. That is easily done by instantiating a QLineEdit. If the user now wants to type in a Chinese character, the first of his actions would be to give this QLineEdit the input focus by either clicking on it with the mouse pointer or navigating to it via the tab chain. As soon as the QLineEdit receives the focus, a private slot in QGuiApplication is called (_q_updateFocusObject), which executes the following steps:

1. check if the QPlatformIntegration provides a QPlatformInputContext object
2. if so, check if the object with the input focus wants to use the input method system or handle key events itself
3. informs the QPlatformInputContext about the new focus object

The first check is easily done, since the QGuiApplication has loaded the QPA plugin itself, so it has access to the QPlatformIntegration instance and can simply call the QPlatformIntegration::inputContext() method to check if a valid QPlatformInputContext object is returned.

The second check is a bit more advanced. To decouple the QGuiApplication from the QWidget interface (and e.g. support also focus handling for QQuickItem), it cannot just call a method on the focus object to query its properties, since that would require QGuiApplication to know their API. Instead QGuiApplication sends a synchronous event (a blocking invocation via QGuiApplication::sendEvent(QEvent*)) to the focus object. Then the focus object fills the event object with the requested information, and when the blocking call returns, the QGuiApplication extracts the information from the event. In short: synchronous event sending is used to decouple QGuiApplication from the public interface of the focus objects.

So what does the event look like and what information can be queried? QInputMethodQueryEvent is the actual event that is used to query the information, and it allows QGuiApplication to query information from the focus object like:

• does it accept input method input (Qt::ImEnabled)
• the text around the current input area (Qt::ImSurroundingText)
• preferred input language (Qt::ImPreferredLanguage) and many, many more (see Qt::InputMethodQuery).

In our case QGuiApplication sends a QInputMethodQueryEvent to QLineEdit and asks for the Qt::ImEnabled flag. QLineEdit responds to that event in the reimplemented QWidget::inputMethodQuery() method, by checking if the Qt::WA_InputMethodEnabled widget attribute is set. This attribute needs to be set by any widget that wants to use the input method system and is set by default on the text input classes (QLineEdit, QTextEdit etc.) in Qt.

The last of the steps executed by _q_updateFocusObject() is to inform the QPlatformInputContext object about the new focus object, so that it can query further information from the focus object if needed later on during the input process. That is simply done by invoking QPlatformInputContext::setFocusObject(QObject *object).

Now that the QLineEdit has the keyboard focus, the user might press a key on the keyboard, which will trigger an input event in the operating system, which will be forwarded to the windowing system and from there call some function in the currently loaded QPA plugin. At that point the QPA plugin would transform the native key event into a QKeyEvent and inject it into the applications event queue by calling QWindowSystemInterface::handleKeyEvent() or QWindowSystemInterface::handleExtendedKeyEvent(). If however an input method system is active, (in that case (QPlatformIntegration::inputContext() returns a valid QPlatformInputContext object), it will send the raw input data to the QPlatformInputContext object instead. How the raw input data are sent to the QPlatformInputContext in that case is not defined by any API, so every QPA implementation is free to choose how to do it. The XCB QPA plugin for example expects the QPlatformInputContext to provide a public slot ‘x11FilterEvent’, which it invokes with QMetaObject::invokeMethod() and passes the xcb_keysym* data as parameters. That dynamic invocation allows us to use different QPlatformInputContext implementations on XCB systems, without having the XCB QPA plugin know the exact class interface. The QNX QPA plugin on the other hand has the QQnxInputContext compiled in, so it has a pointer to that instance and simply calls a method to forward the raw input data to it.

The QPlatformInputContext subclass has got the raw input data now and can forward them to the actual platform specific input method backend (e.g. send to IBus server via DBus). At that point the input method backend might require additional information to process the current raw input data. Example information could be:

• the text around the current input area, to interpret the new input data in a context
• the location of the cursor, to open character selection popup menu next to it

To query this information, the QPlatformInputContext sends a synchronous QInputMethodQueryEvent to the current focus object again, exactly as QGuiApplication has done it before. Once it has retrieved this information from the focus object, and forwarded it to the input method backend, the backend will compose a final character (or sequence of characters) that should be set as the new text on the QLineEdit. The composing is either done programatically (by following some writing system specific rules), or the input method system opens a popup menu with a pre-selection of possible characters from which the user selects an appropriated character.

So how does the composed text get back to the QLineEdit? For that, another synchronous event invocation is done. The QPlatformInputContext creates an instance of QInputMethodEvent and calls setCommitString() on it, with the newly composed text as parameter. Afterwards it sends the event to the focus object.

On the focus object, the QLineEdit in our case, the reimplemented method QWidget::inputMethodEvent(QInputMethodEvent*) is invoked. That method will update the current text of the QLineEdit with the composed text, reposition the text cursor and possibly update the current selection.

At that point, the key press event has reached its destination and the user is going to press the next key on the keyboard.

In addition to the commit string, the QPlatformInputContext could also create a QInputMethodEvent with a preedit string and send it to the focus object before composing has started or while composing is in progress. That preedit string is then shown inside the QLineEdit as intermediate result. The visual appearance of that text can be influenced by setting certain attributes on the QInputMethodEvent.

In the next blog post we will learn how to implement an out-of-process virtual keyboard, that uses Qt’s input method framework to communicate with Qt-based applications. Stay tuned!