SlideViewer and the Display Window

10.06.2014 Sean Harmer No comments

Introduction

Following on from the previous articles on SlideViewer, we shall now investigate another piece of the puzzle towards making SlideViewer usable in practise. Namely, getting the content rendered from our domain specific language and on to the screen or projector for the audience to marvel at.

Those of you that have ever presented at conferences, business meetings or the like will know what a joy it is to configure $LAPTOP_BRAND running $OPERATING_SYSTEM in conjunction with $PROJECTOR or $MONITOR. As a group, the KDAB trainers have a range of options that we can use to hopefully make this easier but largely they boil down to two varieties:

• Mirror part or all of the display that the presenter can see or
• Put the display content onto a separate window

The first option is most easily achieved via system level tools specific to each platform. If it is only possible to mirror the entire display that gives the instructor more stress as then he/she can only see the same content as the audience. No presenter notes, tips, upcoming slides, timing information etc.

The second option is my preferred approach as it means I can have access to all those extra goodies to make my life as a presenter a little easier without distracting the audience. With this in mind, let’s see one possible solution to creating this configuration.

SlideViewer’s Main and Display Windows

SlideViewer is a typical hybrid C++/QML application. Where we start to break the usual pattern is when we come to want to display a subset of our UX, i.e. the slide content, in a separate window as described above. SlideViewer’s main window typically looks something like this:

Let’s examine our requirements:

• Display the content of the current slide in a separate window without other distracting information.
• Don’t create unnecessary copies of items to keep memory usage under control.
• Do not add a large performance overhead – animations should still be smooth.
• Show anything that QtQuick is capable of rendering.
• No need to handle events or input on display window, presenter will drive this from the main window.

We can start tackling the problem quite easily. From the requirements, we need a window on which to render our content. With Qt 5 this is trivial as all we need is a subclass of QWindow. At this stage our options are to use QQuickView which pulls in all of the QML machinery or to do something more simple ourselves. We don’t really want to have a second tree of QML items in our display window that mirrors the scene shown in our main window. This would lead to headaches keeping the two trees in sync with each other and use up more memory than is necessary. So let’s see about using QWindow and taking care of the rendering ourselves.

Creating a subclass of QWindow that will do as we require is fairly simple and we will fill in the details of this as we progress. To make our DisplayWindow class available to QML we register it with Qt in our main function with:

qmlRegisterType<DisplayWindow>("Cpp", 1, 0, "DisplayWindow");

and then within SlideViewer’s main qml file we can instantiate the DisplayWindow with:

import Cpp 1.0

ApplicationWindow
{
    id: rootWindow
    ...
    
    DisplayWindow {
        id: displayWindow
        title: "Presentation Window"
        visible: _settings.showPresentationWindow && !_settings.isPrinting
    }
    
    ...
}

Note that the title and visible properties are inherited from QWindow. The _settings object is a C++ object that has been exported to the QML runtime via the usual setContextProperty() approach and which contains a bunch of properties for various application settings.

Rendering to the Display Window

Recall that QtQuick 2 makes use of a 2D scenegraph to control its rendering. The scenegraph is great and takes care of all the headaches involved with taming the underlying OpenGL objects and draw calls. The fact that the QtQuick 2 scenegraph uses OpenGL is good as it means we can make use of this to help us render into our dispay window.

Wouldn’t it be great if we could get the scenegraph to render a sub-tree of our main window’s QML scene into an OpenGL texture that we could then re-use? Well, good news everybody! It turns out we can (otherwise I wouldn’t have mentioned it). Even better, there is already a Qt Quick 2 item that will do this for us and it is called ShaderEffectSource.

ShaderEffectSource is usually used in conjunction with it’s partner in crime, ShaderEffect to apply some fancy post-processing effects or mesh distortions to a piece of your QML scene. It works by rendering the QML sub-tree rooted at a specified sourceItem into an OpenGL framebuffer object (FBO) with a texture attached to the FBO’s first color attachment point. The resulting texture is then used by ShaderEffect in conjunction with some GLSL shader code to transform it in some way. There are lots of examples that ship with Qt demonstrating this.

In our case we don’t need the post-processing capabilities of ShaderEffect, all we need is to be able to get our hands on the texture rendered to by the scene graph for our sub-tree. The idea is that to render the contents of the DisplayWindow all we do is render a quad (having the same aspect ratio as the source item) that is textured using lookups from the texture containing our rendered QML sub-tree.

Why do I keep talking about sub-trees? Well, we only want to render the current slide in our display window and not the entire main window which is also part of our overall QML scene.

Let’s take another step closer to the solution and make the sub-tree for our current slide get rendered into a texture and somehow expose that texture to our DisplayWindow object. In QML we redirect the current slide into a texture and make the ShaderEffect available very easily with:

ApplicationWindow
{
    id: rootWindow
    ...

    ShaderEffectSource {
        id: displayWindowSource
        sourceItem: content
        hideSource: false
        width: sourceItem ? sourceItem.width : 0
        height: sourceItem ? sourceItem.height : 0
    }
    
    DisplayWindow {
        id: displayWindow
        source: displayWindowSource
        title: "Presentation Window"
        visible: _settings.showPresentationWindow && !_settings.isPrinting
    }
}

where content is the id of our current slide placeholder (we use a Loader item within content to load data when needed). We have added a source property to our DisplayWindow class in the usual way:

class DisplayWindow : public QWindow
{
    Q_OBJECT
    Q_PROPERTY(QQuickItem * source READ source WRITE setSource NOTIFY sourceChanged)
    ...
    
    void setSource(QQuickItem *source);
    QQuickItem * source() const;

    int currentScreen() const;

signals:
    void sourceChanged();
    
private:
    QQuickItem *m_source = nullptr;
};

All we do in the setSource() function is to store a pointer to the ShaderEffectSource item passed in from QML. The cunning part happens when we hook into the Qt Quick 2’s scenegraph rendering.

The rendering in Qt Quick 2 may or may not happen in a separate thread from your application’s main thread. This is determined mainly by the capabilities of your platform but it can be overridden by an environment variable if your platform is capable. With this in mind we can hook into the scenegraph’s usual rendering sequence and do some work of our own in there. We do this by connecting to a signal emitted by the scenegraph, QQuickWindow::afterRendering() that gets emitted once the scenegraph has done its work. At this stage we know that the texture used by the ShaderEffectSource will be populated with our sub-tree’s rendered output so we can borrow it to our own ends. We do this by making sure our rendering function gets called at this point by:

QQuickWindow * const newWindow = m_source ? m_source->window() : nullptr;
connect(newWindow, &QQuickWindow::afterRendering,
        this, &DisplayWindow::update, Qt::DirectConnection);

The use of Qt::DirectConnection takes care of making sure that our DisplayWindow::update() function is called in the correct thread context, i.e. that of the Qt Quick 2 scenegraph.

Within DisplayWindow::update() we have a few tasks to do:

• Make sure we have an OpenGL context with which we can draw
• Prepare any resources needed for rendering: shader programs, buffer objects of vertex data and of course…
• The texture that contains our rendered sub-tree, the current slide.

Initially when we coded up this feature in SlideViewer we tried making use of the scene graph’s OpenGL context as that is guaranteed to be current when our rendering function is called in response to the afterRendering signal. This worked fine on Linux but it seems the OS X Cocoa QPA plugin did not like us using this context to render to a different window surface. In theory this should not be a problem so I suspect there’s a subtle bug in the Cocoa QPA plugin. To work around this issue we actually create a second OpenGL context that is able to share resources (including textures) with that of the scene graph’s context:

auto * const window = m_source->window();
Q_ASSERT(window);
if (!m_context) {
    m_context.reset(new QOpenGLContext);
    m_context->setFormat(window->openglContext()->format());
    m_context->setShareContext(window->openglContext());
    m_context->create();
}

The preparation of OpenGL resources is performed only when required and consists of the usual business of compiling a trivial OpenGL shader program with QOpenGLShaderProgram and storing the vertex positions and texture coordinates of a single quad (two triangles) in a QOpenGLBuffer object.

The only non-obvious part is how to get hold of the texture containing the QML scene. Luckily, this is also straightforward but it does require the use of some private and undocumented APIs:

const QQuickShaderEffectSource * const sourceItem = qobject_cast(m_source);
const QSGTextureProvider * const provider = sourceItem->textureProvider();
const QSGTexture * const texture = provider->texture();
const GLint textureId = texture->textureId();

Once we have this we can bind the shader program, buffer object and then draw the two triangles making up the quad:

m_program->bind();
m_positions.bind();
...
m_indexBuffer.bind();

glBindTexture(GL_TEXTURE_2D, textureId);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
m_program->setUniformValue("texture0", 0);

glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_INT, 0);

And finally, to make sure that we actually get to see what we’ve just rendered we have to be sure to swap the front/back buffers (we’re using double buffering) and we release our context to clean up after ourselves:

m_context->swapBuffers(this);
m_context->doneCurrent();

Summary

So there you have it. We have made a simple sub-class of QWindow that is able to very efficiently render any sub-tree of a QML scene by making use of our knowledge of OpenGL which is used behind the scenes by Qt Quick 2’s scenegraph. In addition we have a nice simple way of exposing the DisplayWindow to QML so that we can easily control it from QML just like any other built-in element.

We are using the QQuickWindow::afterRendering() signal to drive our rendering and since all we are doing is drawing a single large textured quad we have very minimal impact on performance and memoryusage. Any animations included in the QML scene will work frame-perfect in the display window just as they do in the mainwindow.

Of course there’s a little more to it in the actual code as we have to care for window resizing, moving the display window to different desktops or outputs etc. but this is the basic approach.

This of course only scratches the surface of what we could do. Now that we have the QML sub-tree available as a texture we could use this with any number of post-processing or special effects in the display window. Some ideas are fancy transitions, animated 3D backgrounds etc. You could even potentially take this further and use a second QML scene in the display window that contains a custom item which uses our texture from the main window’s scene. Perhaps something for the future.

Sean Harmer is a senior software engineer at KDAB where he heads up our UK office and also leads the 3D R&D team. He has been developing with C++ and Qt since 1998 and is Qt 3D Maintainer and lead developer in the Qt Project. Sean has broad experience and a keen interest in scientific visualization and animation in OpenGL and Qt. He holds a PhD in Astrophysics along with a Masters in Mathematics and Astrophysics.