Shadow Mapping in Qt3D 2.0

Giuseppe D’Angelo

19 January 2015

Shadow mapping

Shadows are not directly supported by OpenGL, and these days there are countless techniques that can be employed to generate them. Shadow mapping is one of the oldest; it's still widely used due to its simplicity and ability to generate good-looking shadows, while having a very small performance cost. The Wikipedia entry on shadow mapping has a very good overview of the modern incantations of this technique. However, for our purposes, we are going to stick to a very basic version of it.

Shadow mapping is typically implemented using a two pass rendering. In the first pass we generate the shadow information, and in the second pass we can render the scene "normally" (i.e., using any rendering technique of our choice), while at the same time using the information gathered in the first pass to draw the shadows.

The idea behind shadow mapping is the following: only the closest fragments to the light are the ones lit. Fragments "behind" other fragments are occluded, and therefore in shadow.

Therefore, in the first pass we draw the scene from the point of view of the light. The information that we then store is simply the distance of the closest fragment in this "light space". In OpenGL terms, this corresponds to having a Framebuffer Object, or FBO, with a depth texture attached to it. In fact, the "distance from the eye" is the definition of the depth; and the default depth testing done by OpenGL will actually store only the depth for the closest fragment.

(A color texture attachment is not even needed -- we don't need to shade fragments, only to calculate their depth.)

Exaggerated shadow map texture of the very same scene represented above.

The image above is the shadow map. That is, the depth stored when rendering the scene from the light point of view; darker colours represent a shallow depth (i.e. closer to the camera). In our scene, the light sits somewhere above the objects in the scene, on the right side w.r.t. the main camera (cf. the previous screenshot). This matches with the fact that the toyplane is closer to the camera than the other objects.

Once we have generated the shadow map, we then do the second rendering pass. In this second pass we render using the normal scene's camera; we can use any desired effect here, like for instance Phong shading. The important bit is that in the fragment shader we apply the shadow map algorithm, that is, we ask: is that fragment the closest fragment to the light? If so, then it must be drawn lit; otherwise, it must be drawn in shadow.

How to answer that question is easy once we have the shadow map generated in the first pass. All it suffices is to remap the fragment in light space, therefore calculating its depth from the light point of view, as well as where its coordinates are on the shadow map texture. We can then sample the shadow map texture at the given coordinates and compare the fragment's depth with the result of the sampling: if the fragment is further away, then it's in shadow, otherwise it is lit.

This is the theory about shadow mapping. Let's see how to turn this theory into code using Qt3D.

Getting started

Let's start from the main.qml file, where we set up the entire scene.

import Qt3D 2.0
import Qt3D.Render 2.0

Entity {
    id: sceneRoot

    Camera {
        id: camera
        projectionType: CameraLens.PerspectiveProjection
        fieldOfView: 45
        aspectRatio: _window.width / _window.height
        nearPlane: 0.1
        farPlane: 1000.0
        position: Qt.vector3d(0.0, 10.0, 20.0)
        viewCenter: Qt.vector3d(0.0, 0.0, 0.0)
        upVector: Qt.vector3d(0.0, 1.0, 0.0)
    }

    Configuration  {
        controlledCamera: camera
    }

    Light {
        id: light
    }

    components: [
        ShadowMapFrameGraph {
            id: framegraph
            viewCamera: camera
            lightCamera: light.lightCamera
        }
    ]

    AdsEffect {
        id: shadowMapEffect

        shadowTexture: framegraph.shadowTexture
        light: light
    }

    // Trefoil knot entity
    Trefoil {
        material: AdsMaterial {
            effect: shadowMapEffect
            specularColor: Qt.rgba(0.5, 0.5, 0.5, 1.0)
        }
    }

    // Toyplane entity
    Toyplane {
        material: AdsMaterial {
            effect: shadowMapEffect
            diffuseColor: Qt.rgba(0.9, 0.5, 0.3, 1.0)
            shininess: 75
        }
    }

    // Plane entity
    GroundPlane {
        material: AdsMaterial {
            effect: shadowMapEffect
            diffuseColor: Qt.rgba(0.2, 0.5, 0.3, 1.0)
            specularColor: Qt.rgba(0, 0, 0, 1.0)
        }
    }
}

The first components we create are a Camera, which represents the camera used for the final rendering, and a Configuration element which allows us to control this camera using the keyboard or the mouse. The parameters of the camera are self-explainatory and there isn't much to say about that.

We then create a Light entity, which represents our light -- a directional spotlight, sitting somewhere above the plane, and looking down at the scene's origin. This light entity is then used by our custom frame graph, ShadowMapFrameGraph, and our rendering effect, AdsEffect, whose instances are created just after the light.

Lastly, we create three entities for the meshes in the scene: a trefoil knot, a toy aircraft, and a ground plane. The implementation of these three entities is straightforward and will not be covered here; they simply aggregate a mesh, a transformation and a material that uses the effect defined above. For extra fun, the toyplane and the trefoil knot transformations are actually animated.

Light

The Light element is defined inside Light.qml:

import Qt3D 2.0
import Qt3D.Render 2.0

Entity {
    id: root

    property vector3d lightPosition: Qt.vector3d(30.0, 30.0, 0.0)
    property vector3d lightIntensity: Qt.vector3d(1.0, 1.0, 1.0)

    readonly property Camera lightCamera: lightCamera
    readonly property matrix4x4 lightViewProjection: lightCamera.projectionMatrix.times(lightCamera.matrix)

    Camera {
        id: lightCamera
        objectName: "lightCameraLens"
        projectionType: CameraLens.PerspectiveProjection
        fieldOfView: 45
        aspectRatio: 1
        nearPlane : 0.1
        farPlane : 200.0
        position: root.lightPosition
        viewCenter: Qt.vector3d(0.0, 0.0, 0.0)
        upVector: Qt.vector3d(0.0, 1.0, 0.0)
    }
}

As I said before, the light is a directional spotlight. Since in the first rendering pass we're going to use the light as a camera, I decided to actually put a Camera sub-entity inside of it, and to expose it as a property. Apart from the camera, the light exposes as properties a position, its colour/intensity, and a 4x4 transformation matrix; we'll see where that matrix gets used, while the rest is straightforward.

Frame graph

In Qt3D 2.0 the frame graph is the data-driven configuration for the rendering. In this example, ShadowMapFrameGraph.qml contains its implementation, which looks like this:

import Qt3D 2.0
import Qt3D.Render 2.0
import QtQuick 2.2 as QQ2

FrameGraph {
    id: root

    property alias viewCamera: viewCameraSelector.camera
    property alias lightCamera: lightCameraSelector.camera
    readonly property Texture2D shadowTexture: depthTexture

    activeFrameGraph: Viewport {
        rect: Qt.rect(0.0, 0.0, 1.0, 1.0)
        clearColor: Qt.rgba(0.0, 0.4, 0.7, 1.0)

        RenderPassFilter {
            includes: [ Annotation { name: "pass"; value: "shadowmap" } ]

            RenderTargetSelector {
                target: RenderTarget {
                    attachments: [
                        RenderAttachment {
                            name: "depth"
                            type: RenderAttachment.DepthAttachment
                            texture: Texture2D {
                                id: depthTexture
                                width: 1024
                                height: 1024
                                format: Texture.DepthFormat
                                generateMipMaps: false
                                magnificationFilter: Texture.Linear
                                minificationFilter: Texture.Linear
                                wrapMode {
                                    x: WrapMode.ClampToEdge
                                    y: WrapMode.ClampToEdge
                                }
                                comparisonFunction: Texture.CompareLessEqual
                                comparisonMode: Texture.CompareRefToTexture
                            }
                        }
                    ]
                }

                ClearBuffer {
                    buffers: ClearBuffer.DepthBuffer

                    CameraSelector {
                        id: lightCameraSelector
                    }
                }
            }
        }

        RenderPassFilter {
            includes: [ Annotation { name: "pass"; value: "forward" } ]

            ClearBuffer {
                buffers: ClearBuffer.ColorDepthBuffer

                CameraSelector {
                    id: viewCameraSelector
                }
            }
        }
    }
}

The code defines a FrameGraph entity, which has a tree of entities as the active frame graph. Any path from the leaves of this tree to the root is a viable frame graph configuration; filter entities can enable or disable such paths, and selector entities can alter the configuration.

In our case, the tree looks like this:

Viewport
- RenderPassFilter
  - RenderTargetSelector
    - ClearBuffer
      - CameraSelector
- RenderPassFilter
  - ClearBuffer
    - CameraSelector

So we have two paths from the topmost Viewport entity. Each path corresponds to a pass of the shadow map technique; the paths are enabled and disabled using a RenderPassFilter, an entity that can filter depending on arbitrary values defined in a given render pass (in our case: a string). The actual passes are not defined here, but in the effect (see below); the frame graph simply modifies its configuration when a given pass is rendered.

Now, in the shadow map generation pass, we must render to an offscreen surface (the FBO) which has a depth texture attachment: this in Qt3D is represented by the RenderTarget entity, which has a number of attachments. In this case, only one attachment is needed: a depth attachment, defined by the RenderAttachment entity using a type of RenderAttachment.DepthAttachment (stating it should store the depth), and a Texture2D entity which actually configures the texture storage used to store the depth information.

Moreover, in this first pass, we must render using the light's camera; therefore, we have a CameraSelector entity that sets the camera to the one exported by the Light.

The second pass is instead way more straightforward, in which we simply render to the screen using the main camera.

The effect

The bulk of the magic happens in the AdsEffect.qml file, where our main Effect entity is defined. As you can imagine from the name, it's an effect implementing the ADS shading model, i.e. Phong, with the addition of shadow mapped generated shadows.

An effect contains the implementation of a particular rendering strategy; in this case, shadow mapping using two passes.

import Qt3D 2.0
import Qt3D.Render 2.0

Effect {
    id: root

    property Texture2D shadowTexture
    property Light light

    parameters: [
        Parameter { name: "lightViewProjection"; value: root.light.lightViewProjection },
        Parameter { name: "lightPosition";  value: root.light.lightPosition },
        Parameter { name: "lightIntensity"; value: root.light.lightIntensity },

        Parameter { name: "shadowMapTexture"; value: root.shadowTexture }
    ]

    techniques: [
        Technique {
            openGLFilter {
                api: OpenGLFilter.Desktop
                profile: OpenGLFilter.Core
                majorVersion: 3
                minorVersion: 2
            }

            renderPasses: [
                RenderPass {
                    annotations: [ Annotation { name: "pass"; value: "shadowmap" } ]

                    shaderProgram: ShaderProgram {
                        vertexShaderCode:   loadSource("qrc:/shaders/shadowmap.vert")
                        fragmentShaderCode: loadSource("qrc:/shaders/shadowmap.frag")
                    }

                    renderStates: [
                        PolygonOffset { factor: 4; units: 4 },
                        DepthTest { func: DepthTest.Less }
                    ]
                },

                RenderPass {
                    annotations: [ Annotation { name : "pass"; value : "forward" } ]

                    bindings: [
                        // Uniforms (those provided by the user)
                        ParameterMapping { parameterName: "ambient";  shaderVariableName: "ka"; bindingType: ParameterMapping.Uniform },
                        ParameterMapping { parameterName: "diffuse";  shaderVariableName: "kd"; bindingType: ParameterMapping.Uniform },
                        ParameterMapping { parameterName: "specular"; shaderVariableName: "ks"; bindingType: ParameterMapping.Uniform }
                    ]

                    shaderProgram: ShaderProgram {
                        vertexShaderCode:   loadSource("qrc:/shaders/ads.vert")
                        fragmentShaderCode: loadSource("qrc:/shaders/ads.frag")
                    }
                }
            ]
        }
    ]
}

The parameters list defines some default values for the effect. Those values will get mapped to OpenGL shader program uniforms, so that in the shaders we can access them. In this case, we expose some information from the Light entity (its position, its intensity, its view/projection matrix defined by its internal camera), as well as the shadow map texture exposed by the frame graph.

In general, it's possible to put such parameters all the way down, from a Material, to its Effect, to one of the effect's Techniques. This allows a Material instance to override defaults in an Effect or Technique. (The bindings array provides the same thing, except that it also allows us to rename some parameters. In our case, it renames the ambient/diffuse/specular values defined in the material to the actual uniform names used by the shader programs.)

We then have a Technique element. In order to be able to adapt the implementation to different hardware or OpenGL versions, an Effect is implemented by providing one or more Technique elements. In our case, only one technique is provided, targeting OpenGL 3.2 Core (or greater).

Inside that technique, we finally have the definition of our two rendering passes. We "tag" each pass with an Annotation entity, matching the ones we've set into the frame graph configuration, so that each pass will have different rendering settings.

The first pass is the shadow map generation. To do so, we load a suitable set of GLSL shaders, which are actually extremely simple -- they do nothing except from MVP projection, to bring meshes from their model space into clip space (and, remember, in this first pass, the light is the camera). The fragment shader is totally empty: there's no color to be generated, and the depth will be automatically captured for us by OpenGL. Note that in this first pass, we also set some custom OpenGL state in the form of a polygon offset and depth testing mode.

The second pass is instead a normal forward rendering using Phong shading. The code in the effect entity is extremely simple: we simply configure some parameters (see above) and load a pair of shaders which will be used when drawing.

The shaders

I will not explain the shader code in too much detail, because that would require a crash course in GLSL. However, I will explain the shadow mapping parts. The first part happens in the vertex shader (ads.vert), where we output towards the fragment shader the coordinates of each vertex in light space:

positionInLightSpace = shadowMatrix * lightViewProjection
        * modelMatrix * vec4(vertexPosition, 1.0);

(Actually, the coordinates get adjusted a little to allow us to easily sample the shadow map texture; that's the purpose of the shadowMatrix, please refer to a book or to the Wikipedia entry on shadow mapping to understand why that's necessary).

The second part happens in the fragment shader (ads.frag), where we sample the shadow map, and if the currently processed fragment is behind the one closest to the light, then the current fragment is in shadow (and only gets ambient contribution), otherwise it gets full Phong shading:

float shadowMapSample = textureProj(shadowMapTexture, positionInLightSpace);

    vec3 ambient = lightIntensity * ka;
    vec3 result = ambient;

    if (shadowMapSample > 0)
        result += dsModel(position, normalize(normal));

    fragColor = vec4(result, 1.0);

And that's it!

Conclusions

In this post I've shown how it's possible to configure Qt3D in order to achieve a custom rendering effect. Although shadow mapping is one of the simplest rendering techniques, the point is demonstrating how Qt3D imposes no particular rendering algorithm or strategy. You can easily experiment with a variety of multipass effects, e.g. introduce stencil shadows, or maybe that effect you've just seen on that SIGGRAPH paper...

About KDAB

Trusted software excellence across embedded and desktop platforms

The KDAB Group is a globally recognized provider for software consulting, development and training, specializing in embedded devices and complex cross-platform desktop applications. In addition to being leading experts in Qt, C++ and 3D technologies for over two decades, KDAB provides deep expertise across the stack, including Linux, Rust and modern UI frameworks. With 100+ employees from 20 countries and offices in Sweden, Germany, USA, France and UK, we serve clients around the world.

25 Comments

Do you need Qt 5.5 to build Qt3D?

Hi,

yes, you need a checkout from the "dev" branch of Qt (which currently corresponds to 5.5).

You don't need to clone everything, only qtbase and qtdeclarative are really needed, plus optionally qimageformats (to be able to load DDS textures). If you're using the "qt5" repository (i.e. init-repository / git submodules), be sure to checkout "dev" in each submodule instead of in the top level repository! Full instructions are available here.

Cool! I got it working. Is there any documentation that I could generate or look at?

Unfortunately not yet... but you're more than welcome to join the #qt-3d hannel on Freenode and ask around, there's usually always someone around (esp. in CET working hours).

What are the default uniforms that are set in the shader programs? I see you can add uniforms through Effect's parameters property. For example modelMatrix, modelView, modelViewNormal, mvp are all defined in ads.vert but aren't referenced in the qml. I'm assuming these are default uniforms set by qt3d? I'm assuming there's others default uniforms like viewMatrix or projectionMatrix?

You're absolutely correct :) There's a set of builtin uniforms that are set by the engine.

Take a look into RenderView::initializeStandardUniformSetters where you can find the list, it's basically the various combinations of model/view/projection/viewport matrices, their inverses and the elapsed time in the application.

Cool thanks.

Please let us know if there's other standard uniforms that you would like to see added to the list. Best bet is to file a JIRA task.

I need to try to implement a simple depth pealing algorithm to try test out the Framegraph / Effect / Entry system. I just haven't had time yet.

I would be interested to see the C++ version of this. I plan on using Qt3d for image generation due to Qt's renown documentation. I do all of my programming in C++.

Hi Jim,

a C++ version of this should not be much different, it would essentially be a 1:1 translation of the scene into C++, therefore becoming a bit more verbose.

For comparison, take a look at the various examples that are shipped both in QML and C++ form (e.g. simple-qml / simple-cpp).

Hello, I got Qt 5.5 dev branch and compiled Qt statically. All goes well except all the example not linked. It shows:

.obj/release/main.o:main.cpp:(.text+0x8a): undefined reference to `_imp___ZN4Qt3D6WindowC1EP7QScreen'
.obj/release/main.o:main.cpp:(.text+0x9e): undefined reference to `_imp___ZN4Qt3D5Quick16QQmlAspectEngineC1EP7QObject'
.obj/release/main.o:main.cpp:(.text+0xbe): undefined reference to `_imp___ZN4Qt3D13QRenderAspectC1EP7QObject'
.obj/release/main.o:main.cpp:(.text+0xc3): undefined reference to `_imp___ZNK4Qt3D5Quick16QQmlAspectEngine12aspectEngineEv'
.obj/release/main.o:main.cpp:(.text+0xdd): undefined reference to `_imp___ZN4Qt3D13QAspectEngine14registerAspectEPNS_15QAbstractAspectE'
.obj/release/main.o:main.cpp:(.text+0xff): undefined reference to `_imp___ZN4Qt3D12QInputAspectC1EP7QObject'
.obj/release/main.o:main.cpp:(.text+0x3d8): undefined reference to `_imp___ZN4Qt3D13QAspectEngine7setDataERK4QMapI7QString8QVariantE'
Makefile.Release:84: recipe for target 'release\playground-qml.exe' failed
.obj/release/main.o:main.cpp:(.text+0x3e9): undefined reference to `_imp___ZN4Qt3D13QAspectEngine10initializeEv'
.obj/release/main.o:main.cpp:(.text+0x427): undefined reference to `_imp___ZN4Qt3D5Quick16QQmlAspectEngine9setSourceERK4QUrl'
.obj/release/main.o:main.cpp:(.text+0x4a7): undefined reference to `_imp___ZTVN4Qt3D5Quick16QQmlAspectEngineE'
.obj/release/main.o:main.cpp:(.text+0x4c1): undefined reference to `_imp___ZN4Qt3D6WindowD1Ev'
.obj/release/main.o:main.cpp:(.text+0x5b5): undefined reference to `_imp___ZN4Qt3D6Window16staticMetaObjectE'
E:/Develop/Qt5.4/Tools/mingw491_32/bin/../lib/gcc/i686-w64-mingw32/4.9.1/../../../../i686-w64-mingw32/bin/ld.exe: .obj/release/main.o: bad reloc address 0x14 in section `.text$_Z17qRegisterMetaTypeIP8QSurfaceEiPKcPT_N9QtPrivate21MetaTypeDefinedHelperIS4_Xaasr12QMetaTypeId2IS4_E7DefinedntsrS9_9IsBuiltInEE11DefinedTypeE[__Z17qRegisterMetaTypeIP8QSurfaceEiPKcPT_N9QtPrivate21MetaTypeDefinedHelperIS4_Xaasr12QMetaTypeId2IS4_E7DefinedntsrS9_9IsBuiltInEE11DefinedTypeE]'

Looking deep in the code, maybe it is probably due to the namespace of "Qt3D" mobule. I am not sure.

I am using Windows with mingw4.91

I notice that all examples requires "play-ground-qml" project, which is the not linked.

They all require the simple exampleresources project. We'll need to look into this. Last time I tried a static build it was on Linux and the examples linked and executed there. For now, I'd suggest using a dynamic build until we can get this resolved. Patches welcome ;)

Is it because of the the export keyword? I noticed that Qt3D Core defines only shared export, not static export:

#if defined(QT3DCORE_LIBRARY)
#  define QT3DCORESHARED_EXPORT Q_DECL_EXPORT
#else
#  define QT3DCORESHARED_EXPORT Q_DECL_IMPORT
#endif

If it's a static build there's no need to export anything. That export keyword is actually just a macro for Q_DECL_EXPORT/Q_DECL_IMPORT which in turn are macros for the platform specific export/visibility compiler intrinsics. In a static build everything is in the same executable.

In addition, the original exampleresources.pri got not compiled. it is suggested to add "lib" prefix at line 6 and 8 before "exampleresources". the final looks like this:

INCLUDEPATH +=  $$PWD

win32 {
    build_pass {
        CONFIG(debug, debug|release) {
            LIBS += $$shadowed($$PWD)/debug/libexampleresources.$${QMAKE_EXTENSION_STATICLIB}
        } else {
            LIBS += $$shadowed($$PWD)/release/libexampleresources.$${QMAKE_EXTENSION_STATICLIB}
        }
    }
} else {
    LIBS += $$shadowed($$PWD)/libexampleresources.$${QMAKE_EXTENSION_STATICLIB}
}

Ah looks like a MSVC vs mingw issue in the .pro and .pri files then.

Problem solved.

Maybe all the #ifdef directives have to be changed. Taken from another module of Qt, Enginio have long been unsuccessfully linked. I've fired quite a lot post to the developers. Now it is Okay with static build. Looking deep in code, it is written like this:

#if defined(QT_SHARED) || !defined(QT_STATIC)
#  if defined(ENGINIOCLIENT_LIBRARY)
#    define ENGINIOCLIENT_EXPORT Q_DECL_EXPORT
#  else
#    define ENGINIOCLIENT_EXPORT Q_DECL_IMPORT
#  endif
#else
#  define ENGINIOCLIENT_EXPORT
#endif

I've changed all the directives in Qt 3D to look similar to this, and get compiled and linked. I suggest change all the directives to adapt to static build in MinGW. or make a bunch of new directives such as "QT3D_STATIC_EXPORT".

Thanks, peppe has pushed a patch with this fix in. Should land once gerrit is fixed.

Hi, I have problems running Qt3D shadow map example. After starting the application I can only see a black screen in the app window. This happens with all multi-pass rendering examples(multiview,...). I have tried to run the same example(s) on another PC with other hardware and windows version. Beside Qt3D I have tried many other multi-pass rendering methods. These all run fine. I hope you can give me fast a useful hint. Thanks in advance Marco

Hi Marco, I think the example was accidentally broken, but it should now work in the 5.6 branch.

hey there,

something im wondering about is: I load a mesh from a .obj file next i want to set a center/origin for this object.

is this possible? for example Transform{ id: .... Rotate{id: ...; origin/center: Qt.vector3d(x, y, z} }

cheers

Hi Dominik,

There are now actually helpers for "rotate around a point" kind of scenarios. Fetch the code from the 5.6 branch and take a look for the QTransform::rotateAround helper (also exposed in QML).

Hi Giuseppe. Congratulations for this article. Very enlightening.

I've been studying this Qt example and found your post. I have one doubt: is it possible to use the Framebuffer Object - the rendered shadow map texture, in this case - as texture source of any QML component instanciated at the root window, above Scene3D?

If so, how to do that? Thanks for any kind of help.

Giuseppe D’Angelo

Senior Software Engineer

Senior Software Engineer at KDAB. Giuseppe is a long-time contributor to Qt, having used Qt and C++ since 2000, and is an Approver in the Qt Project. His contributions in Qt range from containers and regular expressions to GUI, Widgets, and OpenGL. A free software passionate and UNIX specialist, before joining KDAB, he organized conferences on opensource around Italy. He holds a BSc in Computer Science.

Shadow Mapping in Qt3D 2.0

Shadow mapping

Getting started

Light

Frame graph

The effect

The shaders

Conclusions

Trusted software excellence across embedded and desktop platforms

25 Comments

Related Content

Writing a Custom Qt 3D Aspect – part 1

Writing a Custom Qt 3D Aspect – part 2

nanoQuill

Join KDAB Training Day 2025 in Munich

Sign up for the KDAB Newsletter