Overview of Qt3D 2.0 – Part 1
16.12.2014
Introduction
Back in the days when Qt was owned by Nokia, a development team in Brisbane had the idea of making it easy to incorporate 3D content into Qt applications. This happened around the time of the introduction of the QML language and technology stack, and so it was only natural that Qt3D should also have a QML based API in addition to the more traditional C++ interface like other frameworks within Qt.
Qt3D was released alongside Qt 4 and saw only relatively little use before Nokia decided to divest Qt to Digia. During this transition, the Qt development office in Brisbane was closed and unfortunately Qt3D never saw a release alongside Qt 5. This chain of events left the Qt3D code base without a maintainer and left to slowly bit rot.
With OpenGL taking a much more prominent position in Qt 5’s graphical stack — OpenGL is the underpinning of Qt Quick 2’s rendering power — and with OpenGL becoming a much more common part of customer projects, KDAB decided that it would be good for us and for the Qt community at large if we took over maintainership and development of the Qt3D module. To this end, several KDAB engineers have been working hard to bring Qt3D back to life and moreover to make it competitive to other modern 3D frameworks.
This article is the first in a series that will cover the capabilities, APIs, and implementation of Qt3D in detail. Future articles will cover how to use the API in various ways from basic to advanced with a series of walked examples. For now, we will begin in this article with a high-level overview of the design goals of Qt3D; some of the challenges we faced; how we have solved them; what remains to be done before we can release Qt3D 2.0; and what the future may bring beyond Qt3D 2.0.
What Should Qt3D Do?
When asked what a 3D framework such as Qt3D should actually do, most people unfamiliar with 3D rendering simply say something along the lines of “I want to be able to draw 3D shapes and move them around and move the camera”. This is, of course, a sensible baseline, but when pressed further you get back wishes that typically include the following kinds of things:
- 2D and 3D
- Meshes
- Materials
- Shadows
Then, when you move on and ask the next target group, those who already know about the intricacies of 3D rendering, you get back some more technical terms such as:
- Shaders
- Ambient occlusion
- High dynamic range
- Deferred rendering
- Multitexturing
- Instancing
- Uniform Buffer Objects
- That cool technique I saw at SIGGRAPH…
That is already a fairly complex set of feature requests, but the real killer is that last entry which translates into ‘I want to be able to configure the renderer in ways you haven’t thought of’. Given that Qt3D 1.0 offered both C++ and QML APIs this is something that we wished to continue to support, but when taken together with wanting to have a fully configurable renderer this led to quite a challenge. In the end, this has resulted in something called the framegraph.
Framegraph vs Scenegraph
A scenegraph is a data-driven description of what to render. The framegraph is a data-driven description of how to render.
Using a data-driven description in the framegraph allows us to choose between a simple forward renderer; or including a z-fill pass; or using a deferred renderer; when to render any transparent objects etc. Also, since this is all configured purely from data, it is very easy to modify even dynamically at runtime. All without touching any C++ code at all!
Once you move beyond the essentials of getting some 3D content on to the screen, it becomes apparent that people also want to do a lot of other things related to the 3D objects. The list is extensive and wide ranging but very often includes requests like:
- Physics simulation
- Collision detection
- 3D positional audio
- Animation: rigid body, skeletal, morph target
- Path finding and other AI
- Picking
- Particles
- Object spawning
- …
This is obviously a tall order, and one that we couldn’t possibly hope to satisfy out of the box with the limited resources available. However, it is clear, that in order to support these features in the future, we needed to do some ground work now to architect Qt3D 2.0 to be extensible and flexible enough to act as a host for such extensions. The work around this topic took a lot of effort and several aborted prototypes before we settled on the current design. We will introduce the resulting architecture later and then cover it in more detail in an upcoming article.
Beyond the above short and long term feature goals, we also wanted to make Qt3D perform well and scale up with the number of available CPU cores. This is important given how modern hardware is improving performance — by increasing the numbers of cores rather than base clock speed. Also, when analysing the above features we can intuitively hope that utilising multiple cores will work quite naturally since many tasks are independent of each other. For example, the operations performed by a path finding module will not overlap strongly with the tasks performed by a renderer (except maybe for rendering some debug info or statistics).
Overview of the Qt3D 2.0 Architecture
The above set of requirements turned out to be quite a thorny problem, or rather a whole set of them. Fortunately, we think we have found solutions to most of them and the remaining challenges look achievable.
For the purposes of discussion, let’s start at the high-level and consider how to implement a framework that is extensible enough to deal with not just rendering but also all of the other features plus more that we haven’t though of.
At its heart, Qt3D is all about simulating objects in near-realtime, and then very likely then rendering the state of those objects onto the screen somehow. Let’s break that down and start with asking the question: ‘What do we mean by an object?’
Of course in such a simulation system there are likely to be numerous types of object. If we consider a concrete example this will help to shed some light on the kinds of objects we may see. Let’s consider something simple, like a game of Space Invaders. Of course, real-world systems are likely to be much more complex but this will suffice to highlight some issues. Let’s begin by enumerating some typical object types that might be found in an implementation of Space Invaders:
- The player’s ground cannon
- The ground
- The defensive blocks
- The enemy space invader ships
- The enemy boss flying saucer
- Bullets shot from enemies and the player
In a traditional C++ design these types of object would very likely end up implemented as classes arranged in some kind of inheritance tree. Various branches in the inheritance tree may add additional functionality to the root class for features such as: “accepts user input”; “plays a sound”; “can be animated”; “collides with other objects”; “needs to be drawn on screen”.
I’m sure you can classify the types in our Space Invaders example against these pieces of functionality. However, designing an elegant inheritance tree for even such a simple example is not easy.
This approach and other variations on inheritance have a number of problems as we will discuss in a future article but includes:
- Deep and wide inheritance hierarchies are difficult to understand, maintain and extend.
- The inheritance taxonomy is set in stone at compile time.
- Each level in the class inheritance tree can only classify upon a single criteria or axis.
- Shared functionality tends to ‘bubble up’ the class hierarchy over time.
- As library designers we can’t ever know all the things our users will want to do.
Anybody that has worked with deep and wide inheritance trees is likely to have found that unless you understand, and agree with, the taxonomy used by the original author, it can be difficult to extend them without having to resort to some ugly hacks to bend classes to our will.
For Qt3D, we have decided to largely forego inheritance and instead focus on aggregation as the means of imparting functionality onto an instance of an object. Specifically, for Qt3D we are using an Entity Component System (ECS). There are several possible implementation approaches for ECSs and we will discuss Qt3D’s implementation in detail in a later article but here’s a very brief overview to give you a flavour.
An Entity represents a simulated object but by itself is devoid of any specific behaviour or characteristics. Additional behaviour can be grafted on to an entity by having the entity aggregate one or more Components. A component is a vertical slice of behaviour of an object type.
What does that mean? Well, it means that a component is some piece of behaviour or functionality in the vein of those we described for the objects in our Space Invaders example. The ground in that example would be an Entity with a Component attached that tells the system that it needs rendering and how to render it; An enemy space invader would be an Entity with Components attached that cause them to be rendered (like the ground), but also that they emit sounds, can be collided with, are animated and are controlled by a simple AI; The player object would have mostly similar components to the enemy space invader, except that it would not have the AI component and in its place would have an input component to allow the player to move the object around and to fire bullets.
On the back-end of Qt3D we implement the System part of the ECS paradigm in the form of so-called Aspects. An aspect implements the particular vertical slice of functionality imbued to entities by a combination of one or more of their aggregated components. As a concrete example of this, the renderer aspect, looks for entities that have mesh, material and optionally transformation components. If it finds such an entity, the renderer knows how to take that data and draw something nice from it. If an entity doesn’t have those components then the renderer aspect ignores it.
Qt3D is an Entity-Component-System
Qt3D builds custom Entities by aggregating Components that impart additional capabilities. The Qt3D engine uses Aspects to process and update entities with specific components.
Similarly, a physics aspect would look for entities that have some kind of collision volume component and another component that specifies other properties needed by such simulations like mass, coefficient of friction etc. An entity that emits sound would have a component that says it is a sound emitter along with when and which sounds to play.
A very nice feature of the ECS is that because they use aggregation rather than inheritance, we can dynamically change how an object behaves at runtime simply by adding or removing components. Want your player to suddenly be able to run through walls after gobbling a power-up? No problem. Just temporarily remove that entity’s collision volume component. Then when the power-up times out, add the collision volume back in again. There is no need to make a special one-off subclass for PlayerThatCanSometimesWalkThroughWalls.
Hopefully that gives enough of an indication of the flexibility of Entity Component Systems to let you see why we chose it as the basis of the architecture in Qt3D. Within Qt3D the ECS is implemented according to the following simple class hierarchy.
Qt3D’s ‘base class’ is QNode which is a very simple subclass of QObject. QNode adds to QObject the ability to automatically communicate property changes through to aspects and also an ID that is unique throughout the application. As we will see in a future article, the aspects live and work in additional threads and QNode massively simplifies the tasks of getting data between the user-facing objects and the aspects. Typically, subclasses of QNode provide additional supporting data that is then referenced by components. For example a QShaderProgram specifies the GLSL code to be used when rendering a set of entities.
Components in Qt3D are implemented by subclassing QComponent and adding in any data necessary for the corresponding aspect to do its work. For example, the Mesh component is used by the renderer aspect to retrieve the per-vertex data that should be sent down the OpenGL pipeline.
Finally, QEntity is simply an object that can aggregate zero or more QComponent’s as described above.
To add a brand new piece of functionality to Qt3D, either as part of Qt or specific to your own applications, and which can take advantage of the multi-threaded back-end consists of:
- Identify and implement any needed components and supporting data
- Register those components with the QML engine (only if you wish to use the QML API)
- Subclass QAbstractAspect and implement your subsystems functionality.
Of course anything sounds easy when you say it fast enough, but after implementing the renderer aspect and also doing some investigations into additional aspects we’re pretty confident that this makes for a flexible and extensible API that, so far, satisfies the requirements of Qt3D.
Qt3D has a Task-Based Engine
Aspects in Qt3D get asked each frame for a set of tasks to execute along with dependencies between them. The tasks are distributed across all configured cores by a scheduler for improved performance.
Summary
We have seen that the needs of Qt3D extend far beyond implementing a simple forward-renderer exposed to QML. Rather, what is needed is a fully configurable renderer that allows to quickly implement any rendering pipeline that you need. Furthermore, Qt3D also provides a generic framework for near-realtime simulations beyond rendering. Qt3D is cleanly separated into a core and any number of aspects that can implement any functionality they wish. The aspects interact with components and entities to provide some slice of functionality. Examples of future possible aspects include: physics, audio, collision, AI, path finding.
In the next part of this series, we shall demonstrate how to use Qt3D and the renderer aspect to produce a custom shaded object and how to make it animate all from within QML.
About KDAB
KDAB is a consulting company offering a wide variety of expert services in Qt, C++ and 3D/OpenGL and providing training courses in:
KDAB believes that it is critical for our business to contribute to the Qt framework and C++ thinking, to keep pushing these technologies forward to ensure they remain competitive.
Are the new classes like QEntity and QComponent going to be tied to Qt3D or will they be usable for other things?
They are located in the Qt3D module but as far as being coupled to the rest of Qt3D goes, they are relatively independent. The QNode class automatically sends change notifications to something called the QChangeArbiter (if it knows about one). Otherwise they are free to be reused as you see fit but of course you would need to link to the Qt3DCore library, but that is pretty lightweight.
Great job. Unfortunately, I can not compile the library.
My method:
1. git clone git://gitorious.org/qt/qt5.git qt5
2. cs qt5
3. git checkout dev
5. perl init-repository –module-subset=qtbase,qtsvg,qtdeclarative,qttools,qtxmlpatterns,qtdoc,qlalr,qtrepotools,qtqa,qtlocation,qtgraphicaleffects,qtimageformats,qtxmlpatterns,qtquickcontrols,qt3d
6. git clone -b dev git://gitorious.org/qt/qt3d.git qt3d
7. ./configure -developer-build -opensource -nomake examples -nomake tests
8. make -j4
Results:
1. Qt compiled successfully.
2. qtbase/qml – I don’t see here Qt 3D plugins.
3. examples (c++ and qml) don’t compiles / works at all.
Can you help me?
Currently the qt5.git repo is lagging behind a bit far so we need to explicitly get the HEAD of the dev branch for Qt3D and its dependencies. So after your step 6 do:
for i in qtbase qtsvg qtxmlpatterns qtdeclarative qt3d; do cd $i; git checkout dev; git pull; cd ..; done
or the equivalent in windows.
Then do build, I usually use a shadow build (except on Windows where I’ve had trouble with shadw builds) so:
On Unix:
* mkdir qt-5.5.0-dev
* cd qt-5.5.0-dev
* ../configure -developer-build -opensource -confirm-license -nomake tests
* make module-qt3d
Or on Windows:
* configure -developer-build -opensource -confirm-license -nomake tests -opengl desktop
* make module-qt3d
You didn’t see any examples built as you passed in -nomake-examples to your configure line. You can still cd into the examples directory of each module and do qmake, make.
Hope this helps.
OK, so you took an approach with entities very similar to the one used in Unity3D GameObjects. I am not sure if this was a source of inspiration or not, but at the end it is imo a very good decision.
The Unity object model was indeed one of the sources of inspiration but there were also many others after researching a number of engines. Unity is of course a very mature product so we are not aiming to compete with that. What we hope for Qt3D is to allow developers to easily get 3D content into their applications but still have a good degree of control over the renderer without having to learn all the intricacies of advanced OpenGL usage. We do aim to make Qt3D extensible and configurable to support more advanced use cases though.
yeah, Blender uses a similar model. It seems to be the optimal way to represent 3d objects in a scene graph.
Hos does this compare to Ogre and/or the Qt-binding to Ogre?
Well, Ogre and Qt3D are both scenegraphs with some additional bits added. Ogre is obviously much more mature and feature complete as it has been around for much longer. Qt3D has a nice simple QML API though and as we’ll see in Part 2, this makes it very easy to render with OpenGL even if you want to do things like custom materials, deferred rendering or many other things.
Wait, so QEntity and QComponent are staying independent?
QEntity and QComponent are fairly loosely coupled to the rest of Qt3D. They are the main building blocks for the user-facing object tree which consists almost entirely of simple data. The real meat of Qt3D is in the backend QAspectEngine and the aspects themselves.
This is great stuff. How complete is the current project, is it usable? Is Qt3d 2.0 officially released? When is the next article?
Qt3D is still in heavy development but it can be used to render a lot of things already. I’ve had geometry and tessellation shaders running; it’s possible to populate UBOs and uniform arrays from items in the scenegraph; the framegraph can easily be used to change the rendering order, filter, select different cameras, render targets etc.
Part 2 of the article will expand on what remains to be done and will be published in a few days time.
We are aiming for a release alongside Qt 5.5 in the spring but we still have quite a bit to do.
> For now, we will begin in this article with … what remains to be done before we can release Qt3D 2.0;
Can’t find this topic in the article.
Oops, bad editing on my part. The article ended up rather long so I split it into two parts. That section will be in part 2 which will land in a few days time.
Dear Mr. Harmer:
My I use Qt 5.4.0 (community) with your Qt3D? If so, where can I download it?
Thanks! Look forward to your reply!
Unfortunately, Qt3D requires the dev branch of Qt (what will become Qt 5.5.0) as we needed to add some small features to QtGui. So if you want to try it you’ll need to build Qt from git.
Thanks for the great work. I have a short question regarding the tasks. If I remember correctly, you mentioned on the DevDays that TBB is or might be used for the task based implementation. Is there a reason why Qt’s built in functionality (e.g. QThreadPool) is not used?
At present we are using ThreadWeaver which is built on top of QThread as we needed dependency handling between the tasks. Miikka from The Qt Company is looking into extending QtConcurrent to add in the missing bits so that we can use it in Qt3D. The hooks and interfaces are there to replace the task scheduler with Intel TBB though if desired. Just requires someone to do the work. Could easily be made into a compile time configuration option.
Very exciting to know that this project is coming back and to hear about the directions in which the team has decided to take it. Tremendously cool. Thank you and best wishes!
Don’t forget that QT can be used for CAE , not just games. A missing piece is advanced visualization such as provided by VTK. The QT Enterprise Data Visualization has some nice features, but it’s in its own world at the moment.
I use OpenSceneGraph (OSG) with QT and it is possible to bridge VTK and OSG to get advanced visualization capabilities but its a bit messy with duplicated data. I personally have been very happy with OSG and it’s capabilities/performance.
Don’t worry this is firmly in our view too. We work on projects in this domain and we want to use Qt3D here too. There is nothing specific to games here. The architecture is agnostic of use case. You could equally well implement an aspect that checks for machine clearances or similar. Using custom materials and shaders it will also be possible to do things likemarching cubes surface reconstruction on the GPU too.
+1 for CAE, or any data-rich interface. We have too much information to present in a typical 2D or widget-based interface, and we’re looking seriously at 3D to solve this data-presentation problem. Qt3D looks like a GREAT design for what we want!
Hey Charley, if you could give us a little more info on the sort of thing you’re looking for it may help us shape the API better. Feel free to drop me an email or drop in on irc.
Thank you very much for posting this!
Perfect timing – I have quite a colorful class hierarchy in my application and something always felt wrong about that. What you’re describing seems exactly what I need right now.
Really hope you find the time to put up the second part soon (just hit ‘publish’ between two spoons of plum pudding! 😉
Thanks again and merry christmas! 🙂
Could I render a QML(2D) page to a item3D with viewport? I want to rotate a QML page with viewport rotation.
Thanks!
Hi Sean
I have runn the following on a linix box:
git clone git://gitorious.org/qt/qt5.git qt5
cd qt5
git checkout dev
perl init-repository –module-subset=qtbase,qtsvg,qtdeclarative,qttools,qtxmlpatterns,qtdoc,qlalr,qtrepotools,qtqa,qtlocation,qtgraphicaleffects,qtimageformats,qtxmlpatterns,qtquickcontrols,qt3d
git clone -b dev git://gitorious.org/qt/qt3d.git qt3d
for i in qtbase qtsvg qtxmlpatterns qtdeclarative qt3d; do cd $i; git checkout dev; git pull; cd ..; done
mkdir qt-5.5.0-dev
cd qt-5.5.0-dev
../configure -developer-build -opensource -confirm-license -nomake tests
make module-qt3d
the run ends with the following:
tPlatformSupport: created headers.pri file
Project ERROR: Unknown module(s) in QT: dbus
make[2]: *** [sub-platformsupport-make_first] Error 3
make[2]: Leaving directory `/home/dmcvicar/depot/qt5/qt-5.5.0-dev/qtbase/src’
make[1]: *** [sub-src-make_first] Error 2
make[1]: Leaving directory `/home/dmcvicar/depot/qt5/qt-5.5.0-dev/qtbase’
make: *** [module-qtbase] Error 2
any help that you could sugest would be appreciated.
Thanks
David
If you add -no-dbus to the configure step things should work out fine.
Thanks for the tip Simon. Compiled fine, looking forward to playing with qt3d
Hey guys!
Loving the look of Qt3D!
It’s great to see that you’re extending on the work we (on the Qt3D team) did back in the day and making it so much more flexible and current.
Hi Danny. Thank you and if you have any spare cycles, you’re most welcome to dive in. There’s still stuff from Qt3D 1 that we would like to incorporate back in. As ever, not enough hours in the day 😉
Will Qt3D support custom multiple pass rendering? Specifically, I’m interested in doing depth peeling to support transparency. I’m just curious how you would implement a custom multi-pass rendering technique in Qt3d? Will you cover that in a future article? Is there any place, on the web, I can look through the current Qt3d documentation?
Thanks
Hi Philip. Yes Qt3D will allow you to implement multi pass rendering techniques. My colleague Giuseppe is preparing a blog on this to be published shortly. In brief, attach a Material component that references a custom Effect. The Effect element allows you to specify a set of Techniques for various hardware or platforms. Each Technique contains a list of RenderPass elements each of which canset OpenGL state and reference different shaders. Giuseppe’s blog will demonstrate this with a simple shadow mapping effect. In the meantime take s look at the examples in git. Hope this answers your question.
Awesome! Thanks. I’m thinking about replacing my current static rendering engine with Qt3D. It seems like it’s fairly flexible. Just as long as the documentation is good, like your blog posts, I’m definitely interested!
What about Qt 3D on mobile devices? It’ll be great if these things work well on mobile devices.
While 3D game engines are mostly about BEHAVIOR, 3D application are about DATA. And your design is focusing on BEHAVIOR. So this module should be called Qt3DGameEngine.
The data integration parts are being worked on right now, i.e. integrating with QAbstractItemModels or other data sources. This applies to Entities, but also to generating data in buffers (e.g. for instancing) and also in texture data. Any behaviour that is implemented is up to the aspect authors. For the renderer the behaviour is controlled by the data provided to the framegraph so the two are intertwined. There is actually very very little behaviour in the frontend object tree that most users will interact with. So honestly, Qt3D is targetted at being a general purpose framework, it’s just that relating examples to games is a handy vehicle for explaining things. Once the integrations to models etc have been reworked it will be simple to make all manner of 3D data visualisation applications. If there is something you would like to see exposed, then please do let us know, ideally by filing a JIRA request.
Then maybe the game example is confusing (at least to me).
Thanks for your answer.
One question about this great qt improvement:
today Chronos annonced “VULKan” (https://www.khronos.org/vulkan) the next replacement for openGL.
will qt3D 2 be obsolete quickly? No?
No, Vulkan will not obsolete Qt3D. Firstly, Vulkan is not a replacement for OpenGL, it is a new alternative graphics API that may be used in place of OpenGL but OpenGL will continue to be developed for a long time yet. Secondly, the Qt3D architecture if flexible enough that the renderer could be adapted to use Vulkan in preference to OpenGL. The feasibility of this will become more clear as further details about Vulkan solidify but from the information that is already available, the threading model of Vulkan looks like a perfect fit for Qt3D’s task-based architecture.
Thanks for your answer!
I found QNode inherits QObject,
that likes good, but the performance of QObject’s signal/slot is not good. Have you tested the performance with many QNode s?
Hi guys, Qt3D is very nice (but sometimes crashing), but i can’t understand some simple things:
1. How to draw simple 3D line from (0,0,0) to (10,10,10) through Qt3D?
2. How to draw simple 3D circle with center in (0,0,0) and radius 10 through Qt3D?
3. How to use QPainter to draw on textures in Qt3D?