Efficient Custom Shapes in QtQuick : Shaders

A long time ago, I wrote a post about creating custom shapes in Qt Quick, via the scene-graph APIs. That post covered defining suitable geometry to draw a part of a circle, known also as a 'sector', efficiently, since such sectors occur commonly in instrument and vehicle interfaces. I started writing the second part, about implementing the material and shaders to make the complete functioning item.

Then, somehow, life and customers intervened, and I forgot all about publishing the second part, and now it's an embarrassingly long time later. Not as bad as the wait for A Dance With Dragons or Duke Nukem forever, but still, embarrassing. This is all my way of saying, here's a post about writing shaders and QtQuick scene graph materials, which is interesting but also more than three years late, and that WordPress apparently doesn't remind you that you have drafts sitting around.

Making our triangles larger also means we need fewer. The extreme case of this is just to make two triangles (a square) covering the whole circular area of the sector. But that's wasteful, especially as our fragment shader grows more complex, which it's about to do. Also, on a tiled renderer (which means almost every mobile GPU in practice), if we can reduce how many tiles the geometry intersects, we get some additional performance wins. So, we want to have enough triangles that our geometry approximates our real shape, but not so many that we're making the scene-graph or vertex processing elements do unnecessary work.

All about the pixels (baby)

The challenge facing us is: we need to define what color the pixels of our geometry end up with, and that's decided for each pixel by a shader program (written in GLSL for OpenGL, SPIR-V for Vulkan, and similar formats for other QtQuick backends). In the QtQuick scene graph system, shaders are interfaced to C++ via a material (QSGMaterial) subclass.

The main task of our material (and shader), then, is to answer two questions about a pixel:

• Is it outside our shape, part of the border, or somewhere inside?
• What colour should it be? (based on its position in the sector and the various colors defined in properties from QML)

The shader is a program within our program, executed by the GPU, which answers the above two questions for each and every pixel in our triangles, every time the scene is rendered (which is usually sixty times per second in Qt Quick). Fortunately, GPUs are fast and perform this work in parallel.

With the geometry, we had to define an array of X and Y values. But for the material side, we now need to get into the details of shading languages. We're going to focus on OpenGL, since it's the most common. But, fortunately, all shading languages are very similar. The OpenGL shading language is called GLSL. If you’ve ever experimented with the ShaderEffect item in QtQuick, you’ll have a good idea what is required, but there’s some book-keeping needed. Most importantly, we have to define our shader source code as some GLSL code. In my item, I do this as text strings inside the C++ source (you could also put them in a resource and load them via QFile). But, apart from the code, we also need to define the control values we will pass into our shader from C++. These values are called uniform values in OpenGL, and we need to manage them in a particular way for performance reasons.

The basic principle is to put all our control values into a structure, and ensure the scene-graph can compare two examples of our structure for equality. This means, if we have several sectors on the screen using the same colors and border, the scene-graph can identify that they use identical uniform values. Why is this important? Well, the most beneficial optimization strategy that the renderer performs is to group geometry that can be drawn at the same time into batches, to reduce the number of drawing calls and state changes. To allow multiple geometries to be batched, they must meet certain criteria. Having the material state be consistent is one of them. That’s why we follow some standard patterns, inherit from QSGSimpleMaterialShader, and use the macro QSG_DECLARE_SIMPLE_COMPARABLE_SHADER so that our shader can be compared with other instances and hopefully batched together when rendering.

You could skip using the comparable stuff. But if you want to have ten or fifty sectors on screen at once, making them batch-able becomes significant. (Maybe you want to re-create the engine display panel for the 'Classic' series of 737s.)

The job of our material class, then, is to collect all the parameter data, be able to compare it in a robust way (often with some tolerances, for floating point values: qFuzzyCompare helps), and to map the parameters in C++ to uniforms on our shader. It also has to load the shader source from a file, resource to text string. Fortunately, the QSGMaterial subclasses have helpers to make all of this relatively easy, and handle some other complexities, as well (such as variations between desktop and embedded GLSL shaders).

Writing the fragment shader

To answer our 'is it inside our shape?' and 'what colour should it be?' questions for each pixel, we need to convert back into polar coordinates inside our pixel. We need to know our angular position to calculate the gradient color smoothly. And we need the radius to work out, if we're inside or outside the shape, or on the border. That’s actually why we passed some additional data along with each vertex, to give us an easy numerical range to work with in our shader. We use a coordinate system where the center of the sector is always (0,0). To convert back to polar form, we use the atan2() function in GLSL, followed by a length() function to compute the radius. (For those of you familiar with OpenGL, we're using the texture coordinates U and V to encode this simplified position alongside the real X and Y values. We could compute the values from X and Y in the shader with some additional uniforms. But with the limited number of triangles used, the overhead of two additional attributes per vertex is minimal.)

Then, we can check our radius against the inner, outer, and border values (passed in via uniforms), to see what we need to do. If we’re outside the outer radius, we can discard the pixel (analogous to an early-return in a C++ function). Discarding pixels can save some effort for the renderer, if done early in the fragment shader.

If we’re inside the border, we can use the border color. Otherwise, we need to compute an interior color. The built-in GLSL function mix() helps us to smoothly interpolate (blend) between two colors. To implement anti-aliasing, we also check if we’re near the transition between the interior and the border, or between the border and the outside. In those cases, we modify the pixel’s transparency, or mix the color with the border, to give a smooth transition.

1. Outside the shape : discarded
2. On the transition between the the border and the main fill - borderOpacity will be for example 0.5
3. Within the fill: color in interpolated between the four inner colors to give final pixel RGB. In the code below, tNorm will be approximately 0.5 (half way between inner and outer radius)
4. Fully within the border: borderOpacity is 1.0

Let's express that in some GLSL:

Our GLSL fragment shader has main(), just like a regular C++ program, but it runs millions of times per second (in parallel, of course). We start by doing our atan call, and then we proceed to work out our color value based on that.

You may notice the code always applies a border and the possible gradient options. Why not use if-s to avoid this extra work? Well, shader programs really hate branching, whereas multiplying values by zero is optimized explicitly. Thus, it's usually (but not always) advisable to write straight-line code without branching, if possible. This especially applies on lower powered embedded GPUs. Of course, if we had many sectors where a gradient was not required, we could create a different shader program without that feature and select one program or the other in our C++ code. This would keep the GLSL code friendly to our GPU, while allowing us to squeeze as much performance as possible out of the hardware. And our UX designers would never need to care.

Final result

Putting this all together, we define our QSGMaterial subclass, our state struct, and code, to look up the uniform values which pass information from C++ into GLSL. There are quite a few small points to get correct. But, in the end, we have a beautifully rendered sector on the screen, and an API we can give to our UX designers, confident that even if they go crazy with adding sectors, animating them, making them pulse, adding big and small ones, the performance should be excellent.

The good news is that, having done this once, doing it for other shapes is more of the same. On the geometry side, figure out some triangles which cover the pixels (fragments) of your shape. Inside your shader, use the vertex information, any uniforms you need, and possibly some other vertex information to compute if you're within your shape, and what color to give it. It's possible to do much more complex techniques, of course, creating procedural patterns following the many examples on ShaderToy, or defining vertices, dynamically, based on a real-time input (for example, an audio waveform).