Scalable UIs In QML
14.05.2014
A User Experience is not defined to exist in a single form factor any more.
It is expected to be able to flow from one device to another, seemingly adapting itself to each interaction paradigm, and yet retain branding, mood and overall look and feel across multiple platforms.
On the other hand, it’s expected that Designers and Developers create User Interface solutions that feel native across platforms and device types. At the same time a single platform can have a huge amount of variable interface possibilities that need to be taken into account by the UX/UI…
These are just a few problems one needs to take into account when designing apps for cross-platform deployments. At first sight it might seem like an impossible task to tackle. In the next series of Posts I will share my experience in trying to square the circle, on what I have learned and how it can be implemented in QML…
Scalability.
Unfortunately there is no magic “design-once, deploy-everywhere” that can turn a windows 8 desktop application into an iOS touch app and also an embedded device UI. It’s not as simple as scaling an app from a 4 inch screen into a 27 inch one by stating the number of pixels in the x and y dimensions (these days it can be the same). Users have different interaction expectations on different devices.
Scaling the same application on the same platform
Let’s start with what could be considered at first sight the easiest task… making one application for say a mobile platform. For the sake of example, let’s use Android…
So in theory and all things being perfect you would design/implement an UI and you would expect it to work across the entire platform, on all devices, and in a way it does. You can stick with using only simple QtQuick.Controls elements that look native and scale nicely already; but there is a problem here: the number of elements available is limited, and the designer in you wants to mimic a certain intellectual concept into a visual metaphor that helps the user to “learn” the intended UI. For that you need to design your special ButtonFoo or SpinSliderMagic etc, etc…
This is where QML is perfect, it was designed exactly for this, an easy way to declare such custom components that can mimic so much better the intended user interaction paradigm.
You can think of it like this: you have to paint a landscape and you have to do it with just Lego, there will be a moment when you just need a “triangular” non-existing piece to make the whole thing perfect. Because it’s the “triangular” thing that you can share across multiple platforms and unites the entire application across the diverse platforms, or it’s the triangular piece that “just works” for the content/interaction space ratio, whatever the problems you may have, you will need many “triangles”. QML allows you to do just that.
The Problem
In the desktop world, most screens have, up until recently, rather similar ppi’s (pixels per inch) and the differences were many times mitigated by the distance to the user’s eyes. Larger screens had smaller ppi’s but were further away from the user’s eyes. This meant that the fundamental metric to design anything was the PIXEL – the smallest almost indivisible element that you could use. Because of this fact, fonts that apparently used different metrics were in reality using pixels – the logical value for ppi was hard-coded in the several desktop OSs. To avoid that, the font size – widget size ratio would change from screen A to screen B. All of this was only possible because the range in ppi’s was rather limited…
Now let’s take a look at ppi’s on Android devices. It is totally possible to enter a store and look at 2 devices that look similar with radically different ppi’s from around the 90s (where you can clearly discern individual pixels), to the mind-blowing 500s where it’s much more than the human eye can tell apart.
Yet one expects that a given app looks similar enough on both devices.
Using the same approach to element creation that we have on desktop simply does not work here.
If one uses a simple QML file that tries to scale a button-like element to the 2 screens above using a BorderImage element and simply states the distances in pixels, it will result in severely different looking apps to the user.
There are multiple ways to solve this issue and it is my plan to talk about them in the following posts, as, in my opinion, different methods can be used depending on the specific problem. However, to do that we need to expose and understand those very same problems, both from a conceptual as well as from a rendering pixel point of view.
The Usual method
The most common way to solve this issue is via the shipment of several different source images that are targeted at ppi ranges combined with ppi range metrics.
How one can implement such solutions, in current QML? What are the limitations? Where does it work the best? All of this will be the subject of the next post.
So stay tuned as this might be a fairly lengthy subject…
> to the mind-blowing 500′s where it’s much more than human eye can tell apart
That’s not true, that’s just marketing (as in «retina»). A human eye (at the distances on which phones and tablets are generally used) can easily distinguish single pixel offsets at such ppi levels. That’s not some «enormous» ppi levels, they are just better than 90 ppi.
Testcase: http://wstaw.org/m/2014/05/14/plasma-windowedK16160.png (you should view that on 1:1 virtual:physical pixel mapping and without antialiasing).
> it’s much more than human eye can tell apart
If this were the case, then pure black fonts (without both grayscale or subpixel antialiasing) on pure white background would look good.
@Nikita thanks for the comment.
http://i.imgur.com/LvwITsP.png
if you try that very same image on an Oppo Find 7 you will have a real hard time seeing the AA issues.
The gray scale or the subpixel antialiasing on fonts will be much less needed on this sort of screens if needed at all (subpixel antialiasing was allays an abomination IMO so 😉 )
Don’t get me wrong huge fan of high dpi screens just that there is a point were it becomes a rather silly “mine is bigger than yours game…”
“In the desktop world, most screens have, up until recently, rather similar ppi’s (pixels per inch) and the differences were many times mitigated by the distance to the user eyes, larger screens had smaller ppi’s but were further away from the user eyes. ”
On the web, the problem is solved by creating “virtual pixels”: a nexus 4 for example might have a resolution of “1280 x 768”, the webbrowser however uses a devicePixelRatio of 2. One pixel is rendered as 4 pixels. This gives the viewport a virtual resolution of 640 x 384.
This virtual resolution makes text and images in the browser roughly the same size as on classic desktop displays (from the 90s). However, the algorithm is very rough – do not expect elements to have the exact same size on an iphone and on an android phone.
I’m looking forward to your next article and how the problem is solved with QML.
@Daniel Beck thank you, did not knew that 🙂
in a way the 1:2:4… ratios will be what we will be using in the next method. Its far from a perfect solution…
My plan with this posts is not to come up with the perfect solution, but rather to expose most of the problems and some solutions to those problems. So developers/designers have a better arsenal to mange them…
yes please nuno, I’m looking forward to this!
Hi Nuno, I am working on a desktop project which uses Qt 4.8 and want to make it scalable with different OS dpi settings across platforms (windows, mac, linux). Additionally, user can also custom the scale of interface.
What’s the good recommendation? Should I switch to Qt 5 which has devicepixelratio? Personally I never tried Qt5 but I don’t see I can modify the desktop’s devicepixelratio therefore no goal of getting custom scaling?
Currently, I am trying to fix the font by using adjusted point size instead of pixel size. I will do some fix in layout’s sizehint as well. Is this a good solution since this enables custom scaling.
Looking forward to your next post.
Wen using font as the base metric for your UI scalability, via the indirect method of using the pint size you have to remember that desktops made the dpi the font uses independent from the real dpi your screen has… also that the very same dpi will change from one OS to the other….
Regarding the Qt version and wen talking about scalability of widget based apps, I don’t think much as changed. But I’m not completely sure about your specific problem or implementation, so I might be wrong. It’s my plan in this series of posts to explain a bit the issues of scaling stuf on a qml implementation POV, but much of it can be applied to desktop widget based UI design.
What’s interesting, is that this is far from a new problem if you think about it. We have had printing for ages, and printers have had high resolutions for a long time time now.
I recently implemented a report engine for our application, and had to deal with the resolution issue too.
Many solutions seem to build on providing resources in multiple resolutions and selecting the right one by use of magic, and introducing yet another coordinate system that is somehow implicitly scaled. I must say that I really don’t like this approach one bit. I’d much rather see the use of real sizes in a proper way. One issue I noted that it seems the dpi for fonts is fixed in Qt, and to the wrong value at that! If I print on a 1200dpi output device, I’d like to get the correct font size if I set it in points. A point is 1/72th of an inch, after all. I ended up adding my own pointSize property on a Text element that uses the _real_ dpi, instead of the imaginary one Qt uses.
For me, the best way to deal with this would be to make it easy to specify the size of an element (font, line thickness, whatever) in a natural unit: points, inches, mm, whatever. Magic solutions always break down sooner or later.