Qt, range-based for loops and structured bindings

Part 1

Ivan Čukić

17 April 2020

Range-based for loops

C++11 brought us the range-based for loop which allows easy iteration over any iterable sequence, including common containers such as std::vector, std::map, etc.

for (auto name: names) {
    // ...
}

When you write code like the above, the compiler (C++11) sees the following:

{
    auto && __range = names;
    for (auto __begin = begin(names), __end = end(names); __begin != __end; ++__begin) {
        auto&& name = *__begin;
        // ...
    }
}

It takes the begin and end iterators of the names collection, and iterates from one to the other.

Rather simple, but it is a very nice syntactical improvement to the language -- it is a significant reduction of boilerplate code and much easier on the eyes.

But, it was not designed with Qt collection classes in mind. You can check out the "Goodbye, Q_FOREACH" blog post by Marc for one example where the range-based for loop collided with the Qt design principles, most notably with the copy-on-write feature
because of which we got the qAsConst helper function.

Structured bindings

Another new feature, this one from C++17, are the structured bindings.

They add syntactic sugar for decomposing structures. For example, if we have a function that returns a pair of values (std::pair or a QPair) we can declare variables that refer to the values inside of the pair instead of having to access them through .first and .second.

auto [x, y] = mousePosition();

This can significantly improve code readability. In the previous example, the names x and y communicate the meaning much better than .first and .second would.

This is even more the case if we use a function like QMultiMap::equal_range which returns a pair of iterators delimiting the range of values that are stored under a specified key in a QMap. Seeing participants.equal_range("kdab").second would look like we are accessing the second value in the map that is stored under the "kdab" key. If we used structured bindings, we could have sane variable names:

auto [beginParticipant, endParticipant] = participants.equal_range("kdab");

Structured bindings can lead to quite succulent code when combined with the range-based for loops for iteration over STL-compliant map-like containers:

for (auto [key, value]: stdMap) {
    // ...
}

Each key-value pair in stdMap, will be stored in an invisible (unnamed) variable, and key and value will be references to its inner elements.

Sidebar: don't be fooled by decltype(key) telling you key is a normal value, for all intents and purposes, it is a reference.

`QHash`, `QMap` and STL compatibility

Sadly, the previous code snippet can not work with Qt's associative containers because the API is not compatible with that of std::map and similar containers.The problem is in the API of the iterators for QHash and QMap. Instead of returning a key-value pair when you dereference the iterator, it returns only the value. This means that the range-based for loop can only iterate over the values stored inside of the collection, it can not (by default) access the keys:

for (auto value: map) {
    // ...
}

When we use the QHash and QMap iterators directly, we can access the key with .key() and the value with .value() but this does not help us with the range-based for loop.
As we've seen, it is just a syntactic sugar which expands to an ordinary for loop that iterates over a collection automatically dereferencing (calling operator*()) the iterator in each iteration.

So, as far as the range-based for loop is concerned, the key and value member functions do not even exist.

This deficiency of the Qt's associative containers' API has been noticed, and QHash and QMap provide us with alternative, more STL-like, iterators since Qt 5.10 -- the key_value_iterator and the const_key_value_iterator. When dereferenced, these iterators return a std::pair containing both the key and the value. To quote the Qt documentation:

The QMap::key_value_iterator typedef provides an STL-style iterator for QMap and QMultiMap. QMap::key_value_iterator is essentially the same as QMap::iterator with the difference that operator*() returns a key/value pair instead of a value.

This means that we can use the structured bindings with them:

auto [key, value] = *map.keyValueBegin();

But we still cannot use this with the range-based for loop to iterate over all key-value pairs in the collection. The range-based for loop will call .begin() and .end(), and not .keyValueBegin() and .keyValueEnd().

This is easily remedied by creating a simple wrapper over QHash and QMap that will rename .keyValueBegin() and .keyValueEnd() to .begin() and .end() like so:

template <typename T>
class asKeyValueRange
{
public:
    asKeyValueRange(T &data)
        : m_data{data}
    {
    }
 
    auto begin() { return m_data.keyValueBegin(); }
 
    auto end() { return m_data.keyValueEnd(); }
 
private:
    T &m_data;
};

Instead of passing a Qt associative container directly to the range-based for loop, we just need to pass it wrapped inside of the asKeyValueRange class, and we will finally be able to use the idiomatic C++17 way of iterating over key-value pairs with QHash and QMap:

for (auto [key, value]: asKeyValueRange(map)) {
    // ...
}

It is important to note that this is a somewhat simplified implementation of the asKeyValueRange wrapper that takes the reference to the original QHash/QMap, so it might lead to a dangling reference if used improperly.

Iteration over Qt SQL results

Another part of Qt that hasn't been designed with common C++ idioms in mind is the Qt SQL module. Its API is designed as if it were a Java library and not a C++ one. Instead of having normal iterators over the results in the QSqlQuery, we need to use the .next() member function to fetch each result. The .next() member function returns a Boolean value denoting if there is another result, and the result itself is accessible through other QSqlQuery member functions.

Apart from the API, iteration over the QSqlQuery results has another difference to usual C++ iterators. The QSqlQuery does not need an end iterator to know whether there is another result available. If .next() or .isValid() return false, this means we have reached the end of the result set.

Having the iterator itself know when it has reached the end of the collection it belongs to is uncommon in STL, but it still is present. The input stream iterator (std::istream_iterator) is one such beast. We cannot know in advance where the end of an input stream is, we can just read values from the stream until the read fails. We have reached the end when the read failed. Similar to to the QSqlQuery -- we have reached the end when .next() fails -- when it returns false.

In these cases, the end iterator is only needed to fulfil the STL iterator API. It is just a dummy object (usually called a sentinel) that will trigger the validity check on the main iterator when it != end is evaluated.

C++17 brought a small change to the range-based for loop meant for this exact use-case.
When C++17 compiler sees the range-based for loop from the first code snippet we had,
it will convert it to the following code:

{
    auto && __range = names;
    auto __begin = begin(names);
    auto __end = end(names);
    for ( ; __begin != __end; ++__begin) {
        auto&& name = *__begin;
        // ...
    }
}

The difference is easy to overlook.

auto __begin = begin(names), __end = end(names)

is replaced by

auto __begin = begin(names);
auto __end = end(names);

While these two snippets look mostly the same, there is an important difference -- in the first snippet, both __begin and __end need to have the same type whereas they can be different in the second.

This allows us to create a begin function that returns a powerful iterator that knows when it has reached the end of a collection, while the end function returns a dummy value -- a sentinel.

The sentinel can be just a simple empty type:

struct QSqlResultSentinel {};
 
QSqlResultSentinel end(QSqlQuery& query)
{
    return {};
}

The iterator is where the main logic lies. For it to work with the range-based for loop,
it needs to know how to do the following operations:

- It needs to have the operator++ that moves to the next result;
- It needs to have the operator!= which takes a sentinel object and returns whether iterator is valid or not;
- It needs to have the operator* that dereferences the iterator.

The basic implementation of the operator++ and operator!= would look like this:

class QSqlResultIterator {
public:
    QSqlResultIterator(QSqlQuery& query)
        : m_query(query)
    {
        m_query.next();
    }
 
    QSqlResultIterator& operator++()
    {
        m_query.next();
        return *this;
    }
 
    bool operator!=(QSqlResultSentinel sentinel) const
    {
        Q_UNUSED(sentinel);
        return m_query.isValid();
    }
 
    // ...
 
private:
    QSqlQuery& m_query;
};
 
QSqlResultIterator begin(QSqlQuery& query)
{
    return QSqlResultIterator(query);
}

The only thing that remains is for us to decide what the dereference operator should do.

Since each record in the result can contain several values, it would be nice if we provided the operator[] to access those values. Therefore, when we dereference the QSqlResultIterator, we want to get a type that has operator[] defined on it which returns values from the current record.

We have several options on how to do this. The simplest, though not the cleanest,
is to have the dereference operator return the reference to the iterator itself, and then define the operator[] in the iterator class.

class QSqlResultIterator {
public:
    // ...
 
    QSqlResultIterator& operator*()
    {
        return *this;
    }
 
    QVariant operator[] (int index) const
    {
        return m_query.value(index);
    }
 
    QVariant operator[] (const QString& name) const
    {
        return m_query.value(name);
    }
};

If we weren't interested in having the operator[] to access the values, then we could just return the current QSqlRecord with m_query.record() from the operator*().

While this is not a complete iterator implementation (we are missing iterator type traits and similar), it is sufficient for us to be able to use it with a range-based for loop:

for (auto result: query) {
    qDebug() << result["name"] << result["surname"];
}

We'll see what we need to do in order to be able to use structured bindings with Qt SQL in the next part of this post.

About KDAB

Trusted software excellence across embedded and desktop platforms

The KDAB Group is a globally recognized provider for software consulting, development and training, specializing in embedded devices and complex cross-platform desktop applications. In addition to being leading experts in Qt, C++ and 3D technologies for over two decades, KDAB provides deep expertise across the stack, including Linux, Rust and modern UI frameworks. With 100+ employees from 20 countries and offices in Sweden, Germany, USA, France and UK, we serve clients around the world.

9 Comments

Are you planning on adding such features to upstream Qt?

I have no plans to do it in the near future because these are (or can be) simple header-only classes. Maybe for Qt 6 at some point.

Has not been a problem for me. Long live Qt!

Not a problem for me as well - more of an annoyance.

Long live Qt! :)

I love the idea of the SQL iterator. Is there a completed solution available anywhere? (Is there a part 2 to this article?)

I'd like to try using QtConcurrent::map() over the SQL results, but the QtConcurrent methods all require the begin and end of the sequence to have the same type.

Thanks for reminding me. I need to write that one soon. :)

Though, I don't think concurrent access to SQL results is going to be a good idea - the results are not in-memory for them to be random access and many SQL engines are optimized for sequential result access, not a random access. Copying them into a std::vector/QVector would be the better approach (even if you only copy 10k results at a time and concurrently map them).

Will not this work?

for (auto &[key, value]: map.toStdMap()) { // ... }

Sure, but that is very inefficient as it copies the whole QMap into a temporary std::map just to be able to iterate over the collection.

Also, changing the value from inside the for loop would change the value stored in said temporary std::map instance, not in the original QMap.

UPDATE: since Qt 6.4, QMap and QHash offer asKeyValueRange() for this purpose. This was contributed by KDAB (Giuseppe D'Angelo). Even easier :-)

Ivan Čukić

Senior Software Engineer

Author of "Functional Programming in C++" (Manning), KDAB and KDE developer, C++ trainer and consultant.

Qt, range-based for loops and structured bindings

Part 1

Range-based for loops

Structured bindings

QHash, QMap and STL compatibility

Iteration over Qt SQL results

Trusted software excellence across embedded and desktop platforms

9 Comments

Related Content

Using CMake with Qt 5

Reducing relocations with Q_STRINGTABLE

Qt Input Method - Virtual Keyboard

Join KDAB Training Day 2025 in Munich

Sign up for the KDAB Newsletter

`QHash`, `QMap` and STL compatibility