How to build Qt with the Address Sanitizer on Windows
10.12.2020
Some time ago, I wrote about how to build C++ projects with ASAN on Windows. Now, if you happen to deal with Qt projects you may want to take this one step further and sanitize Qt itself.
Why bother with a sanitized Qt build?
Let’s have a closer look on why having a sanitized Qt build around is a good idea.
If you’re already patching Qt for your own needs, sanitizing Qt is the only way to check for bugs on your code. This is the reason why we use sanitizers in the first place.
Otherwise, you may argue that Qt is “good” or “safe” or “bug free” enough. This, in principle, is not a safe assumption. Choosing to take your chances may still leave some of your bugs on the table. Let’s see an example:
#include <QApplication>
#include <QByteArray>
#include <QDebug>
#include <QTimer>
int main(int argc, char* argv[])
{
QApplication app(argc, argv);
QTimer t;
t.setInterval(0);
t.setSingleShot(true);
{
QByteArray ba = QByteArrayLiteral("Test byte array literal");
t.connect(&t, &QTimer::timeout, &t, [d=ba.data()](){
qDebug() << d;
});
}
t.start();
return app.exec();
}
When we build this code with the address sanitizer and run it, we get this output:
Test byte array literal
Great, all good! Or is it? Let’s try to run the same code with a sanitized Qt. This is the output:
=================================================================
==15844==ERROR: AddressSanitizer: heap-use-after-free on address 0x12643130a628 at pc 0x7ff81ca3e54a bp 0x00b1e08f86e0 sp 0x00b1e08f8728
READ of size 24 at 0x12643130a628 thread T0
#0 0x7ff81ca3e572 in _asan_wrap_strlen+0x1b2 (C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\Llvm\x64\lib\clang\10.0.0\lib\windows\clang_rt.asan_dynamic-x86_64.dll+0x18002e572)
#1 0x7ff813417f6e in QString::fromUtf8 C:\qt5\qtbase\src\corelib\tools\qstring.h:572
#2 0x7ff7780318aa in QtPrivate::QFunctorSlotObject<`lambda at C:\test-build-with-sanitized-qt\main.cpp:15:45',0,QtPrivate::List<>,void>::impl C:\qt-build-sanitized\include\QtCore\qobjectdefs_impl.h:439
#3 0x7ff813cd4fc3 in QMetaObject::activate C:\qt5\qtbase\src\corelib\kernel\qobject.cpp:3781
#4 0x7ff813cf1671 in QTimer::timerEvent C:\qt5\qtbase\src\corelib\kernel\qtimer.cpp:255
#5 0x7ff813cbfcf5 in QObject::event C:\qt5\qtbase\src\corelib\kernel\qobject.cpp:1247
#6 0x7ff8158ac7a7 in QApplicationPrivate::notify_helper C:\qt5\qtbase\src\widgets\kernel\qapplication.cpp:3737
#7 0x7ff8158b2470 in QApplication::notify C:\qt5\qtbase\src\widgets\kernel\qapplication.cpp:3598
#8 0x7ff813c19f8d in QCoreApplication::notifyInternal2 C:\qt5\qtbase\src\corelib\kernel\qcoreapplication.cpp:1084
#9 0x7ff813d7ef0d in QEventDispatcherWin32::event C:\qt5\qtbase\src\corelib\kernel\qeventdispatcher_win.cpp:1064
#10 0x7ff8158ac7a7 in QApplicationPrivate::notify_helper C:\qt5\qtbase\src\widgets\kernel\qapplication.cpp:3737
#11 0x7ff8158b2470 in QApplication::notify C:\qt5\qtbase\src\widgets\kernel\qapplication.cpp:3598
#12 0x7ff813c19f8d in QCoreApplication::notifyInternal2 C:\qt5\qtbase\src\corelib\kernel\qcoreapplication.cpp:1084
#13 0x7ff813c1f045 in QCoreApplicationPrivate::sendPostedEvents C:\qt5\qtbase\src\corelib\kernel\qcoreapplication.cpp:1821
#14 0x7ff84512b0b5 in QWindowsGuiEventDispatcher::sendPostedEvents C:\qt5\qtbase\src\platformsupport\eventdispatchers\qwindowsguieventdispatcher.cpp:81
#15 0x7ff813d6fdee in qt_internal_proc C:\qt5\qtbase\src\corelib\kernel\qeventdispatcher_win.cpp:245
#16 0x7ff88793e857 in CallWindowProcW+0x3f7 (C:\WINDOWS\System32\USER32.dll+0x18000e857)
#17 0x7ff88793e298 in DispatchMessageW+0x258 (C:\WINDOWS\System32\USER32.dll+0x18000e298)
#18 0x7ff813d757d5 in QEventDispatcherWin32::processEvents C:\qt5\qtbase\src\corelib\kernel\qeventdispatcher_win.cpp:639
#19 0x7ff84512b046 in QWindowsGuiEventDispatcher::processEvents C:\qt5\qtbase\src\platformsupport\eventdispatchers\qwindowsguieventdispatcher.cpp:74
#20 0x7ff813c0aaaf in QEventLoop::exec C:\qt5\qtbase\src\corelib\kernel\qeventloop.cpp:225
#21 0x7ff813c1bf60 in QCoreApplication::exec C:\qt5\qtbase\src\corelib\kernel\qcoreapplication.cpp:1385
#22 0x7ff778031443 in main C:\test-build-with-sanitized-qt\main.cpp:21
#23 0x7ff7780326b3 in __scrt_common_main_seh D:\agent\_work\9\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
#24 0x7ff888866fd3 in BaseThreadInitThunk+0x13 (C:\WINDOWS\System32\KERNEL32.DLL+0x180016fd3)
#25 0x7ff8890fcec0 in RtlUserThreadStart+0x20 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18004cec0)
0x12643130a628 is located 24 bytes inside of 48-byte region [0x12643130a610,0x12643130a640)
freed by thread T0 here:
#0 0x7ff81ca45094 in _asan_memmove+0x344 (C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\Llvm\x64\lib\clang\10.0.0\lib\windows\clang_rt.asan_dynamic-x86_64.dll+0x180035094)
#1 0x7ff77803141f in main C:\test-build-with-sanitized-qt\main.cpp:18
#2 0x7ff7780326b3 in __scrt_common_main_seh D:\agent\_work\9\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
#3 0x7ff888866fd3 in BaseThreadInitThunk+0x13 (C:\WINDOWS\System32\KERNEL32.DLL+0x180016fd3)
#4 0x7ff8890fcec0 in RtlUserThreadStart+0x20 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18004cec0)
previously allocated by thread T0 here:
#0 0x7ff81ca451a4 in _asan_memmove+0x454 (C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\Llvm\x64\lib\clang\10.0.0\lib\windows\clang_rt.asan_dynamic-x86_64.dll+0x1800351a4)
#1 0x7ff813503701 in QArrayData::allocate C:\qt5\qtbase\src\corelib\tools\qarraydata.cpp:118
#2 0x7ff81350bcaf in QByteArray::reallocData C:\qt5\qtbase\src\corelib\tools\qbytearray.cpp:1905
#3 0x7ff81341493b in QByteArray::data C:\qt5\qtbase\src\corelib\tools\qbytearray.h:569
#4 0x7ff778031313 in main C:\test-build-with-sanitized-qt\main.cpp:15
#5 0x7ff7780326b3 in __scrt_common_main_seh D:\agent\_work\9\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
#6 0x7ff888866fd3 in BaseThreadInitThunk+0x13 (C:\WINDOWS\System32\KERNEL32.DLL+0x180016fd3)
#7 0x7ff8890fcec0 in RtlUserThreadStart+0x20 (C:\WINDOWS\SYSTEM32\ntdll.dll+0x18004cec0)
SUMMARY: AddressSanitizer: heap-use-after-free (C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\VC\Tools\Llvm\x64\lib\clang\10.0.0\lib\windows\clang_rt.asan_dynamic-x86_64.dll+0x18002e572) in _asan_wrap_strlen+0x1b2
Shadow bytes around the buggy address:
0x04a8b7561470: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
0x04a8b7561480: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
0x04a8b7561490: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
0x04a8b75614a0: fa fa 00 00 00 00 00 fa fa fa 00 00 00 00 01 fa
0x04a8b75614b0: fa fa 00 00 00 00 00 fa fa fa 00 00 00 00 00 00
=>0x04a8b75614c0: fa fa fd fd fd[fd]fd fd fa fa 00 00 00 00 00 00
0x04a8b75614d0: fa fa 00 00 00 00 00 00 fa fa 00 00 00 00 00 00
0x04a8b75614e0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
0x04a8b75614f0: fa fa fd fd fd fd fd fa fa fa fd fd fd fd fd fa
0x04a8b7561500: fa fa 00 00 00 00 00 fa fa fa 00 00 00 00 00 fa
0x04a8b7561510: fa fa 00 00 00 00 00 04 fa fa 00 00 00 00 00 04
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==15844==ABORTING
This is because, in the call to connect on line 15, we are copying a pointer to the internal buffer of a QByteArray, which will be already gone when the timer is started. Note that this is not inherently a Qt bug. Additionally, the documentation warns us that the lifetime of that pointer is bound to the one of the byte array. Since all of the allocations are happening inside Qt without sanitizing Qt itself, this bug will go undetected.
Building Qt with the Address Sanitizer
To generate a sanitized Qt build, we are going to use clang. Clang has been providing the Address Sanitizer on Windows for a while. This means we will only be able to create a release build for Qt, as clang on windows doesn’t support debug builds with the Address Sanitizer. If you have a recent version of Visual Studio, you can choose to do the same with the cl compiler in a similar way.
Qt comes with bundled mkspecs, to build with either clang or clang-cl. We will be creating our own sanitized clang-cl mkspec based on the original clang-cl one. To do so, duplicate the folder qt5/qtbase/mkspecs/win32-clang-msvc to qt5/qtbase/mkspecs/win32-clang-msvc-with-address-sanitizer. Next, replace the content of qmake.conf with the following:
include(../win32-clang-msvc/qmake.conf) # Generate debug information CONFIG += force_debug_info # Force release build as debug builds are not supported CONFIG += release QMAKE_CFLAGS_RELEASE_WITH_DEBUGINFO += -fsanitize=address QMAKE_CXXFLAGS_RELEASE_WITH_DEBUGINFO += -fsanitize=address # # Add the path to the clang ASAN runtime and link against QMAKE_LFLAGS_RELEASE_WITH_DEBUGINFO += /LIBPATH:\"C:/Program Files (x86)/Microsoft Visual Studio/2019/Professional/VC/Tools/Llvm/x64/lib/clang/10.0.0/lib/windows\" QMAKE_LFLAGS_RELEASE_WITH_DEBUGINFO += clang_rt.asan_dynamic-x86_64.lib /wholearchive:clang_rt.asan_dynamic_runtime_thunk-x86_64.lib
Now you can start building by specifying the mkspec with -platform win32-clang-msvc-with-address-sanitizer. Also, make sure you pass -no-pch to the configure script. Otherwise, the build will fail.
Note that, since rcc and moc are built with ASAN as well. They may (and will) generate some ASAN warnings at build time. So make sure you have ASAN_OPTIONS=halt_on_error=0 in your build environment. Do the same for building all your projects using this sanitized Qt version.
Some final touches
If you try launching an application built against this sanitized Qt build, it will raise an exception at startup. The error will look like this:
Unhandled exception at 0x00007FFA44FF2807 (Qt5Core.dll) in qtplain.exe: 0xC0000005: Access violation writing location 0x00000000756E6587.
What is this about? Let’s have a look at the disassembled code around the affected area:
0x00007FFA2DD827F4 xor eax,eax 0x00007FFA2DD827F6 xor ecx,ecx 0x00007FFA2DD827F8 cpuid 0x00007FFA2DD827FA mov esi,eax 0x00007FFA2DD827FC mov rdi,qword ptr [__asan_shadow_memory_dynamic_address (07FFA2EDC6E60h)] 0x00007FFA2DD82803 lea r12,[rbx+60h] 0x00007FFA2DD82807 mov qword ptr [rbx+40h],41B58AB3h <-- EIP
What’s going on is that the cpuid instruction, when invoked with eax=0 (see xor eax, eax on address 00007FFA2DD827F4), will write a 12 bytes string spread on the three registries ebx, edx, ecx – in that order – identifying the manufacturer of the CPU.
In my case, for instance, cpuid returns 'GenuineIntel', which translates, adjusting for endianness, to the following values for rbx, rdx, rdx:
rbx: 0x00000000756E6547 . . . . u n e G rdx: 0x0000000049656E69 . . . . I e n i rcx: 0x000000006C64746E . . . . l e t n
A few instructions later, where the application breaks, the instruction on 0x00007FFA2DD82807 is trying to write on ebx + 0x40, whose value is exactly 0x00000000756E6587 (the address mentioned in the error message).
This means the registry wasn’t backed up and restored by the compiler around the cpuid instruction. This is a known clang bug that happens when a heap allocation occurs right after a cpuid instruction with ASAN enabled. Or, more in general, it is a rare bug that results in ebx being mismanaged by the compiler (see this and this).
So what can we do about it?
For lack of better options, a quick workaround for this is to replace all the calls to cpuid (see below) with their return values. To do so, just run all the calls to __cpuid on a small program. Then, copy the values on the Qt source code. The following is a list of all the occurrences:
- https://code.woboq.org/qt5/qtbase/src/corelib/tools/qsimd.cpp.html#227
- https://code.woboq.org/qt5/qtbase/src/corelib/tools/qsimd.cpp.html#249
- https://code.woboq.org/qt5/qtbase/src/corelib/tools/qsimd.cpp.html#283
Conclusion
The Address Sanitizer is a helpful tool for tracking and solving memory bugs. However, limiting ourselves to sanitize only our own code is not enough. Libraries can prevent bad API usage only up to a certain point. There are still cases where damage can be done undetected. Sanitizing consumed libraries will greatly increase the surface for automatic bug detection. Qt’s open source nature allows us to leverage this opportunity.
If you’d like to learn more about the tooling available for C++ developers working with Windows, you might be interested in our Debugging and Profiling C++ Applications on Windows training.
If you like this article and want to read similar material, consider subscribing via our RSS feed.
Subscribe to KDAB TV for similar informative short video content.
KDAB provides market leading software consulting and development services and training in Qt, C++ and 3D/OpenGL. Contact us.
Great post Allessandro !
Any hint where to look at if the clang issue with the CPUID is fixed ?
Also I wonder why the precompiled headers are not working under Windows, is this a windows thingy ?
Normally clang is very good at precompiled headers.
Regards, Leo