Heaptrack v1.0.0 Release First stable release of the fast Linux heap memory profiler
28.02.2017
I’m extremely happy to finally announce the first stable release of heaptrack, the FOSS heap memory profiler for C/C++ Linux applications. You can download the source tarball from the KDE mirrors: https://download.kde.org/stable/heaptrack/1.0.0/src/
Heaptrack is a fast heap memory profiler that runs on Linux. It allows you to track all heap memory allocations at run-time. Afterwards, the accompanying GUI tool can be used to find optimization opportunities in your code by analyzing the recorded profiling data. It allows you to:
- Inspect peak heap memory consumption
- Find memory leaks
- Count overall number of memory allocations and find temporary allocations
- Find small allocations with large overhead
You can use heaptrack pretty much wherever you are using Linux – it has been successfully used on 32bit and 64bit flavors of x86 and ARM platforms, both on embedded projects as well as desktop applications.
Using heaptrack
To use heaptrack to profile your native C/C++ application’s heap memory consumption, please follow these steps:
- Install heaptrack: Following this release, the Linux distributions should start packaging heaptrack soon. Otherwise, see the
README.mdaccompanying the sources. - Launch heaptrack:
heaptrack <your application name>or run-time attach to a process viaheaptrack -p <PID> - Analyze the recorded data:
heaptrack_gui heaptrack.<your-app>.<PID>.gz
Use the GUI to find hotspots in your code, then write a benchmark to get a reproducible setup to verify your performance improvements. Profile this benchmark with heaptrack again to get a baseline measurement, then improve the code and use heaptrack once more. Then you can compare this last data set against the baseline to verify that your code change does indeed result in a noticeable performance improvement.
The long road to the first release
Three years ago, I announced heaptrack’s initial proof-of-concept, which was very positively received by developers. Back then, it was already a usable tool to find issues related to the usage of heap memory in native Linux applications written in C/C++. Since then, heaptrack has already been used successfully by dozens of developers. I personally used it to optimize embedded automotive projects as well as the Qt library and various KDE libraries and applications. So, what took me so long to get a first release out?
I initially started this project out of the typical open source incentive: I used to rely on Valgrind’s massif for heap profiling, but it is very slow and the early data aggregation meant I could not add some useful features to my Massif-Visualizer, such as counting the total number of memory allocations. I decided to scratch my own itch and wrote heaptrack. Relatively quickly I had the proof-of-concept ready, which was “good enough” for many purposes and my motivation to ship a proper first release decreased.
Thankfully, KDAB decided to sponsor a couple of hours of my time to work on heaptrack in order to get a generally usable tool out to the masses. In this time, I mostly concentrated on the GUI for heaptrack. This GUI uses Qt and some KDE helper libraries to build a graphical data analysis of the data collected by heaptrack. The most notable features are:
I hope you will find these data representations as useful as I do!
Getting all of these features properly implemented took quite some time. Tracking all memory allocations of an application easily generates millions of data points. A naive analysis of such an amount of data is going to be very resource intensive – it will consume lots of memory and time on your development machines. So for this first release, I already spent some time optimizing the analysis routines to decrease the resource demands.
The road ahead
I have already some ideas planned for the coming months to further improve heaptrack and its GUI. Most notably, I intend to introduce an “annotate source” feature. This will allow me to always use function-level aggregation in the data views. Additionally, I will work on the charts which are not yet as useful as they can be: I want to add filtering and zooming on the time axis, to inspect individual memory peaks. Especially to speed up the latter, I will also investigate different data formats to improve the analysis speed. Stay tuned!
If you want to get involved with heaptrack, there are multiple ways of doing so:
If you need help with using heaptrack, or with performance tooling in general, do note that we also offer a training course on debugging and profiling Qt applications. KDAB also offers on-site workshops and mentoring to aid you in optimizing your C++ application performance. I’ll gladly teach you how to use heaptrack effectively!
About KDAB
KDAB is a consulting company offering a wide variety of expert services in Qt, C++ and 3D/OpenGL and providing training courses in:
KDAB believes that it is critical for our business to contribute to the Qt framework and C++ thinking, to keep pushing these technologies forward to ensure they remain competitive.
Wow this is quite exciting indeed. I’ve used Valgrind memcheck before, but this looks quite promising. Will give it a go now with KStars, especially as we need to reduce memory footprint on limited resources devices like the Raspberry PI.
The GUI tools look great!
As for the low level part, it looks like you’ve been reinventing the wheel: gperf-tools and jemalloc have been doing this for many years. Why didn’t you build the visualization tools on top of the preexisting software?
Hey Erwan,
the simple answer is that I was not aware of these solutions. See also my initial announcement at http://milianw.de/blog/heaptrack-a-heap-memory-profiler-for-linux and search for “Reinventing the wheel”. Search also for the jemalloc comment and my response to that. Also note that heaptrack should work fine with an application that uses jemalloc internally. And contrary to jemalloc, you can also runtime-attach heaptrack, it’s agnostic to the malloc implementation.
Do you see any advantages of using gperf-tools or jemalloc over heaptrack for memory profiling? I’d be very interested in learning more about those tools. If they are better in any aspect, It would be cool if someone wrote either a conversion tool for gperf-tools and jemalloc output, or separate parsers for heaptrack_gui.
Cheers
Hi Millan,
Getting this backtrace on coredump analysis:
…crash dump removed…
After this application crashes and no further traces …
Is there any know issue with aligned_alloc …please suggest.
Hello SKumar,
please use the heaptrack bug tracker to report bugs instead of comments on a blog post:
https://bugs.kde.org/enter_bug.cgi?product=Heaptrack
Make sure to document a reproducible way to trigger this issue. I’m not aware of any issue with aligned_alloc.
Hi Milan,
You can delete the previous comments . Have one question, is heaptrack code Locale aware ??.
Hey Skumar,
I won’t delete the comments, as others may have similar questions in the future.
Regarding your new question: Can you specify what exactly you are wondering about? Do you ask whether the GUI is translated into other languages? It can be, the question is whether it was translated into your target language. The tracker itself does not care about locales since that is not relevant to tracking memory usage.
I like that Heaptrack does not slow down the running process noticeably. The GUI is certainly very helpful. At our company we have our own LD_PRELOAD based memory leak detector which does call stack merging on the go (and reports leaks in regular intervals). That proved not very suitable for detecting leaks in web service code, because the HTTP requests took too long to handle and caused timeouts in client applications – so I desperately needed something that does not kill performance.
Also the function name resolution is really good (I think our solution is based on dynamic symbol table, Heaptrack probably uses debug info instead).
While on one hand doing the analysis after the monitoring is stopped is a blessing in some situations when you do not want to slow down your processes, on the other hand it prevents monitoring processes over the course of several hours. The log files get too large and the UI app then takes very long to analyse them. I am not critisizing this, I understand the trade-off. It’s just we have some processes where leaks tend to manifest after a longer while…
Certainly Heaptrack helped us considerably this week to track down a nasty leak causing a critical process to eat through 14GB of memory in a couple of hours (exception in frequently called C++ constructor caused class destructor to not be called – good example why we should write code in RAII style by default). This was on a customer installation under RHEL 7.3 which fortunately contains Qt5+KF5-devel packages in the repo and I was able to compile Heaptrack according to the list of dependencies on your Github page without friction.
I tried 5 different leak detectors before and when I was becoming desperate I stumbled on Heaptrack.
All in all, big thank you for this great tool!
P.S. Is there an option to filter out only leaky call stacks?
Is there a mailing list, discussion forum, IRC channel, or any other place than this blog post to ask questions?
I’m wondering about the feasibility of porting the runtime sample/instrumenter (i.e. heaptrack, not heaptrack_gui etc.) to macOS. It may even compile that component out of the box if the required prereqs are installed. Obviously the heavy KDE library dependencies in the GUI front end would make it hard/infeasible to port to macOS, but the main instrumenter/sampler looks like it may be feasible. (This ad-hoc analysis is based solely on my reading through the dependencies on the Github readme page.)
Thoughts?
P.S. I also posted this on the bug tracker because there wasn’t an obvious link to a heaptrack website. I found this through google.
Hey Zaak,
you can use heaptrack@kde.org or the bug tracker. For now, let’s continue this discussion in the bug report you opened: https://bugs.kde.org/show_bug.cgi?id=390485
Thanks
I’ve been using this to track memory leaks in our app. It’s impressive how easily this information is displayed.
Our code was mostly correct, and the main leaks were because of bugs in std::sort which showed up under clang and gcc.