KDAB contributions to Qt 5.7
Last week I visited a new customer who is making medical and industrial devices which have one thing in common: image and video capturing, and letting the user save these files onto a USB key. These devices run embedded Linux and the application is done in Qt (and gstreamer for the video capture). The new requirement, and the reason I was called in, is to be able to save these files over the network, to a samba or a WebDav share. An abstraction framework for copying files over the network to multiple protocols is something I know very well: the KDE framework called KIO implements exactly that, and I happen to be its maintainer. And after the last 4 years of effort in making this much more modular, it is now possible to use KIO with very few dependencies, as we found out. In this two days on-site visit, I first explained many things about KIO (the API, the framework implementation, the available protocols, etc.), and then we started integrating it into their custom embedded Linux distribution.
The KIO framework consists of 3 libraries (KIOCore, KIOWidgets, KIOFileWidgets) and a number of helper programs (daemons). However for this use case, only KIOCore is needed. Some user interface is needed: error and progress notification, the handling of “file already exists, overwrite/rename/skip?”, a dialog for entering user/password for the remote share, a way to display SSL certificate details and approval/refusal… But of course for all this, the KDE user interface (implemented by KIOWidgets) isn’t desired, rather another implementation that integrates better with the existing embedded GUI. The good thing is that the proper core/gui separation in KIO allows for exactly this: using only KIOCore and implementing some C++ interfaces to provide a customized user interface.
After adding an option to the KIO buildsystem for building only KIOCore, we were able to reduce greatly the number of required dependencies. Only the following are needed: extra-cmake-module (buildsystem):
- KCoreAddons (Job API)
- KConfig (configuration framework)
- KDBusAddons (daemon modules)
- KI18n (translation framework)
- KArchive (for compressed HTTP transfers).
A simple patch commenting out some unneeded protocols for this particular use case allowed us to avoid some additional dependencies (e.g. Solid, for the “trash” implementation).
The dependencies listed above are all “tier 1” frameworks, i.e. they only depend on Qt. So we were able to quickly integrate them into the hand-made Linux distribution, and we have KIOCore ready to use. After copying over the unit tests and setting up a few paths, we had all the tests running without errors.
The final hurdle was the process that handles SSL certificate policies, which was implemented as a module for KDED, a generic process that lives in another framework, bringing in more dependencies. I instead made it standalone, which solved the immediate problem. I’m now working on a better solution – a KIO daemon with multiple modules, for the various service that the KIO framework needs in a separate process (cookie handling, etc.)
On the topic of multiple processes, the customer appreciated greatly the fact that KIO protocol implementations (called “kioslaves”) are running in a separate process, because this allows them to look into running them as a different user, to give them limited permissions for security reasons. That user can have a read only access to the saved files (and to nothing else), since the use case is only upload operations.
All this in order to reduce the number of dependencies that KIO has, making it easier than ever to use KIO in any Qt application, so as to benefit from easy to use, asynchronous, well-tested network transparency with more than 90 different protocols available. I’m sure there are other applications with the same need 🙂
Would it be worth looking at splitting the frameworks up more?
The smaller the frameworks, with as few dependencies as possible the more code reuse we would see in other Qt projects
Great post!
I am wondering about the performance though. And i’m not talking about UDSEntry 😉
You surely know that there are quite some bug reports for KIO slaves addressing performance issues in at least smb and ftp. More interesting, it seems to be different per machine. I actually have this same issue. My SMB performance through KIO is… not stellar. The same through a cifs mount is as fast as my network can handle it. That makes me wonder if you found any issues regarding slave performance? Or is it working just fine in the environment where you installed it?
In these only two days to simplify dependencies and integrate KIO with the existing “distribution”, there was no time to test performance. I’m not sure it’s a major concern in this case anyway.
How does this compare to C++ File System TS ?
Is (will) it be able to handle this scenario?
Thanks
is there any other lib or framework same as KIO. but more small and light. I just need ssh and ftp.
Hi,
Thank you for sharing, i am having the same need (transfer folder from embedded linux to windows shared folder), but sadly i am using Qt4, and i think that KIO work only with Qt5.
Do you know if there is a similar framework working with Qt4 ?
KIO exists for Qt4 as well, as part of the kdelibs package. This brings in more dependencies though; things have been more clearly separated between Qt4-based kdelibs and Qt5-based KF5. Alternatively you could try using libsmbclient directly, possibly based on the kio_smb code.
Thank you for sharing! would it can be used in my Qt application that running on windows?
Yes, KIO works on Windows, and all the unittests pass. Proof, from KDE’s continuous integration https://build.kde.org/job/Frameworks/job/kio/job/kf5-qt5%20WindowsMSVCQt5.14/
To use it on Windows, I recommend using Craft in order to fetch, compile and install automatically KIO and all of its dependencies. https://community.kde.org/Craft