One DataModel to filter them all

30.01.2013 Tobias Koenig 3 comments

Today I want to talk about the usage of data models in Cascades. As you might know already, Cascades provides the abstract interface bb::cascades::DataModel, which is used by bb::cascades::ListView to retrieve and display arbitrary data. Cascades also provides a couple of convenience classes that implement this DataModel interface, namely

• ArrayDataModel
• GroupDataModel
• QListDataModel
• XmlDataModel

While the ArrayDataModel, GroupDataModel and QListDataModel are designed to store the data inside the model, the XmlDataModel loads them from an external data source (e.g. a XML file). If you want a data model that loads the data from an existing data structure (let’s say your own business logic objects), then you have to implement your own model, inheriting the bb::cascades::DataModel interface.

Data duplication

Now that we know which models exist and how they handle the data, let’s have a look at the following scenario. We want to implement a simple file browser. It should list all files from a given directory in the file system and show them in 3 ListViews. The first ListView should show all files, the second one only files that are owned by me and the third one only music files of the OGG format that are owned by me. Furthermore we want that the ListView groups the files by the first character of the file name.

The naive approach (which is taken quite often ;)) would be to create three separated GroupDataModel objects (we use GroupDataModel, because it provides the grouping feature for free), iterate over the files in the given directory, test each file against the filter criterion and insert entries into the three models accordingly. While the implementation is quite trivial, this approach has serious drawbacks:

1. If a file matches all three filter criteria (e.g. a OGG file owned by me), we have the same file entry in all three GroupDataModel instances, so we need 3 times the memory.
2. Whenever the content of the directory changes (e.g. a file is added or removed), we have to update all three models.

So if we work on large data sets and want to show multiple different subsets in different ListViews, the naive approach simply won’t scale.

Reduce data duplication: The complex way

The core problem is, that the GroupDataModel forces you to put the data into the model instead of just working as an adaptor between the actual data (a list of file meta data) and the bb::cascades::DataModel interface. So if you want to keep the data in memory only once but have different views on certain subsets, there is currently no way around implementing your own DataModel.

Such a DataModel class would require the following functionality:

• implement grouping according to given grouping criterion
• implement sorting according to a given sort order
• implement filtering according to given filter criterion

And since you want to have three different views, you might end up with three different implementations of DataModels, with each requiring unit testing, code maintenance etc. etc.

… so probably not the way you want to go 😉

Reduce data duplication: The easy way

The problem described above is not a new one. Qt developers faced it for the last 8 years, since Qt 4.0 was published with the model-view framework. It provides basically the same functionality as the Cascades model-view framework, just that it’s more tailored to desktop UIs and not mobile UIs. The problem of viewing different subsets of data without using multiple data models has been solved with so called proxy models. These models are put between the actual data model and the view and act (what the name suggests) as proxies. To the view they look like a normal data model (by implementing the model interface) and to the actual data model (often called source model) they behave like a view that queries data.

In Cascades world that would mean, that we have a custom proxy model class which inherits from bb::cascades::DataModel and also takes a pointer to the source model. Instead of the actual source model, we now pass the proxy model to the ListView. If the ListView wants to retrieve the data, he asks the proxy model and the proxy model forwards the request to the source model. The source model returns the result to the proxy model and the proxy model returns it to the ListView… I think you get drift 😉

Just forwarding the requests and results between ListView and source model wouldn’t make much sense of course, we want the proxy model to actually modify the requests and responses. When we look at the bb::cascades::DataModel interface, we see the following virtual methods:

int childCount(const QVariantList &indexPath);
bool hasChildren(const QVariantList &indexPath);
QString itemType(const QVariantList &indexPath);
QVariant data(const QVariantList &indexPath);

While the first two methods are used to retrieve information about the structure of the model, the last two methods are used to retrieve the actual data. So if the proxy model reimplements all four methods, it can freely adapt the structure and also the content of the source model.

Let’s get back to our original problem: In this case we would use only one GroupDataModel and fill in the meta data of all files inside the directory. This model (we call it ‘sourceModel’ for the moment), can be used directly as ‘dataModel’ for the first ListView in our filebrowser application. Now we would implement a filter proxy model, which filters out all files that are not owned by me. This can be implemented by reimplementing the childCount() and hasChildren() method to reduce the number by the files that are not owned by me. Additionally we have to keep a mapping to know which indexPath in our model corresponds to which indexPath in the source model.

When we have such a filter proxy model (let’s call it ‘ownerFilterModel), we can put it on-top of the ‘sourceModel’ and use it as ‘dataModel’ for the second ListView. If the filter proxy model is implemented in a generic way, we could reuse it with a different filter criterion (filter by file type) on top of the ‘ownerFilterModel’ to list only OGG files that are owned by me.

So now we have a stack of data models:

   ListView1         ListView2           ListView3
       ^                 ^                    ^
       |                 |                    |
sourceModel -> ownerFilterModel -> fileTypeFilterModel

The huge advantage is that only sourceModel contains the data, if ListView3 asks the ‘fileTypeFilterModel’ for data, the ‘fileTypeFilterModel’ just looks up the corresponding indexPath in ownerFilterModel and calls the data() method on it.

The ‘ownerFilterModel’ itself also just maps the indexPath to the corresponding one in ‘sourceModel’ and calls the data() method of ‘sourceModel’. This one now returns the actual data upwards the chain until it reaches ListView3.

Now I can hear you screaming already: Oh my god, so many lookups while traversing the proxy model stack… how slow will that be… Actually it’s not slow at all if implemented correctly, which brings us to the next section…

Say Hello to FilterProxyDataModel

Implementing such a filter proxy model in a generic way is a bit tricky. You do not only have to care about changes of the filter criterion, but also about changes in the underlying source model, which requires updating your internal mapping whenever that happens. And you have to emit the right change signals with the correct index paths at the right time, so that the ListView behaves correctly. All this is something you want to do once and never again.

I can offer you now the chance to do it not even once, but use FilterProxyDataModel right away 🙂

To demonstrate you how to use it, let’s sketch up the implementation of our file browser example. Somewhere we have a function that iterates over the directory and reads in all meta data into the m_fileModel, which is a GroupDateModel.

void FileBrowser::scanDirectory()
{
    m_fileModel->clear();

    QDirIterator it(m_directory, QDir::NoDotAndDotDot);
    while (it.hasNext()) {
        it.next();

        const QFileInfo info = it.fileInfo();

        QVariantMap entry;
        entry["name"] = info.fileName();
        entry["extension"] = info.suffix();
        entry["owner"] = info.owner();
        // add more information here

        m_fileModel->insert(entry);
    }
}

Inside the constructor we would create the m_fileModel and put the FilterProxyDataModel instances on top of it

FileBrowser::FileBrowser(QObject *parent)
    : QObject(parent)
    , m_fileModel(new GroupDataModel(QStringList() << "name", this))
{
    m_ownerFilterModel = new OwnerFilterModel(this);
    m_onwerFilterModel->setSourceModel(m_fileModel);
    m_ownerFilterModel->setOwner("tokoe");

    m_fileTypeFilterModel = new FileTypeFilterModel(this);
    m_fileTypeFilterModel->setSourceModel(m_ownerModel);
    m_fileTypeFilterModel->setFileType("ogg");
}

So what do the OwnerFilterModel and FileTypeFilterModel look like?

class OwnerFilterModel : public FilterProxyModel
{
    Q_OBJECT

public:
    OwnerFilterModel(QObject *parent = 0)
        : FilterProxyModel(parent)
    {
    }

    void setOwner(const QString &owner)
    {
        m_owner = owner;
        invalidateFilter();
    }

protected:
    virtual bool acceptItem(const QVariantList &indexPath, 
            const QString &itemType, const QVariant &data) const
    {
        if (itemType == "header")
            return true;//we don't care about headers

        if (m_owner.isEmpty())//if no owner is given, we accept all
            return true;

        const QVariantMap entry = data.toMap();
        return (m_owner == entry["owner"].toString());
    }
}

You basically just have to implement a method to define the filter criterion (in this case setOwner()) and then reimplement the virtual method acceptItem() to check whether an item matches the criterion or not. The FilterProxyDataModel will ensure that acceptItem() is always called when needed (e.g. new item added to the source model or content of an item has changed). Implementing the FileTypeFilterModel is left as an exercise to the reader 🙂

The FilterProxyDataModel implementation is available at https://github.com/tokoe/cascades inside the filterproxydatamodel/src directory. It’s license is a non-restrictive one, so feel free to copy, modify and use it in your projects.

I hope you enjoy using it. If you should find any bugs (I covered it with many unit tests, but you never know…), please send me a mail to tobias.koenig@kdab.com.