Creating a Qt 5 port to Apple tvOS

Back in November, Apple released the latest generation of it's Apple TV product. Besides the slightly improved hardware, the true new feature is the OS which is now officially based on iOS and comes with the dedicated SDK and App Store! So we started investigating what it would take to port Qt to tvOS and start writing some apps for the big screen.

Turns out, it's pretty straight forward (see other change list for the various modules). The port involves a new mkspec, setting some build flags, disabling a few things tvOS does not do compared to iOS. Et voilà, a working environment to port your QML app, yoohoo! That you can't interact with, boohoo!

Read on to see the current progress and check what remains to be done.

The Platform

From a technical point of view, tvOS 9.0, feature wise, is on par with iOS 9. But it differs in a number of ways:

• It requires bitcode compilation - this is simple compile time flag and will be a requirement for a future version of iOS (10?). Bitcode is an intermediate representation of the binary which enables Apple and iTunes Connect to perform optimisations on your application without need for recompilation or even uploading a new build (for example when a new processor is launched).
• Platform is 64bits only, based on Apple's A8 chip (as in the iPhone 6, previous generation), so dual core CPU and quad core GPU with OpenGL ES 3. It does have 2Gigs RAM though, we have noticed apps don't get killed in the background as much.
• A lot of underlying POSIX commands and or system calls that don't make sense on the platform are actually disabled in the API, like spawning processes, etc.
• It has no support screen orientation, status bar settings, clipboard, (a la iOS) menus, cameras, sensors and neither Webkit nor WebEngine (nor Webview!).
• User interaction is very different, using the provided remote. So no direct touch, presses and touch events are sent to the focus view, and the keyboard is always full screen with a dedicated input UI.
• Beyond the things disabled in the existing iOS build, we've disabled DBus (can't use it anyway) and Widgets (why would you use those for a 10' interface?).
• Supports shared library builds for device target only (not simulator or mixed targets).

The Port

To start the port, we duplicated the iOS mkspec and tweak the build options to target the tvOS device and simulator SDKs, add the required compile flags, etc.

mkspec?

Every platform supported by Qt must provide an mkspec which is composed of a collection of files containing build instructions (such as which modules are supported), what the compiler and linker options are, and many more things.

Importantly in the case of iOS and tvOS, it will instruct qmake to cross-compile for the correct architectures, to build for both the device and simulators, etc.

A tvos config was added to qmake, along with the pre-existing Q_OS_TVOS compile time macro. These are not linked to their iOS counter parts, so ios does not imply tvos or the other way around.

After the initial setup, it became apparent that the differences between iOS and tvOS where quite minimal, mostly related to the choice of the SDKs, the target devices, and so on. So the iOS mkspec (and dependent files) where refactored to minimise the amount of duplication.

Beyond setting up the mkspec, we also needed to make sure the parts of build system and code that had options specific to iOS (using the ios config in qmake or the Q_OS_IOS define in code) where also handled properly for tvOS. This was done in all modules currently available.

Building Qt and your Apps

To build Qt (or your own apps), you will need Xcode 7.2 which includes tvOS 9.1. There's no reason if should not work with the previous release but it has not been tested.

Many of the iOS specific entries in the Info.plist file are also different on tvOS so a new template was added. tvOS also requires asset catalogs for the app icons.

Building Qt itself requires passing the -xplatform macx-tvos-clang option to the configuration script.

As with the iOS builds, the tvOS port currently builds static libraries by default. We have confirmed that building shared frameworks is possible but, in that case, only the appletvos SDK will be built. The option is however not publicly available (pending further testing).

Creator does not currently support tvOS builds properly so building your own apps will require invoking qmake on the command line and opening the generated project in Xcode (gasp!).

Finally, tvOS has pretty strict rules on the size of the final application bundle. This appears to be enforced at the launcher level (rather than at the App Store submission level) which means debug builds are too large and will not run on the device (they work fine in the simulator).

So, build Qt for tvOS, then build your favourite app, run it on your big screen. Tada!

User interactions

Then you try to interact with your app and nothing works. Main issue is that the interaction paradigm for a large screen you're too far away to touch is completely different from using a phone or even a desktop computer. There's no direct touch on any UI component, nor is there any concept of a mouse pointer to be moved around.

Now, the Apple TV comes with a remote that does have a touch surface. And that does report touch events and gestures. But, when using tvOS's native SDK, these are dispatched to the view that currently has the focus. However all the coordinates for the touch gestures always start in the centre of the screen and move from there (i.e. they are not reported in the coordinate system of the button or whatever element has the focus).

So in most QML apps, since there is a single native view, all events are reported to the event dispatch system as happening in the centre of the screen.

The native SDK also has a system to gracefully handle the change in focus based on the layout of the focusable views, so that swiping left, right, up or down will move the focus to the next view in a predictable (and developer controllable) way.

All this is to say that something needs to be done so that QML can provide a similar framework for graceful event dispatching and focus handling.

Meanwhile, you'd have to add a top level fullscreen MouseArea handle clicks, gestures, active items, etc. on your own.

Qt3D and games

An area where interaction may be simpler is gaming and using controllers. Apple provides an SDK for the Made-For-iOS type controllers and the remote bundled with the Apple TV is actually supported as such a controller, including tilt sensors, etc.

Using Qt3D and the upcoming input-device framework, it is possible to use such gaming controllers for game-type applications.

We are also making sure the qtgamepad project in qt-labs also support tvOS and it's remote.

Further work

Well, obvious things: reviews to the current ground work so it can be merged. Then sorting out the interactions issue in a nice manner. And getting Creator to properly configure and deploy the apps.

Also simple message boxes need to be added, and text input (something like a QML version of QInputDialog would be nice).

Hoping to get more people involved for this :)