Skip to content

Setting up Kinect for programming in Linux (part 1)

17.04.2012 34 comments

The Kinect, a Microsoft device originally made for the Xbox360 (a gaming console), has become incredibly popular among developers in the past few months; it allows easy tracking of predefined movements without having to wear special clothes or complicated sensors. Once you’re set-up, you would be able to access functions such as hand tracking, scene analysis (count how many people are in the room, where they are), and much more. The first part of this tutorial will guide you through all the required steps to set up the Kinect on your Ubuntu environment.

The Kinect at a glance

kinect_IR

The device, with its two cameras, and the IR projector on the left.

How does it work? Using the Structured light technique, the Kinect outputs a depth map, which is basically a matrix containing the distance of each pixel according to the camera.

The device, as recognized by the computer.

The Kinect also comes with audio capabilities and a motorized tilt, so we end up with 3 sub-devices:

Bus 002 Device 003: ID 045e:02b0 Microsoft Corp. Xbox NUI Motor
Bus 002 Device 004: ID 045e:02ad Microsoft Corp. Xbox NUI Audio
Bus 002 Device 005: ID 045e:02ae Microsoft Corp. Xbox NUI Camera
  • Xbox NUI Motor
    • Handles the tilt engine and the status LED.
  • Xbox NUI Audio
    • Handles microphones array
  • Xbox NUI Camera
    • Handle the cameras

Communicating with the Kinect

Warning: I first tried to define custom installation directories for the different libraries, but it turned out that they need default paths (like /usr/bin) in order to work properly.

Software

To operate the Kinect, we need two drivers:

To control the LED and the tilt motor, we will use freenect library (open source, unofficial):

To control the video flux and get the depth map, we’ll take openNI drivers (open source, official):

We also need PrimeSense(the company that makes the Kinect) sensor module (open source): Warning: Official PrimeSense driver is not compatible with the Kinect, we need to take a modified version.

Finally, we use a higher level library to get hand tracking, predefined gestures (swipes, push, circles, etc) without having to create the algorithms.

Warning: As this library is not open source and therefore only available as binaries, you only would be able to install/use it on Windows or Ubuntu.

Environment setup

1- Download needed libraries/software.

mkdir KinectLibs; cd KinectLibs
git clone https://github.com/OpenKinect/libfreenect
git clone https://github.com/OpenNI/OpenNI
git clone https://github.com/avin2/SensorKinect

sudo apt-get install cmake libglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev python
  • if you get “Unable to locate package libglut3-dev”, use this command instead:
sudo apt-get install cmake freeglut3-dev pkg-config build-essential libxmu-dev libxi-dev libusb-1.0-0-dev python

sudo add-apt-repository "deb http://archive.canonical.com/ lucid partner"
sudo apt-get update
sudo apt-get install sun-java6-jdk

sudo apt-get install doxygen mono-complete graphviz

 

1- Install openKinect (libFreenect)

# in libfreenect directory, in the KinectLibs dir
mkdir build
cd build
cmake ..
make
sudo make install
sudo ldconfig /usr/local/lib64/

 

  • Once libFreenect is installed, plug the Kinect, then set permission to R/W on the usb devices (motor and camera).
sudo chmod a+rw /dev/bus/usb//
sudo chmod a+rw /dev/bus/usb//

lsusb | grep Xbox

libusb couldn't open USB device /dev/bus/usb/001/006: Permission denied.
libusb requires write access to USB device nodes.
  • Now, let’s see if everything is correctly setup, just run glview, you should get something like

Kinect

Didn’t get that? try the library FAQ

Tip: you can play a bit with the features with these commands:

‘w’-tilt up, ‘s’-level, ‘x’-tilt down, ‘0’-‘6’-select LED mode, ‘f’-video format

On the left there is an openGL representation of the depth map, the pixel color is set according to the point’s distance to the sensor, on the right you can get the regular RGB camera view, or the infrared one (so you can see the infrared pattern, switch between them with ‘f’)

Let’s now have a look on how to setup the gesture recognition libraries.

2- Install OpenNi

We just installed a perfectly fine working library here, that seems to handle all functions of the Kinect, why would we need another one?

It’s because of the high level library, NITE, which works only with OpenNi drivers, but the OpenNi drivers (which are not Kinect specific) can’t control the Kinect motorized tilt or it’s led. So we need both libraries to have full access to the Kinect.

So basically:

  • we will use libfreenect to control the tilt and the led (so the device Xbox NUI Motor, which also handle the led).
  • we will use OpenNi + Sensor module to get the camera streams (the device Xbox NUI Camera)
  • we will use NITE libraries in concert with OpenNI to get the high level API (so gesture recognition, hand/skeleton tracking and so on)
Note: Xbox NUI Audio device is handled by OpenNi, not libfreenect.
# in OpenNI directory, in the ultratronik dir
cd Platform/Linux/CreateRedist
chmod +x ./RedistMaker
./RedistMaker
cd ../Redist/OpenNI-Bin-Dev-Linux-x64-v1.5.2.23/
sudo ./install.sh

Note: it’s Sensor-Bin-Linux-x64-v5.1.0.25for me, but might be different for you, there is only one directory in Redist/ anyway, just replace in case the name is wrong.

4- Install NITE

  • Download the library according to your system, then just run install.sh as root. that’s it.

You’re now all set for using the kinect!

Discover the Kinect potential with the examples

Go into your NITE directory, then

cd Samples/Bin/x64-Release ; ls Sample*

These are the available examples, these cover pretty much all the high level recognition handled by NITE.

You can find detailed documentation of these functions in NITE/Documentation/ directory, here is just a “quick start” guide for each example.

Sample-Boxes

  • move your hand left<>right (quickly) in front of the sensor until the green sliders appears on screen, you now control the slider with your hand, the cubes are zones.

Sample-CircleControl

  • move your hand left<>right (quickly) in front of the sensor until the window gets a green frame, then make circles with your hand, you now control the spoke.

Sample-Players

  • for this one, it’s better if the kinect see your entire body. Get in front of the kinect, you will turn blue, ask some friends to come, they will be detected as well (different colors). Then put your hands up, the kinect now tracks your skeleton. It seems to work only for one player though.

Sample-PointServer

  • not sure about this one, it seems to be a handTracker, in a separate thread.

Sample-PointViewer

  • move your hand left<>right (quickly) or click (push your screen) in front of the sensor until your hand gets tracked.

Sample-SceneAnalysis

  • seems similar to Sample-Players, without skeleton tracking

Sample-SingleControl

  • hold your hand steady in front of the Kinect then gently push forward and come back, you’ll get hand points and gesture tracking in debug output (no openGL fancyness this time)

Sample-TrackPad

  • move your hand left<>right (quickly) or click (push your screen), once you get the focus, move your hand around, squares get selected according to the hand position.

The samples don’t work, I get “Find generator failed Input pointer is null!”

The program needs xml files in order to setup it’s configuration:

  • what we are going to use (handtracking, gesture recognition)
  • Some device info (licences and so)
  • and other settings, have a look here if you want to know everything about these files.

So, the samples are configured to search for the xml files in ../../../Data/ and if they don’t find them, you’ll get this error.

Solution: recompile the samples with another path or put the xml files where the samples expect them.

 

Next month I’ll show you how to use the Kinect in your Qt applications...

 

Categories: KDAB Blogs / KDAB on Qt / Kinect

34 thoughts on “Setting up Kinect for programming in Linux (part 1)”

  2. Dion 05.09.2012 12:16 pm

    hello .

    I am working with the kinect to reconize some gestures but when i try to modifiy the samples for example (pointViewer), i can’t build the file after, i have a probleme with the Glut librairy .compilator tell me that it don’t know any functions of this librairiy by putting many “undefined reference”.

    However, i have include all librairy glut.Now i don’t know where does this probleme come from.

    If can help me.

    Your Sincerely

    Reply
    1. Benoit 12.03.2013 3:12 pm

      Hi Dion,

      It looks like a linkage problem, you not only need to specify the headers file to the compiler, but also where to find the library, it is the option ‘-l’ on gcc, for example.

      Hope that helps,
      Benoit

      Reply
  3. keven 20.11.2012 5:37 pm

    Hi
    Im trying to install openni, the step im having trouble with is running ./RedistMaker
    during the “creating tar……..” the process stops “Redist OpenNi Ended. !!”
    I can post the terminal output if someone is willing to help me

    i was able to install everything before this step except opedjdk6(apt-get could not find it). i had to install openjdk7-jdk which i assume provides the same functionality

    thank you so much!

    Reply
    1. Benoit 12.03.2013 3:13 pm

      Hi Keven,

      Sorry for the late reply, tell me if you’re still having trouble with that, i’ll see what i can do to help.

      Benoit

      Reply
  4. Ansoooo 21.01.2013 4:12 pm

    Hi,
    When I run glview I get this error :

    Kinect camera test
    Number of devices found: 1
    Could not open device

    With lsusb | grep Xbox, I get this :

    Bus 002 Device 015: ID 045e:02ad Microsoft Corp. Xbox NUI Audio
    Bus 002 Device 016: ID 045e:02ae Microsoft Corp. Xbox NUI Camera

    So the Motor device is not detected.
    I think it’s the problem but I don’t know how to solve it and I don’t find any answer on the web.
    Do you have an idea ?
    Thanks a lot !

    Reply
    1. Benoit 12.03.2013 3:16 pm

      Hi Ansoooo,

      It is indeed weird that the motor is not recognized, but it should work without it. Did you find a solution to your problem? I can try to help otherwise.

      Benoit

      Reply
  5. RJ 17.06.2013 7:16 pm

    I have the most recent repos, and I have followed the tutorial. When I run glview though I get:

    $ glview
    Kinect camera test
    Number of devices found: 0

    I can run the NiViewer from the OpenNI samples, and it finds the kinect just fine.

    Reply
    1. Ajith 24.02.2014 12:55 pm

      hey ur prob is the sensorkinect driver. In avin2/SensorKinect the Depth Generator is missing. Download the ph4m/SensorKinect from git hub… itl surely work..!!!!

      Reply
  7. vinicius 17.01.2014 9:48 pm

    I have one error in command “sudo ./RedistMaker” by Sensokinect return this:

    make[1]: *** [x64-Release/XnSensorDepthGenerator.o] Error 1
    make[1]: Leaving directory `/home/pcimobilis/Iniciacao/kinect/OpenNI/Platform/Linux/Redist/OpenNI-Bin-Dev-Linux-x64-v1.5.7.10/SensorKinect/Platform/Linux/Build/XnDeviceSensorV2′
    make: *** [XnDeviceSensorV2] Error 2
    make: *** Waiting for unfinished jobs….

    And not creat past /Redist

    Reply
    1. Benoit 19.01.2014 7:18 pm

      Hi vinicius,

      It looks like Make did not manage to compile the XnDeviceSensorV2 element. You can get more information on the error by typing “make XnDeviceSensorV2” (so it compiles only the problematic element).

      Benoit

      Reply
  8. Abiud Rodas 11.02.2014 4:36 pm

    Hi Benoit, I’ve installed everything and the Kinect works very well when I run the glviewer, also I’ve installed the NITE library but when I try to run any sample i get the follow error:

    Open Device failed:
    DeviceOpen using default: no devices found

    I’m not sure what is wrong, and I dont understand what exactly I have to do about the xml file, and I dont know where they are. Can you help me to clarify these problems please?

    Thanks in advance.

    Reply
  9. Awadhesh 18.02.2014 5:02 am

    It is a very good tutorial, but my ubuntu system is not detecting kinect sensor on usb, while I have run the command “sudo chmod a+rw /dev/bus/usb//”. Infact write permission is not granted to the usb. Please help….

    Reply
  12. Wango 29.08.2014 7:14 pm

    Is it possible to use Openni libs and OpenKinects libfreenct simultaneous on the same Device?

    Reply
  13. Sawabi 10.09.2014 9:33 pm

    Hi,
    I’m a newbie to programing so my question might be primitive for most of you. Anyway i would be very grateful for any kind of help.
    I’m trying to follow the installation instructions above. While installing openKinect i’m facing a problem.
    I was executing the make command when i got the following error

    [ 51%] Built target freenect
    [ 51%] Built target freenectstatic
    Linking C executable ../bin/freenect-chunkview
    ../lib/libfreenect.so.0.5.0: undefined reference to `libusb_get_parent’
    collect2: ld gab 1 als Ende-Status zurück
    make[2]: *** [bin/freenect-chunkview] Fehler 1
    make[1]: *** [examples/CMakeFiles/freenect-chunkview.dir/all] Fehler 2
    make: *** [all] Fehler 2

    Seems like there is a problem with the libusb library. I’m pretty sure that it’s installed correctly.
    Hope someone can help me.
    Thank you

    Reply
  14. Alexey 24.09.2014 12:44 pm

    Hi. Try complete all install by your instruction, but today openni project is closed, there are only lib simpleopenni. I’m install all except NITE, there is no such lib today. How can i use kinect on linux without NITE, or where can i download it?

    Reply
  18. dieme 15.05.2015 9:41 am

    hello,
    i have a question, if some one can answer me please:
    can kinect work with raseberry!!! if yes; what do i need!!!!

    Reply
  20. Loan Cardoso 10.06.2015 12:20 am

    What did you mean by running glview??? I typed glview on my terminal and it didn’t work out. should I time anything else ( like “run glview”)? I’m new to this.

    Reply
  21. Khadim 21.06.2015 7:59 pm

    I spent a lot of time to have the skeleton in ubuntu 14.04 with the xbox kinect . thank you for helping me

    Reply
    1. roro 09.09.2015 9:30 pm

      how to do that help me please because I cannot capture data from kinect by using ubuntu 14.04, opencv 3.0, openni or libfreenect and eclipse cpp

      Reply
Leave a Reply

Your email address will not be published. Required fields are marked *