Kinect
Glossary
A
pixel is the smallest unit of an image.
Skeletonization
The process of inferring the subject's skeleton (where their bones and those connections are)
Introduction
The Kinect was originally developed by Microsoft as an attachment for the Xbox game console. Using a camera with an infrared light not visible to the human eye, it tracks the movement of people and objects in front of it in three dimensions. At $150, it is a low-cost way to capture three-dimensional and gestural information.
The Kinect is a depth camera. Instead of recording the dolor of light, it records the distance of the objects in front of it.It records the distance of the objects that are in front of it by using an IR projector and camera. Using infrared light, the Kinect captures the data of where objects are in space.
Depth images are easy for a computer to understand. Color data makes it is difficult for a programmer to pick out objects, but depth information, makes it much easier to separate an object from its background.
Infrared light has a wavelength that's longer than visible light, which is why we can't see it. The Kinect's infrared projector shines a grid of infrared dots over everything in front of it. The IR camera can capture this grid.
Each Kinect is calibrated to know exactly where each of the dots from its projector appears when projected against a flat wall at a known distance. Any object that is closer than the Kinect's calibration distance will push these dots out of position in one direction and any object that's further away will push them out of position in the other direction. It uses the displacement to figure out where each object is in space.
Using Processing, you can write programs that examine the brightness of each pixel in the scene and determine the distance of every object in front of the Kinect.
Processing stores pixels as 8-bit numbers between 0 and 255.
Lesson 1
Read pages 1-12 of
Greg Borenstein's Making Things See.
Distribute the interviews at the end of the chapter to individual or teams of students. Each individual or team has 3 weeks to prepare a 10 minute presentation on their reading. Presentations are completely open. Students may present videos, personal projects, descriptions of their artist's projects or other related inspiration.
Alternate Research Questions
Students must use at least two sources and present their findings
- Research the technology behind the kinect and explain how it works.
- Present a history of related technologies
- Discuss what socio-political factors influenced the technology behind the Kinect
Other Resources
Kinect Hacks
Be Your Own Souvenir
Combining the Kinect with a 3D printer, Be Your Own Souvenir allowed tourists to create a souvenir of themselves. After striking a pose on the pedestal, the image was converted into GCode which was sent to the 3D printer.
Fabricate Yourself
SimpleOpenNI
SimpleOpenNI is a library that provides access to all of the
data from the Kinect, as well as a series of tools and helpers.
- Install PrimeSense's OpenNI in your applications folder. This application communicates with the Kinect in order to access and process the data
MAC
Download OpenNI_NITE_Installer-OSX-0.xx.zip and Max Rheiner's SimpleOpenNI
- On the MAC you should see the following:
- install.sh
- LGPL.txt
- Readme.txt
- data
- Open Terminal (Command_CTRL+U)
- Change directory to the downloaded folder (type cd and a space, then drag the folder into the terminal window, then press RETURN)
- Run the installer with the command:
Type your password when
prompted.
Windows
Installing OpenNI requires a number of pieces of supporting software that all have to
be installed for things to work.
A company called ZigFu provides a oneclick
installer that puts all the pieces in place for you. - Download the ZigFu DevBundle (or download all the files from here.
If you’ve installed the Microsoft Kinect SDK, you need to uninstall it.
The OpenNI software does not work with the Microsoft Kinect drivers.
- Install the SimpleOpenNI library
in your Processing>Libraries folder
- Quit and restart Processing if it was open
- Plug in the Kinect and make sure it is facing you.
- Test out the install by running one of the built-in examples that ships with
SimpleOpenNI. Inside of OpenNI, find DepthImage and double-click it.
This is a basic sketch. Hit
the play button in the top left corner of the Processing window to run the sketch.(Slow start
up is normal when using SimpleOpenNI.)
- If the sketch fails, you might see this
Invalid memory access of location 0x8 eip=0xb48d8fda
If you see that your Kinect is not plugged in or not fully
powered. Make sure your Kinect is connected to a power outlet and plugged into your
computer’s USB port. If you’re working on a laptop also make sure that your laptop is
plugged into power. Some laptops will provide inadequate
power to their USB ports when running off a battery.