The Xbox 360 was a popular home video game console by Microsoft. It was available in retail stores from 2005 to 2016 in different versions. The Xbox 360 was the first game console which was compatible and equipped with a peripheral high-performance motion sensing input device. These motion sensing input devices were sold under the brand name Kinect.
When playing a console game that supports Kinect there is no need for a game controller or any other hardware. Instead, Kinect provides a natural user interface. The game console is able to detect and interpret the user’s motion. The detected motion data can be used for controlling the game or any other application that runs on an Xbox 360.
Kinect uses a laser grid for depth calculations (left). The visualization (right) uses color gradients for different distances. (Images by Colossus, CC BY-SA 3.0, taken from Wikimedia)
Kinect had been available for Microsoft game consoles from 2010 to 2017. In 2017 Kinect was discontinued. It had turned out, that the user acceptance – and accordingly the number of games supporting Kinect - was not sufficient for the consumer mass market. At the same time, Microsoft continued the development of the Kinect technology in a different product line. Microsoft HoloLens is a pair of mixed reality smart glasses for the corporate market. The HoloLens includes the latest generation of Kinect sensors. A successor for the original HoloLens (which had been released in 2016) is expected for 2019. The HoloLens was introduced to the market for a price of 3.000 US-Dollar. By comparison, a set of the Xbox 360 (with Kinect sensor) had been available for just 300 Dollar.
Motion sensing had failed to prove its commercial success for game consoles, but there are many other cases which can benefit from motion detection. Consequently, the discontinued Xbox 360 is still an obvious choice for every experiment that requires inexpensive but reliable detection of user motion.
Kinect hardware & software
The Kinect system consists of well-adapted hardware and software components. It supports 3D full-body motion capture, voice recognition and facial recognition. The hardware components are the game console Xbox 360 and the peripheral Kinect sensor. Like any other modern game console, the Xbox 360 is basically a compact computer with a proprietary operating system and internet access. Users can install additional apps on the console if needed.
Kinect is able to perform a 3D-scan on an object, here a man standing on the right. This data can be used for 3D-visualisation on screen. (Image by Steven Walling, CC BY-SA 3.0. , taken from Wikimedia)
The Kinect sensor is a black rectangular bar which is designed to be mounted at a flat screen monitor. It includes a multi-array microphone, a depth sensor and an RGB-camera. The depth sensor works as any 3D scanner, it is able to detect items or human beings and their motion. The depth sensor uses an infrared laser and it will work even in low-light environments. Acoustic source localization is provided by the multi-array microphone.
The software consists of Kinect system software which Microsoft integrated into the Xbox 360 dashboard. Third party applications and open source drivers for Kinect are available as well. Microsoft does not support the use of open source drivers, but these are nevertheless quite popular for use scenarios that are not covered by the manufacturer endorsed software. All in all 35 million Kinect units for Xbox 360 were sold before Microsoft discontinued the product line. For the foreseeable future, there will be no shortage of used Kinetic hardware or corresponding applications.
Using Kinect in the HRI-lab
Kinect was used in the HRI-lab to monitor users’ nonverbal behaviors, examples would be to get information on how well people are doing an exercise by tracking e.g. location of their hands on the y-axis, or how far people are away from the robot by monitoring the distance between them and the sensor on the z-axis. In that way, Kinect can help researchers to collect important data from experiments.