Innovation in gaming and technology gets us pretty fired up at BLITZ. And when a new interaction peripheral (and engagement model) is released, the first thought that comes to our minds is “let’s see if we can hack it.” In this case, Director of Technology Noah Gedrich and Software Developer Yosef Flomin can take their bows for linking the Xbox Kinect™ hardware to Adobe Flash via Node.js.

Their interest was piqued after Hector Martin and PrimeSense™ released their open-source drivers, resulting in an online demo showing Kinect’s uses through a PC. Having worked with multiple UI technologies over the years, Gedrich and Flomin realized that if they could get Kinect to publish information that any UI platform could support, a slew of developers could use this same code to create breakthrough motion-based experiences using familiar markup languages.

The trick? A simple socket server. Using a C++ application to send all the skeleton data to a socket server, they were able to connect other technology to the socket, enabling use of the data.

When Kinect was first announced, we immediately began dreaming of the different ways we could use it to create engaging experiences that leveraged physical interaction for our clients. Unfortunately, in the earliest days, we would have been tied to Xbox’s proprietary XDK and only able to publish to the Xbox itself. With this breakthrough, we’re arming any Flash, Silverlight or Unity developer around the world with an intuitive way to implement physical interaction models into their work. Be it a large-scale installation or a desktop application, marketers, agencies and developers can save a great deal of time, energy and money — opening up the potential of Kinect beyond Xbox to any platform supporting socket connections increases the creative possibilities exponentially.

Watch the video below or on Vimeo

Flash Kinect Demo from BLITZ Agency on Vimeo.

Try it out for yourself. Follow Gedrich and Flomin’s personal process below, complete with necessary links and helpful sites.

Setting up Kinect Drivers:
Check out Dan Ivanovs blog on installing the drivers, OpenNi and NITE from PrimeSense.
http://cleoag.ru/2010/12/10/kinect-OpenNi/

Setting up our socket server:
For our socket server, we used node.js. To get started with node.js on Windows, we first needed cygwin installed with the appropriate packages. Grab the cygwin setup at:
http://cygwin.com/install.html

When you have setup running take a look on github at https://github.com/ry/node/wiki/Building-node.js-on-Cygwin-(Windows) for the correct packages and steps that you’ll need to take to correctly install node.js

Great. Now let’s take a look at a simple node.js server at http://nodejs.org/. We’ve modified that basic server a bit to have our data coming through on an interval:

Now that we have that setup, we can modify any of the PrimeSense samples to send off the Kinect data to the socket server and register any of our other platforms as listening tools.

A basic flash connection to the socket server will look something like this:

Now we can easily parse the string according to the format we sent from the Kinect sample. We can then use that data in Flash.

We hope that helps and will aid in your Kinect hackery. Shoot us a line or drop a note in the comments if you have any thoughts on this approach. And don’t be shy with your own workarounds and Kinect projects — we’d love to see what the community comes up with.

UPDATE:
We got a lot of requests for the modifications to the primesense sample that we used to connect the c++ side to the node.js server. For your downloading pleasure we now have this Kinect+Flash c++ code.

This file should update the main.cpp file from the SingleControl app example. The two modified sections are commented as “SOCKET SERVER INITIALIZATION” and “SEND DATA TO SOCKET SERVER.” As you can expect with code like this, BLITZ isn’t really offering any support and if somehow it manages to destroy your computer, we didn’t do it.

18 comments