Ultimate Löve Toy (kinect + mindstorm nxt 2.0)

[hagish]

I just finished the first milestone in building the ultimate love toy.
I added kinect and mindstorm nxt 2.0 support into love (both native löve modules).

Code: https://bitbucket.org/hagish/love-kinect-mindstorm

It's based on the current love tip (rude/love at bitbucket). At the moment the code is linux only, but it should be possible to port it to other systems.
Kinect support is based on libfreenect and nxt2 on a win/linux lib (i don't have a clue if there is mac support).

Current kinect features:
* Cam image
* Depth image
* Set LED status

Current mindstorm nxt 2.0 features:
* Connect via Bluetooth
* Control 1-3 motors
* Receive sensor information

The attachment contains the löve no-game screen with the 2 kinect cam images. And if this would be a video you could see that the planet rotates if you hold sensor button 1 and stops if you release it.

I'll add some documentation and stuff and then I'll upload it to bitbucket.
---

[middlerun]

Kinect in Löve! My ultimate dream...

Is it possible to get skeleton data from the Kinect into Löve? That would open up some amazing possibilities.

[Robin]

That would be very awesome!

[hagish]

Currently there is no skeleton detectoin because libfreenect does not provide this data. I'm not sure if there is a free lib that can do it.
One could integrate OpenNI and PrimeSense for skeleton detection. But PrimeSense is not free in an open-source way (as far as I remember).

[vrld]

Awesome! Do we have access to the camera and depth image as in ImageData?

Is it possible to get skeleton data from the Kinect into Löve?

The skeleton data is not calculated on the kinect itself, but done in software. All you can get from the kinect is a camera image (actually two), a depth image and sound from a microphone array. Libraries like OpenNI and the Microsoft SDK use the depth image and rather complicated algorithms to get skeleton data.

[hagish]

Yes there is a getDepthAtXy : number, getDepthImage : ImageData and getRgbImage : ImageData.

[hagish]

I just cleaned up my commit history and pushed the current version: https://bitbucket.org/hagish/love-kinect-mindstorm

[Lafolie]

Most impressive.