Project: Head-tracking on PS3 – Now without glasses

Well, the R&D guys at SCEA went and did it. They’re using facial recognition to track on head on the screen, instead of using IR LEDs strapped to your head. They demonstrated it at GDC, cool stuff!

The feat was reported by Kotaku, Gizmodo, Joystiq and others, but essentially they all point to the same video on mtv.com which has a silly IP ban for UK, Japan and Canada. Those countries clearly have no right to know about glassless headtracking.

Same video on youtube with no IP ban:

Project: Head-tracking on PS3

Thomas Miller IV from SCEA game studios posted a cool video about his Desktop VR port to the PS3 (embedded below). This is a very interesting concept and well worth watching.

It uses the Playstation Eye webcam to track IR LEDs that you wear on your head on the side of a pair of glasses. As you move your head, it moves the 3D view on the screen accordingly, giving you a great impression of depth. When you look at the face of a cube and lean left to look at its side, you actually see its side.

I built the glasses out of junk remote controls lying around. Took the IR LEDs, the remote battery compartment, even the on/off switch. Glued the lot together on my friend’s sunglasses and added a couple of wires. Job done. No resistors needed that I could tell, the IR LEDs use 1.6V, so they should be fine wired straight to a couple of 1.5V batteries in series. Post in the comments to tell me otherwise before they blow up and I’ll owe you one. I could have used button batteries to make it swish and subtle, but then it would have blown my budget of £0, so I went for the fat ones from the remote controls.

We taped developed film on the Playstation Eye as an IR filter (see photos), as recommended in the video. It works pretty well. To improve the image quality it’s recommended to open the webcam, and remove the IR filter (it’s a pale green film on the lens), but it’s not strictly necessary to do it. (Note that the IR filter inside the webcam prevents IR light from passing through, whereas our developed film tends to only let the IR light through, not sure what the proper vocabulary is there, they probably have different names).

The PS3 demo written by tmiv is pretty simple, and neat in that way. It takes the image from the webcam and looks for the two dots of light from the IR LEDs (also supports one dot for controlling using a normal remote control). As the dots move on the X/Y plane, it moves the camera accordingly, giving you the impression that you are looking on the side of the object. The effect is pretty uncanny! By using the distance between the points, it knows when you’re getting closer or further from the webcam, and changes the depth too.

–
Video from Thomas: