Being John Malkovich...'s Face

So let's say you've got a webcam, some photos of a celebrity, and a Computer Science degree. Why not make a celebrity facial puppeteering system? In this 2010 research demonstration, several University of Washington researchers demonstrate a Being John Malkovitch Malkovich-style puppeteering method, in which a webcam watches the puppeteer's face and the computer selects similar frames from another face, causing the virtual "puppet" to act along. Given a sufficiently large database of frames, presumably this could get pretty smooth. Here's the abstract:

Given a photo of person A, we seek a photo of person B with similar pose and expression. Solving this problem enables a form of puppetry, in which one person appears to control the face of another. When deployed on a webcam-equipped computer, our approach enables a user to control another person’s face in real-time. This image-retrieval-inspired approach employs a fully-automated pipeline of face analysis techniques, and is extremely general—we can puppet anyone directly from their photo collection or videos in which they appear. We show several examples using images and videos of celebrities from the Internet.

Now let's get all

Malkovitch

Malkovich* up in here:

Being John Malkovich - ECCV 2010 from Ira Kemelmacher on Vimeo.

* = If you haven't seen Being John Malkovich, this isn't really a spoiler, but it has to do with issues of puppetry, identity, and inhabiting other people. The film includes a famous sequence involving a room full of Malkoviches, though they were done with prosthetics (if Adaptation is to be believed).

(Via the always-dependable Kottke.org.)