Mashing Up Pixels and Puppets

Long time readers of the blog are probably familiar with Machin-X: Digital Puppetry, which I started back in December of 2005 as I became increasingly interested in Machinima and other ways that puppetry is being combined with new technology. Machinima is still developing technically and artistically and many “Machinimists” are realizing that they are creating a new type of puppetry . The Academy of Machinima Arts and Sciences has even started giving out an annual “Mackie” award for Best Virtual Performance: Puppeteering.

Unfortunately, one of the major drawbacks of Machinima is that in even the best films the expressiveness of the characters is awful compared to what someone can do with even the simplest sock puppet. Most Machinimists come from gaming and computer graphics backgrounds and when they talk about this problem the discussion often focuses on finding new and better graphics software to work with. It seems to me though that while better graphics couldn’t hurt Machinima, what it really needs most is better, more expressive puppetry capabilities.

After all, who knows more about manipulating things and creating performances in real-time than puppeteers?

This isn’t a new idea. Jim Henson was one of the first people to see the potential of digital puppetry and began experimenting with a digital version of Kermit in the mid-1980s, which led to Waldo C. Graphic. While Waldo is considered the first true digital puppet, he was at least partially based on work that various researchers had been doing as early as the late 1970s. Today, expensive high-end real-time animation systems are being built and used by companies like Henson (HDPS) and Industrial Light and Magic (Zviz) to create digital performances in real-time. What’s really lacking is an affordable, accessible tool that’s available to independent creators.

To try and address this problem for the past two months I’ve been working on an open-source digital puppetry system called Panda Puppet. When it’s completed, Panda Puppet will be a free plug-in for Blender – the powerful free 3D creation suite – that will enable anyone to create and perform digital puppetry. I’ve just finished working out Panda Puppet’s basic control system and I’ve posted a video of the latest head control system test on YouTube. This is just an early test. and there is a lot of work to be done before Panda Puppet is ready for full fledged filmmaking, but I’m pretty happy with my progress so far.

One of the cool things about Panda Puppet is that you can use almost anything you want to control a digital character – a mouse, joysticks, MIDI keyboards, even the Wiimote (for my tests I am using an off-the-shelf USB joystick). If you can plug it in to a computer, chances are it can be used to perform digtal puppetry with Panda Puppet.

Another neat thing is that because Panda Puppet works with Blender it’s possible to record a performance and then edit it using Blender’s animation tools before rendering out a final movie that looks much better than most of the Machinima being produced today.

I am still writing the code to make that possible, but when I’m done it will be possible to take a real-time performance that looks like this:

Digital mouse rendered in real-time by the Panda Puppet Digital Puppetry System

…and render it out to look like this:

Digital mouse rendered in post by the Panda Puppet Digital Puppetry System

It it’s not quite Pixar, but it’s not bad either.

For more information on Panda Puppet click here for all of the relevant posts on Machin-X.

Oh, and if you’re interested in this, want to get involved and have a background in 3D modeling, texturing and/or rigging characters for animation we should talk.