Summary Legend and children’s entertainment innovator Jim Henson may have passed away in 1990, but his work and his vision have lived on. The Jim Henson Company and Jim Henson’s Creature Shop are now run by Henson’s children, Lisa and Brian, who have taken their father’s ideas in directions that would have surely made the dearly departed genius very proud. Now, the company behind The Muppets, Bear in the Big Blue House, and Fraggle Rock, to name only a few, has created Sid the Science Kid, an animated show for preschoolers that follows the explorations of a curious child scientist unraveling the secrets behind such pressing questions as why bananas get ‘brown and mushy’ and exactly how those juice box straws work.

Encouraging curiosity and promoting science-readiness, Sid the Science Kid is the first complete series to be created using the Henson Digital Puppetry Studio. The Creature Shop used the proprietary system – which makes ample use of Autodesk® Maya® software and mental ray® rendering software – to help capture 40 episodes worth of digitally puppeteered performances on a virtual stage and in real time. Sid the Science Kid debuted nationally on PBSKids in September 2008.

The Challenge What exactly happens when a banana goes ‘bad’? If you don’t know, then an enthusiastic, inquisitive young boy is ready to tell you. In fact, Sid – the animated star of Jim Henson’s Sid the Science Kid – and his friends are willing and able to answer many other important science questions, all while singing, dancing, and having an all-around good time. Almost as interesting as Sid’s discoveries, however, are the forward-thinking techniques used by Jim Henson’s Creature Shop to achieve the show’s unique look-and-feel.

“People all too often think of Jim Henson as simply a puppeteer,” says Nicole Goldman, VP of Publicity at The Jim Henson Company. “He was a master puppeteer, of course, but he was also an innovator. Even before his tragic death in 1990, Jim was working on something we now call ‘digital puppetry,’ in which digital technology enables a real time combination of physical puppetry and computer-generated animation. Since then, Jim’s son, Brian has expanded that idea and taken it in directions even his imaginative father could not have imagined.”

Image courtesy of Jim Henson Company

The Jim Henson Company (www.henson.com)

Project: Sid the Science Kid (pbskids.org/sid) Autodesk® Maya® software

All 40 of our artists use Maya, and we were pushing the system to the absolute limit on Sid the Science Kid. —Steffen Wild Visual Effects Supervisor The Jim Henson Company

When Bananas Go Bad. The Jim Henson Company Uses Groundbreaking ‘Digital Puppetry’ and Autodesk Maya for Sid the Science Kid.

Image courtesy of Jim Henson Company

Indeed, the Henson Digital Puppetry Studio has been up and running since the year 2000, and Sid the Science Kid is the most complete realization of the departed founder’s idea. Simply put, the digital puppetry process combines advanced motion capture techniques that capture the performer’s body movements, with synchronized facial puppetry and voices of puppeteers, and 3D virtual environments and effects into a single, real-time scene.

“This is a new and interesting way to work,” admits Steffen Wild, Visual Effects Supervisor. “Not only did we have to create believable and instructive 3D representations of the characters and scientific concepts, we had to figure how to get it done in a fully real-time environment. We had no less than nine virtual characters occupying eleven key environments, and each of them had to believably interact with real-world props such as ropes and pulleys. In addition to all of that, everything was set up on a multitude of levels, from motion capture to animation clean-up to final renders. All 40 of our artists use Maya, and we were pushing the system and all of our resources to the absolute limit.”

The Solution Although the Henson team began working on Sid the Science Kid a little over a year ago, the Digital Puppetry Studio has been in the works for nearly nine years. Some of that time, the forward-looking minds at the Creature Shop have been waiting for computer power to catch up to their ideas.

“We’re essentially taking our control system from driving animatronics to driving CG characters in real time,” explains Jeff Christie, CG Supervisor. “From the beginning of that transformation, Maya has been our software of choice, largely because of the software’s immense user base.”

On Sid the Science Kid, however, it was the Maya software’s scripting capabilities that proved most empowering, according to Christie.

“Only Maya enables us to set up such incredibly elaborate animations and make it repeatable,” he says. “The Maya software’s exceptionally powerful MEL and Python scripting has saved us a lot of time and money. Over the years, we’ve used Maya to create a system of standard facial animation and motion capture setups. On Sid, we started with 2D drawings of concepts that were then transformed into maquette sculptures. The maquettes were then scanned and brought into Maya so we could use them as a baseline to build meshes. We could then rig the mesh setup and transfer it to different character designs. That way, we could maintain the same vertex count and positioning when transferring the rigs between setups.”

In the end, all nine virtual characters required three rigs each: a real time rig for performing on stage; a clean-up rig for post production; and a render rig with final, baked baseline animation.

“The ability to use our database together with Maya referencing enabled us to be very nimble during our quest to complete 40 episodes in this cohesive fashion,” says Christie.

Only Maya enables us to set up such incredibly elaborate animations and make them repeatable. The Maya software’s exceptionally powerful MEL and Python scripting has saved us a great deal of time and money. —Jeff Christie, CG Supervisor, Jim Henson’s Creature Shop

The Results Brian Henson is now co-CEO (with his sister, Lisa) of The Jim Henson Company, and what was his father’s vision for digital puppetry has been transformed into his own unique version. From the look of Sid the Science Kid, it seems unlikely the studio would stop using the revolutionary way of working in future. In turn, their innovative use of Autodesk Maya seems destined to continue.

“We’re using Maya in a truly real-time production setup, following a live action paradigm,” says Wild. “That’s definitely new. Characters are rigged, modeled, animated, etc, all while we keep a very close eye on system performance. Character animation becomes a process in which the director can truly interact with performers, floor managers, script supervisors, and more, just the way he or she would in a live action sitcom. The big difference is that all of the data is assembled in a Maya scene file. Approximately 4000 frames may be required for a single take, and we have 4 real time cameras working at once. All the frames are rendered, almost like dailies, and sent to editorial. When the edited episode comes to post production, the team of artists can clean up and color correct according to specific frame ranges. Final renders are done in mental ray. We render one color pass without any shadows, then a separate occlusion pass containing all shading, and a matte pass that allows us to separate our characters from the backgrounds to make them pop more and desaturate our backgrounds.”

And, if you’re still unclear on the process itself, Sid the Science Kid himself can take you through it all at www.creatureshop.com/sid.php.

Autodesk and Maya are registered trademarks or trademarks of Autodesk, Inc., and/or its subsidiaries and/or affiliates in the USA and/or other countries. mental ray is a registered trademark of mental images GmbH licensed for use by Autodesk, Inc. All other brand names, product names, or trademarks belong to their respective holders. © 2008 Autodesk, Inc. All rights reserved.