13
From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan Gao, Xiaomao Wu, Zhihua Chen Department of Computer Science and Engineering Shanghai Jiao Tong University, Shanghai 200030, China [email protected] [email protected] [email protected] [email protected] Abstract. A survey is given for the techniques of human motion synthesis from the viewpoint of human computer interaction. The focus is on an overview based on the interaction of these animation systems between human and computer. To design a good interactive system, we concern the underlying structure of such a system more than the concrete interface that the system will adopt. Therefore, we first analyze the tasks of animators for designing a human motion synthesis system. After reviewing the evolution of the techniques of human motion synthesis, we try to reveal how such evolution improves interaction between human and computer. Finally, we demonstrate our motion synthesis system and give some suggestions for future research directions. 1 INTRODUCTION Many synthetic scenes involve virtual humans, from special effects in the film industry to virtual reality and video games. However, despite great advances in recent years, creating effective tools for synthesis of realistic human motion remains an open problem in computer animation. This is particularly due to two reasons. One is because animating virtual humans range from various motion types such as walking, running and diving etc. Another is because people have proven to be adept at discerning the subtleties of human movement and identifying inaccuracies. An ideal character animation system should be able to synthesize believable motion, and, at the same time, provide sufficient control to the generated motion. “Believable” here has the same meaning proposed by Bates in his term “believable agent” [1] – a believable character is one who seems lifelike, whose actions make sense, who allows you to suspend disbelief. This is not the same thing as realism, which is dominated by physics and biomechanics. In computer animation, believability is generally provided by animator through his fine control to the synthesized motion. So to interact with the virtual human is in a way equal to control it to perform user-defined tasks. Research in three fields is relevant to the problem of animating human motion: robotics, biomechanics, and computer graphics [2]. And there have been tremendous literatures about human motion synthesis. In this paper, we did not want to survey existing motion synthesis methods; our main concern is to give an overall introduction to the evolution of human motion synthesis techniques from the viewpoint of HCI (Human Computer Interaction). According to the definition of SIGCHI [3], Human-computer interaction (HCI, also abbreviated as CHI) is a discipline concerned with the design, evaluation and implementation of interactive computing

From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

  • Upload
    others

  • View
    5

  • Download
    0

Embed Size (px)

Citation preview

Page 1: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

From Keyframing to Motion Capture:

The Evolution of Human Motion Synthesis

Lizhuang Ma,Yan Gao, Xiaomao Wu, Zhihua Chen

Department of Computer Science and Engineering

Shanghai Jiao Tong University, Shanghai 200030, China

[email protected] [email protected]

[email protected] [email protected]

Abstract. A survey is given for the techniques of human motion synthesis from the viewpoint of

human computer interaction. The focus is on an overview based on the interaction of these

animation systems between human and computer. To design a good interactive system, we concern

the underlying structure of such a system more than the concrete interface that the system will

adopt. Therefore, we first analyze the tasks of animators for designing a human motion synthesis

system. After reviewing the evolution of the techniques of human motion synthesis, we try to

reveal how such evolution improves interaction between human and computer. Finally, we

demonstrate our motion synthesis system and give some suggestions for future research directions.

1 INTRODUCTION

Many synthetic scenes involve virtual humans, from special effects in the film industry to virtual reality and video games. However, despite great advances in recent years, creating effective tools for synthesis of realistic human motion remains an open problem in computer animation. This is particularly due to two reasons. One is because animating virtual humans range from various motion types such as walking, running and diving etc. Another is because people have proven to be adept at discerning the subtleties of human movement and identifying inaccuracies.

An ideal character animation system should be able to synthesize believable motion, and, at the same time, provide sufficient control to the generated motion. “Believable” here has the same meaning proposed by Bates in his term “believable agent” [1] – a believable character is one who seems lifelike, whose actions make sense, who allows you to suspend disbelief. This is not the same thing as realism, which is dominated by physics and biomechanics. In computer animation, believability is generally provided by animator through his fine control to the synthesized motion. So to interact with the virtual human is in a way equal to control it to perform user-defined tasks.

Research in three fields is relevant to the problem of animating human motion: robotics, biomechanics, and computer graphics [2]. And there have been tremendous literatures about human motion synthesis. In this paper, we did not want to survey existing motion synthesis methods; our main concern is to give an overall introduction to the evolution of human motion synthesis techniques from the viewpoint of HCI (Human Computer Interaction).

According to the definition of SIGCHI [3], Human-computer interaction (HCI, also abbreviated as CHI) is a discipline concerned with the design, evaluation and implementation of interactive computing

Page 2: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

systems for human use and with the study of major phenomena surrounding them. In a simpler word, HCI is the study of the interaction between people and computers. Its basic goal is to improve interaction between user and computers, by making computers more user-friendly and easier to use.

Nowadays, research in HCI has been spectacularly successful and has fundamentally changed computing. This can be seen from the ubiquitous graphical interface, visual software development toolkits and various computer input devices. In spite of great progress in HCI, there is widespread misconception about HCI, which equal it to interface layout concerning menu arrangement, color selection and so on. In fact, for a good user interface design, there are a lot of thing to do far more than interface layout. As Lewis[4] mentioned, “No matter how well they are crafted, the interface will be a failure if the underlying system doesn't do what the user needs, in a way that the user finds appropriate. In other words, the system has to match the users' tasks.” That is to say, while the interface is important, we must also look beyond it to the underlying structure of the worlds we want to model.

So to design a good animation system for human computer interaction, the most important thing is not to adopt which types of input devices or interfaces but how to transform the interaction information hidden behind the interfaces. In other words, we concern control contents more than control forms. In this paper, we review the evolution of human motion synthesis and try to reveal the tendency of the change of interaction information between animator and computer. Moreover, we evaluate the influence on interface design caused by the different motion synthesis methods.

The remainder of this paper is organized as follows: In section 2 we discuss the general human computer interaction model of human motion synthesis tools and section 3 presents current motion synthesis techniques. Section 4 introduces some new interaction methods applied in motion synthesis field. In section 5, we show the experimental results of our synthesis system. Finally, we conclude with a brief discussion in Section 6.

2 Human Computer Interaction Model

A computer animation system can be regarded as a system that maps input to output. Its output is the result motion clips and its input varies according to system design. The whole interaction process can be shown as follows:

Human

Visual Representation(display)

Input

(Control)Transformation

Machine (computer)

In the above model, system design is an iterative process: after some kinds of transformation, the

user’s input is mapped to visual representation; then input is fine tuned according to the display result. Such process iterates until the satisfactory result is achieved.

Therefore, we conclude that to design a good animation system is somewhat equal to construct a good mapping from input to output. For human motion synthesis, this becomes a more challenging task because human motion involves much more degrees of freedom than the usual computer input devices can provide. Furthermore, we argue that to design a good synthesis, the most important thing is to

Figure 1. Human computer interaction model.

Page 3: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

make a deep understanding to user’s demands. If we did not understand what user’s demands are, we can not design a system that the users desire. As a result, we express the design tasks of a human animation system as “believable motion” and “minimal user assistance” while its interaction tasks can be expressed mainly by “efficiency” and “low input dimensions”.

Here, “believable motion” is explained in the above section while “minimal user assistance” means that system can generate desired motion with animator’s assistance as few as possible. “Efficiency” emphasizes that system can interact with animators in real-time while “low input dimensions” favors a few input parameters to be controlled by user.

3 Evolution of Human Motion Synthesis

In this section, we give an overview of the development of human motion synthesis techniques. In addition, we analyze the advantages and limitations of these techniques.

3.1 Human Model

There are two kinds of human models: geometric model and articulated model. Geometric model use geometric primitives such as polygon or spline surface or solid volume to construct human model while articulated model treats human joints as rigid objects and only concerns joint position and orientation information. Geometric model can provide visual reality, at the cost of many redundant degrees of freedom. Therefore, most human motion synthesis systems prefer articulated model to geometric model. In articulated model, the joints are connected together in a hierarchical, tree-like structure. Figure 2 shows the 18-joints human articulated model that is generally used in human motion synthesis field.

3.2 Basic Human Motion Synthesis methods

The basic approaches to human motion synthesis for articulated figures include: keyframing, procedural methods and motion capture. Though these techniques emerge in sequence, we cannot conclude that one technique excels the others absolutely. Sometimes hybrid systems need to be adopted.

3.2.1 Keyfarming The most common approach used by animators is keyframing. The animators position the body for

important frames (called keyframes). The computer generates the intermediate frames (called the

Figure 2. Human articulated model.

HeadNeck

rShoulder

rWrist lWristHiprHip lHip

rKnee lKnee

lElbow

lShoulder

rElbow

lAnklerAnkle

rCollar lCollar

Page 4: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

inbetweens) by interpolating the values at adjacent keyframes. Keyframing is a natural extension of the most traditional and flexible of animation methods: draw each frame by hand.

Keyframing should make choices with regard to: the control parameters by which a keyframe is specified; the interpolation functions between keyframes. Joint angles or control points of splines are often used as control parameters. The trajectory of an animation parameter is referred to as a motion curve. In addition, timing is another important control parameter, which is typically controlling by doing things such as pushing keyframe markers (control points) along a horizontal axis. On the other hand, cubic splines are commonly used as interpolation functions; however, some systems [5] use interpolation functions that are drawn by animator manually.

The earlier animation systems such as [6][7] are all keyframing systems. In keyframing, animator interactively manipulates low-level motion parameters such as joint angles and coordinates. To create movement, the user configures the rigid body for the articulations at “key” frames and relies on the system to generate intermediate frames. The process of interaction between human and computer for keyfarming systems is shown in figure 3.

User task(Believable motion)

Transformation

Computer

Input

User

Result

Keyframe systems provide users with absolute control over the creative process. However, it is a notoriously time consuming task to synthesize complex movements such as walking and running. The main reason is that a typical articulated figure model usually has at least 50 degrees of freedom. The animator must then painstakingly animate each of these degrees of freedom, one at a time. This intrinsic limitation makes keyframing systems hard to meet “efficiency” and “low input dimensions” demands. So, interaction between human and computer are poor for keyframing systems.

3.2.2 Procedural Methods In order to reduce the burden on the animator, there has been a large amount of research to develop techniques based on procedural animation. Procedural methods explicitly describe how movements are generated, and use computation to propagate changes in high-level parameters to changes in low-level parameters. Therefore, procedural methods typically have a small number of high-level parameters which can be modified to change the generated motion.

Procedural methods can be divided into three parts: kinematic animation, dynamic simulation and spacetime constraints method. The process of interaction between human and computer for procedural animation systems is shown in figure 4.

Figure 3. HCI model for keyframing

Page 5: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

User task(Believable motion)

Transformation

Computer

Input

User

Result

Knowledge

From figure 4, we can see that, relative to keyframing, knowledge is added in the interaction model

of procedural animation systems. Knowledge can be any of biomechanical knowledge, physical laws and so on. Such knowledge can help animators to generate believable motion clips at the cost of the reduction of user controls.

Kinematic Animation. Kinematics studies the geometric properties of the motion of the objects independent from the underlying forces that cause the motion [8]. Few systems used pure kinematic methods because pure kinematics seldom generates believable motion. However, Inverse Kinematics (IK) which aims to control virtual figure’s postures from a set of constraints is a widely used kinematic technique. The problem of inverse kinematics is to find the joint angles, given the end effector position and orientation.

A number of methods can be used to solve the inverse kinematics. Broadly speaking, inverse kinematics algorithms can be characterized as analytical or numerical. For very simple robotic manipulators with few degrees of freedom, analytical (or closed-form) solutions can be found by direct resolution of the non-linear equations [9]. Korein [10] provided an interesting analytic solution for a 7-DOF arm that deals with joint limits. XiaoMao Wu[11] extended his work to 12-DOF human limbs.

In contrast to the analytical methods, numerical approaches iteratively converge to a single solution based on an initial guess. Whitney [12] introduced the resolved motion-rate control, which used a generalized inverse to solve a Jacobian matrix. Zhao and Badler [13] formulated the inverse kinematics problem of a human figure as a constrained optimization problem. Grochow[14] proposed style-based ik based on a learned model of human poses.

IK reduces the animator’s tedium by providing means for pose creation without the explicit setting of joint angles. IK seems to be a perfect interactive method because it meets the “efficiency” and “low input dimensions” demands quite well. However, IK methods can not create continuous motions but discrete keyframe. The underlying method for generating motion still relies on strictly kinematic methods. Unfortunately, kinematic methods do not produce convincing movement without a considerable amount of effort on the animator’s part. In fact, IK are usually treated as a useful component of other motion synthesis tools.

Dynamic Simulation. Dynamic simulation generates character motion by applying control to physically based models and using forward integration. Simulation ensures physically plausible motion by enforcing the laws of physics. However, Forward dynamics approach is difficult to create motions with some desired goals because animator has no control over the motion once the initial conditions are specified. Moreover, it is almost an impossible task for animator to specify all the forces and torques as inputs to initiate and guide a motion.

Figure 4. HCI model for physical simulation

Page 6: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

One approach to gain control over a dynamic system is to design controllers. A controller is a specification of forces or torques as functions of the positions and velocities (called the state) of a figure. Given a controller and the initial state, the motion can be computed by integrating over time. The design of controller benefits much from biomechanics which provided a useful source of predictive models for specific motions. As a result, specific controller can only perform specific task. We illustrate the HCI model of dynamic simulation systems in figure 5.

User task(Believable motion)

Supervisor

Controller 1 Controller n...Computer

command, parameters(walking,running etc.)

walking ruuning

User

Abundant studies have been done in dynamic simulation. Among them, Hodgins et al. [15]

handcrafted controllers based on finite state machines and proportional-derivative servos to produce running, bicycling, and vaulting motions. Liu and Popovic [16] demonstrated that motions dominated by dynamics, such as leaping or flipping, can be synthesized almost entirely from conservation laws, without computing joint torques explicitly. Genetic programming has also been successfully used to automatically generate dynamic controllers for some articulated figures [17]. Faloutsos[18] proposed a framework for composing controllers in order to enhance the motor abilities.

Dynamic simulation can generate physical plausible motions with few high-level input parameters. However, physical plausibility doesn’t mean believability. Dynamic simulation often produces motion that lacks important features of natural human motion. Moreover, it remains an open problem that how to design complex composite controllers capable of synthesizing a full spectrum of human-like motor behaviors. Dynamic simulation can meet the “efficiency” and “low input dimensions” demands, which makes it suitable for interaction. However, dynamic simulation fails in the “believable motion” and “minimal user assistance” demands, which makes it unsuitable for human animation system.

Spacetime Constraints Method. Spacetime constraints method [19], proposed by Witkin and Kass, provides both realism and control of character motion. In the spacetime framework the user first specifies both pose constrains and dynamic constraints. In addition to these constraints, the user also specifies an objective function. The algorithm solves this constrained optimization problem to find the motion trajectories which minimize the objective function while at the same time satisfying the constraints.

Most spacetime constrains methods use minimizing the total power consumption of all of the character’s muscles as their objective functions. For example, Cohen [20] developed an interactive spacetime constraints system using a hybrid symbolic and numerical processing techniques. Liu [21] improved the performance by representing the trajectories of the generalized degrees of freedom with a hierarchical wavelet basis.

However, current algorithms do not scale up to the complexity of characters one would like to animate. The time complexity of the spacetime formulation remains a huge impediment for its

Figure 5. HCI model for dynamic simulation.

Page 7: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

application to complex characters. Moreover, spacetime method is extremely sensitive to the starting position of the optimization process: if optimization starts far from the solution, the optimization method may converge to local minima. In consequence, spacetime constraints method fails in the “efficiency” demand and it is hard to meet the “believability” task.

3.2.3 Motion Capture All the previous approaches deal with biomechanical, mechanical or empirical knowledge on human motion. On the contrary, motion capture method works directly on data which are recorded by motion capture system. Motion capture involves measuring an object’s position and orientation in physical space. This is achieved by using sensors placed at strategic points on the body. These points are usually joints, or specific places of interest. In general, the term motion capture refers to any method for obtaining data that describes the motion of a human or animal.

Though motion capture can provide all the detail and nuance of live character motion, the process of acquiring motion data is labor intensive and costly. Moreover, the captured data are closely related to the specific capture environment so that they cannot be directly used in new environments. In order to make motion capture widely available, it is desirable to develop tools that make it easy to reuse and adapt existing motion data. The model of interaction between human and computer for motion capture techniques is shown in figure 6.

User task(Believable motion)

Transformation

Computer

Input

User

Result

examplemotion data

There have been an abundance of research results to develop motion editing tools to modify motion

data. Bruderlin and Williams [22] applied a number of different signal processing techniques to motion data to allow editing. Witkin and Popovi´c [23] presented a motion warping technique for the same purpose. Unuma et al. [24] used Fourier principle to interpolate and extrapolate motion data in the frequency domain. Guo and Robergé [25] and Wiley and Hahn [26] applied spatial domain techniques to linearly interpolate a set of example motions. Lee and Shin [27] provided an interactive multiresolution motion editor for fast, fine-scale control of the motion. Rose et al. [28] adopted a multidimensional interpolation technique to blend multiple motions all together.

More recently, there have been many projects aimed toward allowing an animator to create new animations based on motion capture data. Brand and Hertzmann [29] introduced ‘Style Machines’, which is a statistical model that can generate new motions with different styles. Li et al. [30] divided the data into motion textons and then synthesized new motion by switching from one texton to the next. Kovar et al. [31] presented Motion Graphs which used to organized motion data. New motion sequences can be synthesized through traversing the graph. Arikan and Forsyth [32] organized motion data as a hierarchy graphs and generated human motions by cutting and pasting operation.

Figure 6. HCI model for motion capture.

Page 8: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

Motion capture is the most promising technique for human motion synthesis. It can satisfy the “believable motion” and “minimal user assistance” demands which make it competent for realistic motion synthesis task. However, there exist some limitations. If motion data are modified beyond allowed “region”, the result motion is unacceptable. On the other hand, motion capture fails to satisfy the “low input dimensions” demand which makes its interaction poor.

3.3 Hybrid methods

The above three techniques (keyframing, procedural approaches and motion capture) are often combined to produce systems with their respective advantages. For example, motion capture data can be used a source of information in the construction of a simulation. Short simulations can be used to blend between motion capture segments. Keyframing techniques can be applied to motion that was generated either procedurally or with motion capture. Here, we give some examples of these hybrid systems.

Keyframing can be combined with motion capture. Many motion editing tools such as motion warping [22], motion signal process [23] can also be treated as keyframing systems. The only difference between them is their inputs: motion editing tools use motion data while keyframing systems sketch from blank. Pullen and Bregler[33] allow an animator to keyframe motion for a subset of degrees of freedom of the character and use a motion capture library to synthesize motion for the missing degrees of freedom and add texture to those that were keyframed. His method takes advantage of the benefits of motion capture approach without sacrificing the control of keyframing method.

The hybrid systems of procedural methods and motion capture are used popularly in human motion synthesis. For example, spacetime constraints method often uses motion data as initial state to ensure the convergence to a plausible solution. Among them, Gleicher [34] retargeted the motion to a target character which has the same skeleton structure but different skeleton lengths with a source character. Rose [35] used metabolic energy which is approximated by torques over time as objective to create motion which ties in two separate motion data sequences. Popovic [36] reduced the complexity of the spacetime optimization problems by the approach of motion transformation, which combines the expressive richness of the input animation sequence and the controllability of spacetime optimization to synthesize a wide range of realistic character motions. Dynamic simulation also uses motion data in order to enhance believability over the result motion. Zordan et al. [37] demonstrated the power of such hybrid approach with characters that hit and react under a variety of conditions including simulations that play table tennis and box. Shapiro and Pighin[38] implemented a framework that is capable of combining dynamic simulation and kinematic animation techniques (including motion capture based techniques) in a unified fashion.

Hybrid methods integrate the advantages of different motion synthesis approaches. In these methods, motion capture approaches play a center role by introducing motion details and personality to the synthesized motions while dynamic simulation provides high-level control for interaction.

4 New Interaction Techniques in Human Motion Synthesis

As the progress of HCI, many interaction means, other than conventional mouse-based techniques (menus, scroll bars, buttons, etc.), have been developed to improve the interaction between human and computer. These new interfaces include sketching, script, gesture, speech and video etc. They help to

Page 9: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

overcome the “curse of dimensions” problem: the character’s motion is high dimensional and most of the available input devices are not. In this section, we investigate the application of these interfaces in human motion synthesis.

4.1 Script Based Systems

Script language, which aims to provide high-level control, is often used as a good interactive means between human and computer. For example, many commercially available animation systems such as Jack[39] or LifeForms[40] incorporated the specification of low level motion commands, even though these commands do not extend easily to controlling the motion of articulated figures. Perlin’s improv system [41] integrated procedural methods with scripting language; therefore, it provides tool to create actors that respond to users in real−time, with personalities and moods consistent with the author’s goals and intentions. Arikan [42] proposed a framework that allows a user to synthesize human motion while retaining control of its qualitative properties. In his system, the user paints a timeline with annotations: like walk, run or jump. The system then assembles frames automatically from a motion database so that the final motion performs the specified actions at specified times.

4.2 Sketch Based Systems

Sketching provides an intuitive way for users to express themselves between human and computer. It is especially effective for novice users. Terra [43] presented a novel approach for generating timing information for keyframed motion. His approach allows the user to sketch the timing information using a mouse. The user’s input is analyzed and features of the use’s input are mapped to features of the keyframed motion. The keyframes are then distributed in time according to the timing of the user's input path. Thorne and his colleagues [44] presented a sketch-based technique for creating character motion. In this system, the user not only sketches the character to be animated, but also sketches motion curves that are segmented and mapped to a parameterized set of output motion primitives.

4.3 Foot-Ground Force Based Systems

Some researchers used foot-ground contact information as input for motion synthesis. For example, Yin and Pai[45] presented a FootSee System which used a foot pressure sensor pad to interactively control avatars for video games, virtual reality, and animation. During the training phase, his system capture full body motions with a motion capture system, as well as the corresponding foot-ground pressure distributions with a pressure sensor pad, into a database. At run time, animation is created by searching the database for the measured foot-ground interactions.

4.4 Gesture Based Systems

Gestures are a promising approach to human computer interaction because they often allow several parameters to be controlled simultaneously in an intuitive fashion. Zhao [46] developed a framework for the procedural generation of expressive and natural-looking gestures for computerized communicating agents. In his system, input gestures are parameterized based on the theory of Laban

Page 10: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

Movement Analysis (LMA) and its Efforts and Shape components. New gestures are then synthesized based on predefined key pose and time information plus Effort and Shape qualities.

4.5 Video Based Systems

Vision-based interface, due to the higher dimensional nature of the input, gives the most control over the details of the avatar’s motion. Lee [47] constructed a graph and generated motion via three user interfaces: a list of choices, a sketch-based interface, and a live video feed. In his vision-based interface, the user acts out the desired motion in front of a camera. Visual features are extracted from video and used to determine avatar motion.

5 Our Experiment Results

We have developed a human motion synthesis platform by combining IK with motion capture techniques. The user modifies end-effector’s positions of the character through mouse. The system computes the other degrees of freedom of character automatically and generates whole motion through interpolation. Our graphic user interface is very friendly and convenient for manipulation. The results are shown in figure 7 and figure 8.

Figure 7. (a) Original motion: walk and look behind midway. (b) The left motion is edited to look behind and wave midway.

Figure 8. (a) Original motion: jump-kick motion. (b) The left motion is edited to kick higher.

6 Conclusions

Creating realistic human motion by computer has long been a challenging task and needs animator’s tedious assistant work. To reduce animator’s task, many motion synthesis techniques have been developed during the past decades. In this paper, instead of introducing these techniques, we focus on

(a) (b)

(a) (b)

Page 11: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

analyzing interaction between human and computer of these animation systems. Therefore our paper acts a bridge between computer animation and human computer interaction.

The interaction of motion synthesis tools suffers from the problem of “curse of dimensions” –the dimensions of human’s degrees of freedom are high while input devices have low dimensions. To state how this problem influence the system’s design; we review the history of human motion synthesis techniques and give a comparison to three main techniques: keyframing, procedural methods and motion capture techniques. Keyframing systems can generate believable motion but depend fully on animator’s experience. Procedural systems are particularly suitable to synthesize some types of motion by giving a few high-level parameters; however, the synthesized motion lacks believability and designing controllers for various motion types remains an unsolved problem. Motion capture technique has been regarded as the most promising technique among them, which can generate believable motion efficiently. But it is still a difficult task for motion capture systems to provide high-level control.

New interfaces also help to alleviate the “curse of dimension” problem greatly. These new interfaces such as video-based interface, gesture-based interface etc. can provide more useful information to user than the usual mouse and keyboard interface. We believe that by using these new interfaces and hybrid synthesis methods, new interaction tools, which need user provide his desired high-level tasks, can be developed.

REFERENCES

[1] J. Bates. The role of emotion in believable agents. Communications of the ACM, 37(7):122-125, 1994.

[2] J. K. Hodgins, W. L. Wooten, D. C. Brogan, and J. F. O’Brien. Animating human athletics. In Proceedings of

SIGGRAPH 95, Computer Graphics Proceedings, Annual Conference Series, pp.71-78, Aug 1995. [3] Hewett, Baecker, Card, Carey, Gasen, Mantei, Perlman, Strong and Verplank. ACM SIGCHI Curricula for

Human-Computer Interaction. available at http://sigchi.org/cdg/cdg2.html

[4] C. Lewis, and J. Rieman. Task-Centered user interface design(online book). available at http://hcibib.org/tcuid

[5] G. Stern. Bbop: a program for 3-dimensional animation. Nicograph’83 proceedings (Tokyo, Japan),

pp.403-404, December 1998.

[6] J. Gomez. TWIXT: A 3D animation system. Computers and Graphics, 9, Pergamon Press Ltd., pp.291- 298,

1985. Originally in Proceedings of Eurographics‘84.

[7] D. Sturman. Interactive keyframe animation of 3D articulated models. Course notes, SIGGRAPH Course

Number 10, Computer Animation: 3D Motion Specification and Control, pp.17-25, July 1985.

[8] A. S. Abalı, Animation of human motion with inverse kinematics using nonlinear programming, Master Thesis,

Bilkent, 2001

[9] R. Paul. Robot manipulators:mathematics, programming and control. MIT Press, 1981. [10] J. U. Korein. A geometric investigation of reach. MIT Press, Cambridge, 1985.

[11] Xiaomao Wu, Lizhuang Ma, Zhihua Chen, Yan Gao. A 12-DOF Analytic Inverse Kinematics Solver for

Human Motion Control. Journal of Information & Computational Science, 1(1):137-141, Sep 2004.

[12] D.E. Whitney. Resolved motion rate control of manipulators and human prostheses. IEEE Trans. on

Man-Machine Systems, Vol. MMS-10, No.2, pp.47-53, 1969

[13] J. Zhao and N. I. Badler. Inverse kinematics positioning using nonlinear programming for highly articulated

figures. ACM Transactions on Graphics, 13(4):313-336, 1994. [14] K. Grochow, S. L. Martin, A. Hertzmann, and Z. popvic. Style-based Inverse Kinematics, ACM Transactions

on Graphics 23(3), July 2004

Page 12: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

[15] J. Hodgins, W.Wooten, D. Brogan, and J. O’Brien. Animating human athletics. In Proceedings of ACM

SIGGRAPH 95, Annual Conference Series, pp.71-78. ACM SIGGRAPH, Aug. 1995. [16] C. K. Liu and Z. Popovi´c. Synthesis of complex dynamic character motion from simple animations. In

Proceedings of ACM SIGGRAPH 2002, Annual Conference Series, ACM SIGGRAPH, July 2002. [17] Larry Gritz and James Hahn. Genetic programming for articulated figure motion. In The Journal of

Visualization and Computer Animation, Vol. 6, pp.129-142, 1995.

[18] P. Faloutsos, M. van de Panne, and D. Terzopoulos. Composable controllers for physics-based character

animation. In Proceedings of SIGGRAPH 2001, ACM SIGGRAPH, pp. 251-260, 2001. [19] A. Witkin and M. Kass. Spacetime constraints. In Computer Graphics (SIGGRAPH 88 Proceedings), vol. 22,

pp.159-168, August 1988.

[20] M. F. Cohen. Interactive spacetime control for animation. In Computer Graphics (SIGGRAPH 92

Proceedings), vol. 26, pp.293-302, July 1992.

[21] Zicheng Liu, S. J. Gortler, and M. F. Cohen. Hierarchical Spacetime Control. In Computer Graphics

(SIGGRAPH 94 Proceedings), pp.25, July 1994.

[22] A. Witkin and Z. Popovic. Motion warping. Computer Graphics (Proceedings of SIGGRAPH 95),

29:105-108, August 1995. [23] A. Bruderlin and L. Williams. Motion signal processing. Computer Graphics (Proceedings of SIGGRAPH

95), pp.97-104, 1995. [24] M. Unuma, K. Anjyo, and R. Takeuchi. Fourier principles for emotion-based human figure animation.

Computer Graphics (Proceedings of SIGGRAPH 95), 29:91-96, August 1995. [25] S. Guo and J. Roberg´e. A high-level control mechanism for human locomotion based on parametric frame

space interpolation. In Proceedings of Computer Animation and Simulation ’96, Eurographics Animation

Workshop, pp.95-107, Springer-Verlag, 1996. [26] D. J. Wiley and J. K. Hahn. Interpolation synthesis for articulated figure motion. In Proceedings of IEEE

Virtual Reality Annual International Symposium ’97, pp. 157-160.IEEE Computer Society Press, 1997.

[27] J. Lee and S. Y. Shin. A hierarchical approach to interactive motion editing for human-like figures. In

Proceedings of ACM SIGGRAPH 99, Annual Conference Series, pp. 39–48. ACM SIGGRAPH, August 1999.

[28] C. Rose, M. F. Cohen, and B. Bodenheimer. Verbs and Adverbs: Multidimensional motion interpolation.

IEEE Computer Graphics and Application, 18(5):32-40, October 1998.

[29] M. Brand, and A. Hertzmann. Style machines. In Proceedings of ACM SIGGRAPH 2000, Annual Conference

Series, ACM SIGGRAPH, pp.183-192, 2000.

[30] Y. Li, T. Wang, and H. Shum. 2002. Motion texture: A two-level statistical model for character motion

synthesis. In Proceedings of ACM SIGGRAPH 2002, Annual Conference Series, ACM SIGGRAPH, 2002.

[31] L. Kovar, M. Gleicher, and F. Pighin. Motion graphs. ACM Transactions on Graphics, 21(3):473-482, 2002.

[32] O. Arikan, and D. A. Forsythe. Interactive motion generation from examples. ACM Transactions on

Graphics, 21( 3): 483-490, 2002.

[33] K. Pullen, and C. Bregler. Motion capture assisted animation: Texturing and synthesis. In Proceedings of

ACM SIGGRAPH 2002, Annual Conference Series, ACM SIGGRAPH, 2002.

[34] M. Gleicher. Retargetting Motion to New Characters. In Proceedings of SIGGRAPH 1998, pp.33-42, Aug.

1998.

[35] C. Rose, B. Guenter, B. Bodenheimer, and M. Cohen. Efficient generation of motion transitions using

spacetime constraints. In Computer Graphics (Proceedings of SI GGRAPH96), pp.147-154, 1996.

[36] Z. Popovic, and A. Witkin. Physically Based Motion Transformation. In Proceedings of SIGGRAPH 1999,

pp.11-20, Aug. 1999.

Page 13: From keyframing to motion capture the evolution of …ruth/elvis/data/documents/Lizhuang Ma...From Keyframing to Motion Capture: The Evolution of Human Motion Synthesis Lizhuang Ma,Yan

[37] V. B. Zordan and J. K. Hodgins. Motion capture-driven simulations that hit and react. In ACM SIGGRAPH

Symposium on Computer Animation, pp. 89–96, July 2002

[38] A. Shapiro, F. Pighin, and P. Faloutsos. Hybrid control for interactive character animation. Pacific

Graphics’2003, short paper, pp. 455-461, 2003.

[39] N. Badler, C. Phillips, and B. Webber. Simulating Humans: Computer Graphics, Animation, and Control.

Oxford University Press, 1993.

[40] A. Bruderlin, T. Schiphorst, and T. Calvert. The animation of human movement: How visual can we get? In

Computer Graphics (SIGGRAPH’90), in Course Notes 8 Human Figure Animation: Approaches and

Applications, pp.245-260, August 1990.

[41] K. Perlin, and A. Goldberg. Improv: a System for scripting interactive actors in virtual worlds. Proceedings of

Siggraph 96, ACM Press, NewYork, pp. 205-216, 1996.

[42] O., Arikan, D. A. Forsyth, and J. F. O’Brien. Motion synthesis from annotations. ACM Transactions on

Graphics, 22(3):402-408, 2003.

[43] S.C.L. Terra, and R.A. Metoyer. Performance timing for keyframe animation. ACM SIGGRAPH/

Eurographics Symposium on Computer Animation, 2004.

[44] M. Thorne, D. Burke, V. D. Panne. Motion doodles: An interface for sketching character motion. Proceedings

of SIGGRAPH 2004, pp. 424-431, 2004.

[45] K. Yin and D. K. Pai, FootSee: an Interactive Animation System, in Proceedings of the Eurographics

/SIGGRAPH Symposium on Computer Animation, San Diego, July, 2003

[46] L. Zhao. Synthesis and acquisition of Laban Movement Analysis qualitative parameters for communicative

gestures. PHD thesis, University of Pennsylvania, 2002.

[47] J. Lee, J. Chai, P. S. A. Reitsma, J. K. Hodgins, and N. S. Pollard. Interactive control of avatars animated with

human motion data. In ACM Transactions on Graphics, 21(3):491-500, July 2002.