1
AR Storyboard: An Augmented Reality based Interactive
Storyboard Authoring Tool
Midieum Shin, Byung-soo Kim, Jun Park
Department of Computer Science, Hongik University, Seoul, Korea, [email protected]
Abstract
In early stages of production, storyboards are used for
visually describing the story and the script. In this paper,
an Augmented Reality based storyboard-authoring tool is
introduced. Proposed tool is easy-to-use, and provides
intuitive interface for scene composition and camera
pose/motion control. Using AR Storyboard, non-
experienced users may compose 3D scenes for a
storyboard using interfaces in his/her real environments.
1. Introduction
In early stages of production, storyboards are used to
translate the story and the script into images, and organize
scenes [1]. They focus on the storytelling, general
composition, actions, and camera motions rather than
technical details [2]. As computer animation gains
popularity, digital storyboard tools became available [3,4].
However, most of these tools are primarily drawing and
editing tools with graphical interfaces. In many scene
images, placement of items (characters, stage properties,
and the camera) is one of the most important factors for
efficient storyboard production. Functionalities and
interfaces for scene composition and camera motion
control are either lacking or insufficient in these tools.
The goal of this research is to develop a storyboard-
authoring tool that provides intuitive interface for scene
composition and camera pose/motion control. The
proposed interactive storyboard-authoring tool, AR
Storyboard, is based on marker-based Augmented Reality
technologies. Using AR Storyboard, non-experienced
users may compose 3D scenes for a storyboard using
interfaces in his/her real environments.
2. AR Storyboard System
AR Storyboard is composed of a computing system, a PC
camera, and a set of “Item blocks” (Fig. 1). An “Item
block” represents a character or other stage object. As
item blocks are placed within the camera view, the
composed scene of corresponding 3D models is rendered
in Augmented Reality view. The author, then, may
capture, store, and edit the scene images to create a
storyboard. As user’s desktop space replaces the stage,
and item blocks replace characters and stage properties,
AR Storyboard may be used to create storyboards in a
simulated screen-filming environment.
Figure 1. Concept
of AR Storyboard
Figure 2. Examples of Item Blocks
2.1. Item Blocks
An “Item block” is a thick paper block, on which visual
information is printed. We defined five types of item
blocks (Fig. 2): characters, backgrounds, stage properties,
actions, and facial expressions. These blocks are
categorized into two major groups: static blocks
(characters, backgrounds, and stage properties) and
dynamic blocks (actions and facial expressions). While
static blocks are used for placing characters, buildings,
and properties on the stage, dynamic blocks provide
additional animation information for the characters. 3D
models and animations for the items were created using
“Cosmo World”[5] and stored in “wrl” format.
To reduce the number of blocks, we allowed for multiple
choices of items in a single block. For example, from a
character block, a male, a female, an aged man, or a baby
character may be selected. Our novel magnetic selector
(in a circular shape) was used for convenient selection.
As the magnetic selector was placed at a proper position
Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR’05)
0-7695-2459-1/05 $20.00 © 2005 IEEE
2
on the item block (magnets are also installed under the
blocks in order to guide selector positioning), the
corresponding item is recognized and rendered by the
system.
Currently, we have prepared eleven item blocks: 2
character blocks, 2 action blocks, 2 facial expression
blocks, 1 background block, and 4 stage property blocks.
Because one block provides four multiple choices, total
44 items are available: as an example, there are 2 sets of 4
characters.
It is difficult to predict practically how many item blocks
are required for general-purpose storyboard creation.
However, we may compare with existing 2D-storyboard
authoring tools. For example, “Storyboard Quick” [6]
provides 5 characters, 5 actions, 64 stage properties, and
52 backgrounds: no facial expression is available. Seeing
that a commercial storyboard tool does not provide a wide
variety of items, proposed AR Storyboard may be
practically useful if tens of items are added.
2.2. Scene Composition A scene is composed, first, by preparing item blocks for
the scene components. After scene components are
determined, corresponding item blocks are placed on
user’s desktop within the camera view. The position and
orientation of the static blocks determine the pose of the
corresponding 3D models in the scene. However,
dynamic blocks can be placed anywhere within the
camera-viewable region. Several combinations of item
blocks may compose complex scene and behavior: for
example, combination of an aged man character, a
running action, and smiling expression compose a smiling
and running aged man (Fig. 3). Action blocks and
expression blocks were prepared separately for each
character: there are 64 combinations of character-action-
expression.
Figure 3. Example of a character animation composition: a
smiling and running aged man
As an example, we composed a scene from the film
“Matrix”. We prepared and placed item blocks for a
building roof, a male character, a bullet-avoiding action,
and a bullet item (Fig. 4-left). Composed animation is
rendered repeatedly in the user’s AR view. The user may
capture desired still images, simultaneously controlling
the camera pose (Fig. 4-right).
Figure 4. Scene composition (a scene from “Matrix”): item
block placement (left) and composed scene (right)
2.3. Camera Control For calculating position and orientation of static blocks,
ARToolKit library was used [7]. As transformation
between the camera and the static blocks can be
computed using AR technologies, the virtual camera for
scene images could be controlled using the PC camera.
For smooth transition and fixation, camera was installed
at the tip of a swing-arm lamp. Using AR Storyboard,
users may control the camera pose/motion simulating
screen filming process.
3. Conclusions
AR-based interactive storyboard-authoring tool was
introduced. Proposed tool provides interface for scene
composition and camera pose/motion control. Character
posture and stage properties could be composed in the
scene image. By simulating virtual screen filming
environment, AR Storyboard provides an easy-to-use and
intuitive interface. ARToolKit tracking range is limited.
To allow freedom of camera motion, advanced tracking
technology is required.
4. References
[1] John Hart, "The Art of Storyboard: Storyboarding for
Film, TV, and Animation", Focal Press, 1999
[2] Isaac Victor Kerlow, The Art of 3-D Computer
Animation and Imaging, 2000, Wiley
[3] J.A. Landay and B.A. Myers, "Sketching interfaces:
toward more human interface design", IEEE Computer,
Volume 34, Issue 3, March 2001 Page(s): 56-64
[4] http://www.sharewareconnection.com/storyboard-
tools.htm
[5] http://www.sgi.com/products/software/cosmo/worlds.
html
[6] http://www.powerproduction.com/quick.html
[7] http://www.hitl.washington.edu/artoolkit/
Proceedings of the International Symposium on Mixed and Augmented Reality (ISMAR’05)
0-7695-2459-1/05 $20.00 © 2005 IEEE