FLEXPAD:HIGHLY FLEXIBLE BENDING INTERACTIONS

FOR PROJECTED HANDHELD DISPLAYS

JOURNAL:ACM, NEW YORK AUTHORS:JURGEN STEIMLE, ANDREAS JORDT, PATTIE

MAES

Presented byAiswarya Gopal(P2ELT13002)

Athira.L(P2ELT13023)

WHAT IS FLEXPAD…???

Interactive system having a depth camera, projector, plain paper and hand-held displays

ABSTRACT

Track deformed surfaces from depth images

Capture complex deformations and provide fine details

Prevents occlusions from fingers and hands

Avoids use of markers

INTRODUCTION

Manipulation of real world objects for interaction with computer systems Integration of physical and digital

information Depth sensors

Degree o freedom People bend pages in book Bending interaction

CONTD…

Two main contributions Algorithm for capture of complex

deformations Detection of hands and fingers

Evaluation Two evaluation studies

RELATED WORK

Deformation capturing Technologies used▪ Light Space▪ Kinect Fusion▪ Omni touch▪ Proxy particles

Disadvantages

CONTD….

Flexible Display Interfaces Two types of work▪ Deformable sheet or tape▪ Deformation of handheld displays

FLEXPAD OVERVIEW Components used

Kinect camera Projector Sheet of paper Foam or acrylic

Flexible display material Any sheet can be used as passive display Two materials of sheet

FLEXPAD IMPLEMENTATION

Kinect depth sensors Looks at

Removal of hands and fingers Global deformation model How global deformation model

OPTICAL SURFACE MATERIAL ANALYSIS FOR HAND AND FINGER DETECTION

Figure 3: Detection of skin by analyzing the point pattern in the Kinect infrared image. Center: Input. Right: Classification

MODELING THE DEFORMABLE SURFACE

Figure 4: Dimensions of the deformation model (left) and examples of deformations it can express (right).Figure 4: Dimensions of the deformation model (left) and examples of deformations it can express (right).Figure 4: Dimensions of the deformation model (left) and examples of deformations it can express (right).Figure 4: Dimensions of the deformation model (left) and examples of deformations it can express (right).Figure 4: Dimensions of the deformation model (left) and examples of deformations it can express (right).Figure 4: Dimensions of the deformation model (left) and examples of deformations it can express (right).Figure 4: Dimensions of the deformation model (left) and examples of deformations it can express (right).

LIMITATIONS

Trap deformations directly observed directly from Kinect sensors

Trade-off between tracking stability and set of detectable deformations

APPLICATIONS

High flexibility

1:1 mapping

EXPLORING AND ANALYZING VOLUMETRIC DATASETS

Figure 5: Exploring curved cross-sections (a), flattening the view (b), comparing contents across layers (c)Figure 5: Exploring curved cross-sections (a), flattening the view (b), comparing contents across layers (c)Figure 5: Exploring curved cross-sections (a), flattening the view (b), comparing contents across layers (c)Figure 5: Exploring curved cross-sections (a), flattening the view (b), comparing contents across layers (c)

ANIMATING VIRTUAL PAPER CHARACTERS

Figure 6: Animating virtual paper characters

SLICING THROUGH TIME IN VIDEOS

Figure 7: Slicing through time in a video by deformation

EVALUATION

Two evaluations are done Tracking performance User performance of deformation

TRACKING PERFORMANCE

Figure 9: Classification of skin. Top: depth input. Center: infrared input. Bottom: depth image after classification with skin parts removed.

USER PERFORMANCE OF DEFORMATION

RMS90: 2.10 mm (1.1) RMS90: 1.91 mm (1.2) RMS90: 2.67 mm (1.6)RMS150: 2.64 mm (1.3) RMS150: 3.07 mm (1.7) RMS150: 6.1 mm (4.2)

RMS90: 4.58 mm (1.9) RMS90: 4.82 mm (2.2) RMS90: 4.93 mm (2.1)RMS150: 5.45 mm (2.7) RMS150: 5.15 mm (2.5) RMS150: 6.38 mm (3.8)

RMS90: 4.58 mm (1.9)

RMS90: 4.82 mm (2.2)

RMS90: 4.93 mm (2.1)

RMS150: 5.45 mm (2.7)

RMS150: 5.15 mm (2.5)

RMS150: 6.38 mm (3.8)

RESULTS

Figure 11: Average trial completion time in seconds. Error bars show the standard deviation.

CONCLUSIONS

Summary of study findings Highly flexible displays Single and dual deformations obtained

with high precision level of +/-6 degrees High accuracy Average error was below 7mm

CONT…

Touch input on deformable displays Desired touch area is at the center

Active flexible displays Available displays are limited

Smart materials Need for deformable and stretchable

THANK YOU