eurovis10-hoeferlin

Eurographics/ IEEE-VGTC Symposium on Visualization 2010G. Melançon, T. Munzner, and D. Weiskopf(Guest Editors)

Volume 29 (2010), Number 3

Video Visualization for Snooker Skill Training

M. Höferlin1, E. Grundy2, R. Borgo2, D. Weiskopf1, M. Chen2, I. W. Griffiths2 and W. Griffiths3

1Universität Stuttgart 2Swansea University 3Terry Griffiths Matchroom

Abstract

We present a feasibility study on using video visualization to aid snooker skill training. By involving the coaches

and players in the loop of intelligent reasoning, our approach addresses the difficulties of automated semantic

reasoning, while benefiting from mature video processing techniques. This work was conducted in conjunction

with a snooker club and a sports scientist. In particular, we utilized the principal design of the VideoPerpetuoGram

(VPG) to convey spatiotemporal information to the viewers through static visualization, removing the burden

of repeated video viewing. We extended the VPG design to accommodate the need for depicting multiple video

streams and respective temporal attribute fields, including silhouette extrusion, spatial attributes, and non-spatial

attributes. Our results and evaluation have shown that video visualization can provide snooker coaching with

visually quantifiable and comparable summary records, and is thus a cost-effective means for assessing skill

levels and monitoring progress objectively and consistently.

Categories and Subject Descriptors (according to ACM CCS): I.3.3 [Computer Graphics]: Picture/ImageGeneration—Display algorithms I.3.m [Computer Graphics]: Miscellaneous—Video visualization

1. Introduction

Cue sports encompass a family of skill-based games wherea player uses a wooden cue to strike billiard balls on a table.Snooker is one such sport, which has been popular in manyEnglish-speaking countries since the 19th century, and in re-cent years its popularity grew rapidly in Asia. The game ofsnooker has benefited greatly from the arrival of color tele-vision in the early 1970s, and affordable video technologyin the 1980s. Today, videos are used extensively in snookercoaching. In comparison with most other sports, however,snooker is yet to benefit from modern technology-based per-formance analysis and coaching.

In snooker skill training, all snooker coaches encounterthe difficulty of analyzing the progress of a player quanti-tatively and making an objective comparison between play-ers. Although videos provide an effective means of recordingraw data, watching videos is time-consuming and making acomparative judgment by juxtaposing videos is generally in-effective. An average snooker shot takes about 2-3 seconds,while a cue strikes in the blink of an eye. High-speed filmingis often necessary, but watching such slow motion videos ineveryday training is agonizingly laborious.

In this work, we address the above-mentioned difficulties

by applying video visualization to snooker skill training. Inparticular, we utilized the principal design of the VideoPer-

petuoGram (VPG) [BBS∗08], which provides a focus + con-text visualization of a video stream. We extended the designof the VPG to accommodate the need for depicting multi-ple attribute fields, including silhouette extrusion of objects,spatial time series (e.g., the center of a snooker ball), andnon-spatial time series (e.g., ball size). This work was con-ducted in conjunction with a snooker club, which is led bya former world champion and offers coaching to both pro-fessional and amateur players. It is a feasibility study to de-termine whether video visualization can be deployed in asnooker club to aid snooker training, and if so, what furtherinvestment in equipment, research and development is nec-essary for realizing a technology usable in everyday train-ing. Our results and evaluation have showed that coaches canderive quantifiable and comparable information from staticvideo visualizations, and for the demonstration videos, thevisualization is considered to be a cost-effective means forassessing skill levels and monitoring progress objectivelyand consistently. The study also indicates that a substantialdevelopment effort (at the scale of 10 person-years) is likelyrequired to provide coaches with a suite of visual designs

c© 2010 The Author(s)Journal compilation c© 2010 The Eurographics Association and Blackwell Publishing Ltd.Published by Blackwell Publishing, 9600 Garsington Road, Oxford OX4 2DQ, UK and350 Main Street, Malden, MA 02148, USA.

M. Höferlin et al. / Video Visualization for Snooker Skill Training

(a) cue and ball interaction (b) ball trajectory (c) alignment and delivery (d) grip and wrist motion

(e) six example frames (t = 0,62,124,186,248,310) from an example video, “pink-pot-b2”, for testing spin avoidance skill.

(f) six example frames (t = 0,62,124,186,248,310) from another video, “pink-pot-c2”, for testing spin avoidance skill.

Figure 1: Example frames extracted six typical snooker skill training videos. (a-d) represent different actions that are inter-

esting to snooker coaches. (e-f) represent two video sequences showing different spin avoidance actions (both available as

supplementary materials).

for a variety of cue actions, ball interactions and postures tomeasure different skills.

2. Application Background

In this section, we first describe the specific needs of snookerskill training, which provide the motivation for this work. Wethen describe the objectives of this work, and the equipmentused in capturing the videos.

2.1. Snooker Skill Training

A good snooker player possesses a wide range of skills.While a beginner needs to master basic skills such as main-taining correct stance and grip, forming a bridge with thehand (a V-shaped channel), aligning the cue, and deliver-ing a strike [Eve91], a professional needs to possess neces-sary mental qualities such as motivation, commitment, con-centration, confidence, and decision making under pressure[Cli81, WL02]. This application work focuses on a set ofskills at the intermediate level. Players at such a level havealready acquired basic skills as well as a reasonable feel ofthe interaction between different entities (i.e., body, sight,cue, ball, table, etc.).

For players at the intermediate level, snooker coaches areinterested in the following skills [Gri96]: speed of delivery,application of power, stun, screw, side, cue alignment, spindelivery, and spin avoidance. For the feasibility study, wefocused on spin avoidance: the ability to strike the cue ballwithout applying unintentional spin.

Videos can help maintain a good training record, captur-ing problems and improvements in aspects that are not easilyobservable during training. Figure 1 shows four different as-pects in (a)-(d) that coaches would examine closely. Figure1 also shows key frames extracted from two video sequencesin (e-f) capturing different spin avoidance actions. Whilewatching videos is intuitive and effective in many cases, it istime-consuming, and difficult to make objective comparison.Snooker coaches are longing for modern technologies thatcan help coaching and training. In comparison with manysports, such as tennis and soccer, the deployment of mod-ern technology in snooker for skill training and performanceanalysis is rare. In addition to scientific and technical chal-lenges, the technology developed for snooker training mustalso address the challenges of keeping the costs and resourcerequirements low.

2.2. Application Stakeholders and Data Capturing

Most snooker training clubs in the UK are organized aroundone or two coaches. As in most sports, coaches and profes-sional players are highly motivated and have a great urge tolearn and use new technology. However, they have very lim-ited prior exposure to advanced visualization. This is verydifferent from many other visualization applications thatmainly involve users in the scientific and medical commu-nities. It thereby presents an extra challenge to this work.

This work is conducted in conjunction with a snooker clubthat provides training to both professional and amateur play-ers. A few of the professional players trained in the club are

c© 2010 The Author(s)Journal compilation c© 2010 The Eurographics Association and Blackwell Publishing Ltd.


ranked among the top 20 players in the Official 2000 WorldSnooker Rankings. The club considered investing in a suiteof ceiling-mounted and computer-controlled video captureand replay equipment, but was uncertain whether perfor-mance analysis based on watching videos is scalable fromtraining a few professional players to many amateur players.

We thus initiated this work as a feasibility study to helpthe club answer the following questions:

1. Can video visualization complement video watching inperformance analysis and progress monitoring?

2. Can coaches recognize visual signatures in video visual-ization, and will they be willing to learn such skills?

3. What are the estimated costs and resource requirementsto make video visualization a usable technology in asnooker club?

4. Does video visualization have the potential to help in-crease the use of a costly suite of ceiling-mountedand computer-controlled video capturing and replayingequipment, thus justifying the investment?

Because the proposed ceiling-mounted video capturingequipment was not available to this feasibility study, wemade temporary provision to use portable and relatively low-cost equipment to capture video data. Two filming sessionwere held. The first session allowed us to finalize the equip-ment requirements. All videos used in the work were filmedin the second session. In this feasibility study, we used twoCasio Ex-FH20 cameras, which support high-speed filmingat up to 1000 fps. After the trial run, we decided to use420 fps at 224× 168 resolution. This setting would allowus to capture the high speed actions, such as ball spin andcue vibration, which naked eye cannot easily observe. Al-though the resolution is less desirable for both video pro-cessing and visualization, it provides a worst-case scenarioto test the technology. As a snooker hall is usually not well-lit and high speed filming demands good lighting, we pro-vided four 500 W halogen floodlights mounted on two tele-scopic masts. Without computer-controlled camera synchro-nization, we used a 20 Hz strobe light to help synchronizevideos in the processing stage.

For the feasibility study, we focused on a particular cueaction namely spin avoidance. We used four videos (“pink-pot-b1”, “pink-pot-b2”, “pink-pot-c1”, and “pink-pot-c2”)to assess the technical feasibility and provide the evaluationwith a practical case study. The videos show two differentshots, b and c, each captured from the side (1) and the front(2). Some key frames of the front view videos are shownin Figure 1(e-f). It is not feasible for human or machine vi-sion to quantify spinning from normal snooker cue ball. Incomparison with other spatial transformation (e.g., transla-tion and scaling), spinning is also more difficult to visual-ize [CBH∗06]. We made use of a training cue ball, with itstwo halves colored in black and white respectively. Some ofour visualizations were purposely designed to depict spinsusing this property.

3. Related Work

Video-based analysis is commonly used in modern sportsto aid performance analysis and improvement. In practice,most such analysis is carried out by repeated video view-ing and qualitative discussions in front of a television. Manyattempts have been made to use automated computer visiontechniques. The current advances in this area are representedby a special issue in Computer Vision and Image Under-

standing [MP09]. These attempts generally fall into the fol-lowing categories:

• Tracking — Techniques in this category provide the basisfor high-level analytical tasks by establishing the motiontrajectories of interesting objects or players. For example,Ren et al. reported a technique for tracking a soccer ballfrom multiple fixed camera views [ROJX09]. Kristan et

al. presented an algorithm for tracking multiple players inseveral indoor sports applications [KPPK09]. In general,there is a large volume of literature on tracking techniquesand their applications in sports [AC99, Gav99]. Advancesin this aspect include sophisticated algorithms for han-dling motion-blurred images [CG09], and graph-based as-sociation of tracked objects [YCK08].

• Indexing and retrieval — Techniques in this category al-low videos of sport events to be temporally segmented, in-dexed with known information (e.g., event hierarchy, sen-sor parameters, land marks, etc.), and stored in a multi-media database for future contents-based retrieval or fur-ther video processing. For example, Pingali et al. reportedsuch a system for tennis games [POJC02]. Assfalg et al.

presented a system that segments a sports video into shotsof studio interviews, statistical graphics, audience, play-ing fields and close-up views of players. They workedwith videos of 10 different sports, with varying accuracy,e.g., 38% (javelin), 57% (diving), 69% (tennis), 80% (soc-cer), and 88% (track). The technology in this aspect is rel-atively mature, though human involvement in correctingerroneous classification is necessary in practice.

• Event Classification — Techniques in this categoryare intended to generate semantic description of eventsin videos using automated reasoning. For example,D’Orazio et al. developed a vision system for classifyingdoubtful goal scoring situations in soccer matches usingcameras located along the goal line [DLS∗09]. Perse et

al. presented a method for detecting three phases, namelyoffensive, defensive and time-out phases, in a basketballgame, achieving 92% accuracy [PKaVP09]. In general,the successes in this area are limited by both restrictedfilming conditions and simplicity of semantic classes.

The term “visualization” in sports commonly refers tomental practice and rehearsal [SK94], which is not the topicof this paper. In this work, “visualization” is considered acomputational process for extracting meaningful informa-tion from sporting data, and conveying it visually to theusers. One of the common uses of visualization in sports is



Figure 2: The basic steps of the video processing pipeline.

to depict the motion trajectories of moving objects and play-ers. Pingali et al. presented a system for tennis with visualdesigns for viewing trajectory lines, coverage maps, land-ing points, 2D charts and virtual 3D replay. They demon-strated that visualization can provide insights into perfor-mance, style, and strategy in sports [POJC01]. Such visualdesigns are now commonly seen in sports broadcasts. Grauet al. presented a system for reconstructing soccer matchesfrom multiple cameras, and allowing replay from arbitraryviewpoints where moving objects are approximated by bill-boards, visual hulls or view-dependent geometry [GHK∗07].Denman et al. presented a tool for summarizing ball trajec-tories overlaid on a video frame. They also used 2D graphsto illustrate ball position and speed [DRK03].

Video visualization is a growing subject. Daniel andChen [DC03] proposed to apply volume rendering tech-niques to visualize information in video data. Botchen et al.

demonstrated the feasibility of visualizing video streams inreal-time with GPU-based multi-field rendering [BCWE06].Chen et al. conducted a user study, confirming that noviceusers can learn to recognize visual signatures in video vi-sualization [CBH∗06]. Dony et al. [DMR05], Assa et al.

[ACC05], and Caspi et al. [CAMG06] presented techniquesfor extracting important objects from different key frames,and reconstructing a storyboard by combine different tem-poral objects into a single still image. Several researchersstudied the techniques for viewing multiple video streamswith their spatial context. For example, Wang et al. used3D environment models to contextualize spatially-relatedvideos [WKCB07], and conducted a user study to evaluatedifferent design options [WKCB07]. Romero et al. made anattempt to visualize video captured from an overhead cam-era [RSSA08]. Botchen et al. [BBS∗08] presented a focus +context design of video visualization, combining key frames,spatial and non-spatial attributes into a static visualization.They created a visual representation of a continuous videostream, called VideoPerpetuoGram (VPG), in a manner sim-ilar to an electrocardiogram (ECG) or a seismograph. Wehave adopted this visual design in this work.

Overall, in comparison to the literature in video process-ing, the body of work in video visualization is limited. To ourknowledge, there has not been any application case study onsports video visualization.

4. From Videos to Spatiotemporal Attributes

This work adopted the same principal pipeline of [CBH∗06,BBS∗08], supporting video visualization with a video pro-cessing stage. This section discusses the processing require-ments and briefly outlines the video processing stage, whileSection 5 addresses video visualization.

One objective of this feasibility study is to assess ifcoaches can recognize visual signatures [CBH∗06] fromvideo visualization. This requires the processing stage to ex-tract different temporal information. For example, for thetwo above-mentioned shots and their video sequences (likethose in Figures 1(e-f)), this includes:

a the silhouette of a ball,b the different color segments of a ball,c the center of a ball, or of each segment,d the color separation line on the black-white cue ball.

There are many solutions in the computer vision litera-ture for such problems. In this work, we make use of matureand fast video processing techniques to extract the desiredfeatures, but avoid complex techniques such object classifi-cation and tracking and learning-based pattern recognition.For example, we may apply filters to extract a pixel map ofsilhouettes of all balls, and then forward such a pixel mapdirectly to the visualization pipeline without making explic-itly recognition as if there is any ball in the pixel map or howmany. In this way, the potential errors for automatic machinerecognition are avoided, and usually visualization artifactscaused by minor filtering errors are easily tolerated by thehuman vision system.

We designed a video processing pipeline (see Figure 2)based on the generalized symmetry transform [RWY95]to estimate the centers and radii of the balls and to seg-ment them. The remainder of the pipeline is based on basictechniques provided by OpenCV [BK08]: we use a linearKalman filter to track the balls, and classify them accord-ing to pixel colors in their area. We then further segmentthe black-white cue ball in the same way. Finally, we extractseveral other features for the black and white segments, e.g.,the number of pixels, the separation line, and the segmentcenters. Further details on the video processing pipeline areprovided in the supplementary material of this paper.



pink-pot-b2 pink-pot-c2

(a) trajectories of ball centers

(b) trajectories of color segments

(c) object silhouette volume

(d) the separation edge on the black-white cue ball

Figure 3: Visualizing four different types of spatial at-

tributes.

(a) pink-pot-b2 (b) pink-pot-c2

Figure 4: Visualizing non-spatial attributes. The radius of

the yellow tube-like object shows the temporally varying ra-

tio of white pixels on the black-white cue ball.

5. Multi-Strand VideoPerpetuoGram (VPG)

The visualization component is based on the VideoPerpetuo-Gram (VPG) framework originally designed for the inte-gration of spatial and temporal aspects in video visualiza-tion [BBS∗08]. We follow the focus + context design of

VPG, with a few technical modifications to accommodatethe needs of this application. The technical contributionsto visualization are the extensions of the VPG to com-bine different attributes in the same visualization (multi-attributes extension), and the possibility to have several tem-porally synchronized visualization images displayed simul-taneously (multi-strand extension). As shown by Botchen et

al. [BBS∗08], the rendering of the VideoPerpetuoGram isreal-time and the modifications made in this paper maintaininteractive frame rates. In this section, we outline the generalconsiderations for the visual design, then consider the visu-alization of individual spatiotemporal attributes extracted inthe video processing stage (see Section 4), and finally dis-cuss the combination of different attributes in the same visu-alization, and the multi-strand VPG.

5.1. General Design Principles

The targeted audience of this feasibility study includessnooker coaches and players. We assume that these usersare not familiar with advanced visualization systems. Basedon this, the following design principles were defined, andlater improved according to the feedback from the applica-tion partners.

Color Mapping. Color mapping should enable visualiza-tion to match the colors on snooker table as close as possible.The trajectory of a snooker ball should ideally be depictedusing the original ball color. When there is a conflict, suchas with the background, a uniquely identifiable color shouldbe used within the visualization.

Providing Context. Following the VPG design, the con-text (i.e., key frames of a video) should be present in everyvisualization. One or just a few key frames are often ade-quate as the scene is relatively static.

Minimizing Navigation. Since the primary advantage ofvideo visualization is to save time, we should not replacethe time for watching video with the time for interaction andnavigation. Hence, users are given a few fixed camera posi-tions from which visualizations are generated. Similarly, or-thographic projection is employed to avoid misleading per-spective foreshortening.

Visualization Literacy. We believe that visualization lit-eracy can be improved. We should prepare coaches and play-ers with simple visualization designs, and gradually intro-duce more complex designs.

5.2. Visual Mapping of Spatial Attributes

Attributes are pieces of information extracted from a videoduring the video processing stage. As shown in Section 4,such information is obtained frame by frame. Hence, all at-tributes are temporal. Some attributes, such as the center ofa ball or the line separating a black-white cue ball, are inher-ently spatial, and can be placed in 3D space with the time asthe third dimension. We term these spatial attributes.



Displaying spatial attributes in 3D is relative intuitive. Asshown in Figure 3(a), the center of a ball can be displayedin conjunction with a few key frames. Similarly, we can alsodisplay the centers of three colored segments pink, black andwhite as in Figure 3(b). These time series of positions are vi-sualized by rendering thick lines implemented as geometrictubes and indicating the trajectories of the balls.

Note that for video “pink-pot-c2”, which did not manageto avoid spin, the patterns in (b) are more observable than in(a). This is because that the centers of the black and whitesegments of the cue ball revolve around each other duringthe spin. Nevertheless, the tubes in (a) are suitable to showthe trajectories of the balls.

The object silhouette can be displayed as a volumetric ob-ject passing through the key frames, as in Figure 3(c). Thetransfer function used is similar to Ebert et al. [EMRY02],where the RGB values in the video are preserved, and al-pha is derived from image analysis results. While the ob-ject silhouette volumes are inappropriate to show the degreeof spin, they can provide the trajectories of balls preservingoriginal radii and colors.

The separating edge of the black-white ball is depictedwith a ribbon. A spinning black-white ball results in a twist-ing or broken ribbon, providing a visually intuitive way tocompare cue actions for spin avoidance. For example, asshown in Figure 3(d), a poor shot will causes twisted or bro-ken ribbons, while a good shot will not.

5.3. Visual Mapping of Non-Spatial Attributes

Certain temporal attributes extracted by the video processingdo not have any inherent spatial location, yet still form a timeseries. We term these non-spatial attributes. For example,the numbers of black and white pixels in the two videos inFigures 1(e-f) give a good indication of whether the black-white cue ball is spinning.

In principle, non-spatial attributes can be displayed ade-quately using a 2D plot. However, this contradicts the prin-ciple of providing context (e.g., video frames), whenever onecan. We thus designed a context + focus visualization for theratio of white pixels on the black-white cue ball (Figure 4).We define this ratio as: #white pixels

#white pixels + #black pixels . In the vi-sualization, this ratio is mapped to the radius of a tube thatextends out from the corresponding ball position in the keyframe. We were curious about how potential users would re-act to such visualization (see Section 6).

Note that some attributes and their visual mappings pro-vide insights into similar properties. For example, both tra-jectories of ball centers (Figure 3(a)) and object silhouettevolume (Figure 3(c)) show ball trajectories. The spin of ashot is conveyed by different visual patterns: trajectories ofsegment centers (Figure 3(b)), the ribbon showing the sep-aration edge on the black-white cue ball (Figure 3(d)), and

the non-spatial ratio visualization (Figure 4). Therefore, it isnot mandatory to show all visual patterns, but the user maybenefit from several, independent visualizations in the caseof borderline shots.

5.4. Multi-Attributes VPG

Multiple attributes can be combined within a single VPG,as demonstrated in Figure 5. In addition to the key frame(s),each visualization shows the object silhouette, and the cen-ters of the black and white segments. A special case of theVPG can be created by looking along the temporal axis to-wards the initial key frame; implicitly registering data withinthe static scene (Figure 5(a)). This special-case layout is sim-ilar to many visual representations of trajectories in videos.While this visual design has the advantage of being intuitive,it suffers several drawbacks. It lacks of temporal referencefor information, such as speed. It also has limited degreesof freedom for the orientation of tracking geometries. Forexample, the trajectories of color segments in Figure 3(b)can go up and down as well as sideways. It is not easy todepict an up and down motion with the visual design in Fig-ure 5(a). The VPG supports arbitrary, uniform scales of thetime axis. Of course, the trajectories in Figure 5(b) are space-time trajectories while the ones in Figure 5(a) are their pro-jection along the time axis. The latter is more familiar tomost of novice users. Nevertheless, for the spin avoidancevideos, the coaches are not interested in the actual path ofthe balls, but the juddering or deformation of the trackinggeometries. Furthermore, interpretation of space-time trajec-tories can be learned (as demonstrated in a large controlleduser study [CBH∗06], see also discussion in Section 6).

Allowing the simultaneous display of volume data andsurface geometry is a further extension to the VPG. The dis-play of surface and volume data together is achieved by firstrendering the opaque surface geometry (with depth write anddepth test enabled), then rendering the slices through the vol-ume (with blending and depth test enabled, but depth writedisabled).

5.5. Multi-Strand VPG

A major extension of the VPG is to enable simultaneous dis-play of several, temporally synchronized visualization im-ages. Multiple time-series visualizations, which we termmulti-strand, are shown side-by-side, to visualize a collec-tion of spatial and non-spatial attributes (Figure 6). At-tributes with similar spatial context may also be separatedto prevent visual overload of the user, and occlusion of theVPG. For example, the separating edge should normally bein a different strand from that for the ball centers, as theyfrequently occlude each other.

The multi-strand VPG is motivated by the navigation costanalysis by Ware [War04]. He discusses various methods fornavigating through information spaces and their time costs:



pink-pot-b2 pink-pot-c2

(a) front view

(b) side view

Figure 5: A VPG that combines object silhouette volume and

trajectories of color segments.

in the ideal visualization, all information is available on asingle high-resolution screen, because a single saccadic eyemovement will change from one space to another in about150ms. Meanwhile, the cognitive effort for alternative nav-igation methods, such as a hypertext link, takes at least 2s.Therefore, we have chosen a side-by-side visualization withseveral attributes of up to two shots.

6. Results and Evaluation

In this feasibility study, we designed and rendered a varietyof VPG visualizations. These included:

• Spatial attributes, with key frames as context (Figures3(a-d)).

• Non-spatial attributes, with key frame context (Figure 4).• Combined object silhouette volume and trajectories of

color segments, with key frames as context (Figure 5).• Multi-strand VPG (Figure 6).

It is not generally feasible to organize a user study involv-ing a large number of snooker coaches because there are onlyabout 20-30 registered snooker coaches in the UK, spreadingin different parts of the country. We organized a validationmeeting with five potential users to evaluate the visual re-sults of the feasibility study. The five participants includedtwo full-time snooker coaches, C1 and C2, one of whom is aformer world champion, one sports scientist, S, who is alsoa cricket coach, one intermediate-level amateur player A1,and one basic level amateur player, A2, who has a full lengthsnooker table at home.

Coach C2, who is a co-author of this paper, manages asnooker club and trains intermediate and basic level players.The participants’ knowledge about visualization is largely

limited to basic graph plotting, such as in spreadsheet.Hence, an evaluation through open discussions allows us toengage the potential users not only to provide the feedback,but also to learn and appreciate merits of visualization.

We prepared 6 sets of questions, and organized the discus-sions using PowerPoint slides that also showed some exam-ple videos and visualization results. In addition, we preparedtwo sets of paper copies of sample visualization results onforms on which a coach could write further comments fora trainee. None of the participants saw any visualization re-sults prior to the meeting. The discussions, which took aboutan hour, are summarized in a table in the Appendix (includedin the supplementary materials).

During the meeting, we introduced video visualizationgradually, by first showing a 2D example, such as a stockmarket graph, and then showing a video visualization thatuses a similar visual metaphor. This was very effective inhelping the participants to understand the results of videovisualization. In general, the coaches who took part in themeeting liked the visualizations in Figures 3(a-b), 4, 5(a),and 6. They had some difficulties with translucent volumesas in Figure 5(b). This may be partly because those volu-metric effects are inherently more difficult to appreciate andpartly because that we did not have effective 2D stimuli tointroduce this concept. The visualization results in Figure3(d) were not available to the meeting.

A few visualizations similar to Figure 4 were shown in themeeting. We used Minard’s map as an introduction, whichenthused the participants and stimulated much discussions.Figure 4 is an improved version of what were shown in themeeting.

Participants were convinced that video visualization of-fers an effective means for communication, comparison, andarchiving. They also appreciated our observation that the au-tomatic video analysis is not as readily usable as video visu-alization. Some participants were relieved that the technol-ogy is not ready to replace the coaches, while some remainedhopeful for a fully automatic technology.

In terms of the four questions listed in Section 2.2, wesummarize our observations based on the discussions inthe validation meeting, further engagement with the partici-pants, and the authors’ experience throughout this feasibilitystudy:

1 Snooker coaches are longing for a technology that re-duces time spent watching videos, and aid their analysisand monitoring tasks. The coaches who took part in theevaluation were not concerned with the fact that visual-ization needs human interpretation, and liked the idea thatthey are not replaced by unreliable machine intelligence.However, for visualization to be usable, the functionalitydelivered in types of this feasibility study has to be scaledup to some 10-20 cuing actions.

2 Coaches can learn to recognize visual signatures. In par-



Fig

ure

6:

Am

ulti-stra

nd

Vid

eoP

erpetu

oG

ram

com

parin

gth

etw

osh

ots

“pin

k-pot-b

”a

nd

“pin

k-pot-c”

,ea

chca

ptu

redfro

mth

esid

e(1

)and

the

front

(2).

The

front

view

video

sequen

cesare

the

sam

eas

show

nin

Fig

ure

1.

On

the

upper

and

low

erleft

side,

the

front

views

of

the

shots

are

visualized

by

keyfra

mes,

pin

kball

centers,

and

the

separa

tion

lines

on

the

bla

ck-white

cue

balls

represen

tedby

ribbons.

The

side

views

are

disp

layed

on

the

right

side.

The

attrib

utes

show

nhere

are

keyfra

mes,

object

silhouettes,

and

ball

centers.

The

functio

nplo

tson

the

leftsid

esh

ow

the

ratio

of

white

pixels

of

the

bla

ck-white

cue

ball.

The

four

curves

corresp

ond

toth

efo

ur

differen

t

video

sequen

ces.T

heir

colo

rsm

atch

with

the

colo

rsof

the

boundin

gboxes

conta

inin

gth

evid

eonam

es.W

edecid

edto

use

a2D

plo

tto

disp

lay

the

non-sp

atia

lattrib

utes

since

the

spatia

lco

ntext

can

be

pro

vided

by

tempora

lalig

nm

ent,

horizo

nta

lsp

atia

lalig

nm

ent,

and

reference

lines

toth

easso

ciated

strands

of

video

“pin

k-pot-b

2”

and

“pin

k-pot-c2

”.

Adeta

iledco

mpariso

nis

only

required

betw

eenpara

llelview

s,w

hich

limits

the

cognitive

effort.

Additio

nally,

som

efea

tures

can

be

visualized

inall

VP

Gs,

such

as

the

trajecto

ryof

the

pin

kball.

This

may

help

the

user

totra

nsfer

the

info

rmatio

neven

toorth

ogonal

views,

while

info

rmatio

nis

not

need

edto

be

com

pared

betw

een

them

.



ticular, we were surprised by how quickly they trans-ferred their learning of Minard’s map to the visualiza-tion of non-spatial attributes (Figure 4). Learning stimuli(e.g., simpler examples) and metaphors might play an im-portant role in learning visual signatures.

3 Visualization can be used in many aspects of snooker skilltraining. In addition, the participants identified a numberof potential uses of such visualization, including (i) in-dividual training records in the form of a collection ofreports similar to the sample paper copies, (ii) a collec-tion of visualization results from professionals as bench-marks, (iii) introduction of standard skill tests, and (iv)visual records of snooker equipment testing.

4 It is common for ordinary people to overestimate whatcan be accomplished by machine intelligence. The ini-tial expectation of the snooker coaches was that computervision techniques are readily available to analyze cap-tured video automatically, for example, by categorizingand recognizing various types of shots, and subtle dif-ference in cuing actions and poses. In this study, we ob-served that for automatic video analysis to deliver reliableresults, it requires more sophisticated environment (e.g.,lighting and ball textures), accurate calibration, and highresolution and high speed cameras. The costs and incon-venience of setting up the environment for each trainingtest will likely outweigh the benefits from time reductionin viewing the videos. This feasibility study made a con-vincing case for using video visualization as the primarytechnology for supporting snooker training.

5 The snooker club has since made its investment to installceiling-mounted cameras, and is actively seeking finan-cial sponsors for developing video visualization technol-ogy through a consortium of several companies and uni-versities.

For video visualization, once a visual design is finalizedand implemented, it can generally be transferred to othershots. However, this is usually not true for video process-ing. The area coverage of a shot, the number of balls, light-ing, etc. could force some modification to a video process-ing pipeline. If more automatic classification and recogni-tion techniques were used, the semantics of each shot haveto be hard-coded into vision algorithms. Hence, for deliver-ing video visualization system, with 10 times of the func-tionality delivered in this feasibility study, and taking intoaccount the development cost for this feasibility study, weestimate about 10 person-years that are necessary for realiz-ing a usable technology in the form of an industrial product.This includes 4 person-years for generic software features,1 person-year for management, and 10 × 0.5 person-yearsfor test-specific software features. For developing automaticvideo analysis to support a similar level of functionality, weestimate at least 30 person-years that are necessary for re-alizing a usable system. This includes 3 person-years forgeneric software features, 2 person-year for management,and 10 × 2.5 person-years for test-specific software devel-

opment (e.g., training data capturing, video processing, su-pervised learning, classification and results presentation).

7. Conclusions

In this paper, we have reported a feasibility training on theapplication of video visualization in snooker. The study hasfound that the use of automatic video analysis techniques inpractice has been hindered by several factors, such as vary-ing accuracy, poor portability from one sport to another, andsemantic bottleneck in supervised machine learning. Exist-ing commercial systems are limited largely to motion track-ing and video editing and annotation. Viewing videos is stillthe main operational mode after a video is processed by suchas system.

To make a convincing case for using video visualizationto reduce the burden of viewing videos, we developed a pro-totype system in conjunction with a snooker club. We haveadapted and extended the concept of VideoPerpetuoGram(VPG) to accommodate various attributes generated at thevideo processing stage. To extract these attributes even un-der different camera views, we adopted reliable and well-known computer vision techniques. We have introduced amulti-attributes extension as well as a multi-strand VPG.Both can be utilized in other applications whenever severalattributes should be visualized in one or more VPGs simulta-neously. We have learned that video visualization is a more“transferable” technology than automated machine vision,so it would cost less to develop in a short to medium term.We have learned that video visualization has to cover a rea-sonable number of common tasks in snooker training beforeit becomes cost-effective to deploy. It was a huge reward tofind out that novice users can learn to comprehend and ap-preciate video visualization, and to recognize visual signa-tures. Although they showed preference for more intuitivevisualization as a means to communicate with the players,they enjoyed the collaboration, and determined to continuetheir investment in new technologies.

This feasibility study opens up many new opportunitiesfor further research. We are in particular interested studyingthe correlation between different snooker videos. We alsohope to identify new applications of video visualization, andadvance the video visualization technology to serve such ap-plications.

Acknowledgment

This work was partially funded by a UK EPSRC grantEP/G006555, by DFG as part of the Priority Program “Scal-able Visual Analytics” (SPP 1335), and supported by DAAD(PPP project ID 50023502). We are grateful to Adrian Mor-ris for supporting the data capture exercise and coordinat-ing the consortium for further development. We would liketo thank those who took part in the evaluation, including SirTerry Griffiths OBE, Dr. Richard Griffiths and Alison Parker.



References

[AC99] AGGARWAL J. K., CAI Q.: Human motion analysis: areview. Computer Vision and Image Understanding 73, 3 (1999),428–440.

[ACC05] ASSA J., CASPI Y., COHEN-OR D.: Action synopsis:pose selection and illustration. ACM Transactions on Graphics

24, 3 (2005), 667–676.

[BBS∗08] BOTCHEN R. P., BACHTHALER S., SCHICK F., MORI

M. C. G., WEISKOPF D., ERTL T.: Action-based multi-fieldvideo visualization. IEEE Transactions on Visualization and

Computer Graphics 14, 4 (2008), 885–899.

[BCWE06] BOTCHEN R. P., CHEN M., WEISKOPF D., ERTL T.:GPU-assisted multi-field video volume visualization. In Proc.

International Workshop on Volume Graphics (2006), pp. 47–54,135.

[BK08] BRADSKI G. R., KAEHLER A.: Learning OpenCV:

Computer Vision with the OpenCV Library. O’Reilly, 2008.

[CAMG06] CASPI Y., AXELROD A., MATSUSHITA Y., GAM-LIEL A.: Dynamic stills and clip trailers. The Visual Computer

22, 9 (2006), 642–652.

[CBH∗06] CHEN M., BOTCHEN R. P., HASHIM R. R.,WEISKOPF D., ERTL T., THORNTON I. M.: Visual signaturesin video visualization. IEEE Transactions on Visualization and

Computer Graphics 12, 5 (2006), 1093–1100.

[CG09] CAGLIOTI V., GIUSTI A.: Recovering ball motion froma single motion-blurred image. Computer Vision and Image Un-

derstanding 113, 5 (2009), 590 – 597. Computer Vision BasedAnalysis in Sport Environments.

[Cli81] CLIFFORD W. G.: Winning Snooker. Foulsham, 1981.

[DC03] DANIEL G. W., CHEN M.: Video visualization. In Proc.

IEEE Visualization (2003), pp. 409–416.

[DLS∗09] D’ORAZIO T., LEO M., SPAGNOLO P., NITTI M.,MOSCA N., DISTANTE A.: A visual system for real time de-tection of goal events during soccer matches. Computer Vision

and Image Understanding 113, 5 (2009), 622–632.

[DMR05] DONY R. D., MATEER J. W., ROBINSON J. A.: Tech-niques for automated reverse storyboarding. IEE Journal of Vi-

sion, Image and Signal Processing 15, 4 (2005), 425–436.

[DRK03] DENMAN H., REA N., KOKARAM A.: Content-basedanalysis for video from snooker broadcasts. Computer Vision and

Image Understanding 92 (2003), 176–195.

[EMRY02] EBERT D. S., MORRIS C. J., RHEINGANS P., YOO

T. S.: Designing effective transfer functions for volume render-ing from photographic volumes. IEEE Transactions on Visual-

ization and Computer Graphics 8, 2 (2002), 183–197.

[Eve91] EVERTON C.: Snooker & Billiards: Technique * Tactics

* Training. Crowood Press, 1991.

[Gav99] GAVRILA D. M.: The visual analysis of human move-ment: a survey. Computer Vision and Image Understanding 73,1 (1999), 82–98.

[GHK∗07] GRAU O., HILTON A., KILNER J., MILLER G.,SARGEANT T., STARCK J.: A free-viewpoint video system forvisualisation of sport scenes. SMPTE Motion Imaging Journal

(2007), 213–219.

[Gri96] GRIFFITHS T.: Snooker – basic skills [vhs], 1996.

[KPPK09] KRISTAN M., PERŠ J., PERŠE M., KOVACIC S.:Closed-world tracking of multiple interacting targets forindoor-sports applications. Computer Vision and Image Understanding

113, 5 (2009), 598–611.

[MP09] MAGEE D., PERS (EDS.) J.: Computer vision basedanalysis in sport environments. Computer Vision and Image Un-

derstanding 113, 5 (2009), 589–662.

[PKaVP09] PERŠE M., KRISTAN M., ANDG. VUCKOVIC S. K.,PERŠ J.: A trajectory-based analysis of coordinated team activi-tyin a basketball game. Computer Vision and Image Understand-

ing 113, 5 (2009), 612–621.

[POJC01] PINGALI G., OPALACH A., JEAN Y., CARLBOM I.:Visualization of sports using motion trajectories:providing in-sights into performance, style, and strategy. In Proc. IEEE Vi-

sualization (2001), pp. 75–82.

[POJC02] PINGALI G. S., OPALACH A., JEAN Y. D., CARL-BOM I. B.: Instantly indexed multimedia databases of real worldevents. IEEE Transactions on Multimedia 4, 2 (2002), 269–282.

[ROJX09] REN J., ORWELL J., JONES G. A., XU M.: Trackingthe soccer ball using multiple fixed cameras. Computer Vision

and Image Understanding 113, 5 (2009), 633–642.

[RSSA08] ROMERO M., SUMMET J., STASKO J., ABOWD G.:Viz-A-Vis: Toward visualizing video through computer vision.IEEE Transactions on Visualization and Computer Graphics 14,6 (2008), 1261–1268.

[RWY95] REISFELD D., WOLFSON H., YESHURUN Y.: Contextfree attentional operators: the generalized symmetry transform.International Journal of Computer Vision 14 (1995), 119–130.

[SK94] SHEIKH A. A., KORN E. R. (Eds.): Imagery in Sports

and Physical Performance. Baywoord, 1994.

[War04] WARE C.: Information Visualization: Perception for De-

sign, 2nd ed. Morgan Kaufmann Publishers Inc., San Francisco,CA, USA, 2004.

[WKCB07] WANG Y., KRUM D. M., COELHO E. M., BOWMAN

D. A.: Contextualized videos: Combining videos with environ-mentmodels to support situational understanding. IEEE Trans-

actions on Visualization and Computer Graphics 13, 6 (2007),1568–1575.

[WL02] WILLIAMS K., LOWE T. (Eds.): Snooker: Know the

Game. A & C Black, 2002.

[YCK08] YAN F., CHRISTMAS W., KITTLER J.: Layered dataassociation using graph-theoretic formulation with applicationsto tennis ball tracking in monocular sequences. IEEE Transac-

tions on Pattern Analysis and Machine Intelligence 30, 10 (2008),1814–1830.


Documents

eurovis10-hoeferlin