8
Humeri M 3D in adve N mwmWe.-tkN Siegmund Pastoor Three-dimensional displays provide an unambiguous visual representation of the spatial structure of natural scenes and computer-generated virtual environments and thus have proven substantial advantages over conventional displays in remote guidance and inspection tasks and in application fields such as medical imagery and architectural and molecular modelling. Recent years have seen increasing efforts to extend 3D technologies into the domain of image communications. These efforts received support from human factors studies indicating that 3D displays are highly appreciated by prospective users of image communications systems for their enhanced psychological effects (telepresence and communicative presence). On the other hand, these studies have revealed heavy techno- logical requirements that must be met in order to avoid visible image distortions and increased visual strain. Keywords: 3DTV, 31) videoconferencing, subjective assessments, stereoscopic displays, muir/view displays, perspective interpolation As markets for consumer- and microelectronics become saturated, research and development in the field of tele- vision technologies intensifies. Consequently, numerous attempts were made to improve picture quality within the framework of current standards, leading to new products or upgrades of existing equipment. In the course of time, this evolutionary process was supplemented by certain revolutionary steps, such as the introduction of colour TV and, nowadays, of high definition TV- obliging both studios and private consumers to switch over to the next hardware generation. There is no reason to assume that this process has now come to an end. The ultimate challenge of television has recently been defined as 'to provide generous sup- port for any demand arising in the creative and artistic Heinrich-Hertz-Institut fiir Nachrichtentechnik Einsteinufer 37, D-10587 Berlin 10, Germany Paper received: 13 April 1993 Berlin GmbH, process of designing audio-visual presentations, with the only limitations set by visual and auditory thresholds and the human information processing capacity t. Judg- ing HDTV from this point of view, there still remains considerable potential for improvement. Among experts, it is the general consensus that three-dimensional TV, adding binocular and motion parallax information to high-definition images, should be the next step after HDTV. The anticipated course of development implies that 3DTV will have to compete with quality standards given by advanced and fully developed HDTV technology. At present, there is no doubt about the potential benefits of three-dimensional presentations as compared to flat pictures. On the other hand, it seems wise to assume that within the foreseeable future it will not be possible to overcome the technological barriers of a truly three- dimensional imaging system. The main problem is to display an optical array that permits accommodation to different fixation point distances within the reproduced volume of a pictorial scene, thereby exploiting the per- ceptual benefits of the natural limitation of the depth-of- focus of the human eye2. Holographic imaging promises the ultimate solution; however, it is currently impossible to make any reliable prognosis on the time and efforts needed to develop holographic technologies suiting the practical demands of TV systems. For the time being, it seems reasonable to concentrate on stereoscopic methods displaying sets of planar images of three-dimensional scenes taken from different viewpoints so as to provide stereoscopic depth and motion parallax. At present, it is not clear at which performance level and to what extent stereoscopic methods have the potential to satisfy user demands on future image communications systems. Human factors studies intend to clarify this issue and to provide the basis to evaluate various technological solutions. This paper gives an over- view of relevant studies of the Human Factors Depart- ment, Heinrich-Hertz-Institut Berlin, and attempts to outline an approach to 3DTV currently under discussion. 0141-9382/93/030150-08 © 1993 Butterworth-Heinemann Ltd 150 Displays Volume 14 Number 3 1993

Human factors of 3D displays in advanced image communications

Embed Size (px)

Citation preview

Humeri M 3D in adve N mwmWe.-tkN Siegmund Pastoor

Three-dimensional displays provide an unambiguous visual representation of the spatial structure of natural scenes and computer-generated virtual environments and thus have proven substantial advantages over conventional displays in remote guidance and inspection tasks and in application fields such as medical imagery and architectural and molecular modelling. Recent years have seen increasing efforts to extend 3D technologies into the domain of image communications. These efforts received support from human factors studies indicating that 3D displays are highly appreciated by prospective users of image communications systems for their enhanced psychological effects (telepresence and communicative presence). On the other hand, these studies have revealed heavy techno- logical requirements that must be met in order to avoid visible image distortions and increased visual strain.

Keywords: 3DTV, 31) videoconferencing, subjective assessments, stereoscopic displays, muir/view displays, perspective interpolation

As markets for consumer- and microelectronics become saturated, research and development in the field of tele- vision technologies intensifies. Consequently, numerous attempts were made to improve picture quality within the framework of current standards, leading to new products or upgrades of existing equipment. In the course of time, this evolutionary process was supplemented by certain revolutionary steps, such as the introduction of colour TV and, nowadays, of high definition T V - obliging both studios and private consumers to switch over to the next hardware generation.

There is no reason to assume that this process has now come to an end. The ultimate challenge of television has recently been defined as 'to provide generous sup- port for any demand arising in the creative and artistic

Heinrich-Hertz-Institut fiir Nachrichtentechnik Einsteinufer 37, D-10587 Berlin 10, Germany Paper received: 13 April 1993

Berlin GmbH,

process of designing audio-visual presentations, with the only limitations set by visual and auditory thresholds and the human information processing capacity t. Judg- ing HDTV from this point of view, there still remains considerable potential for improvement. Among experts, it is the general consensus that three-dimensional TV, adding binocular and motion parallax information to high-definition images, should be the next step after HDTV.

The anticipated course of development implies that 3DTV will have to compete with quality standards given by advanced and fully developed HDTV technology. At present, there is no doubt about the potential benefits of three-dimensional presentations as compared to flat pictures. On the other hand, it seems wise to assume that within the foreseeable future it will not be possible to overcome the technological barriers of a truly three- dimensional imaging system. The main problem is to display an optical array that permits accommodation to different fixation point distances within the reproduced volume of a pictorial scene, thereby exploiting the per- ceptual benefits of the natural limitation of the depth-of- focus of the human eye 2. Holographic imaging promises the ultimate solution; however, it is currently impossible to make any reliable prognosis on the time and efforts needed to develop holographic technologies suiting the practical demands of TV systems. For the time being, it seems reasonable to concentrate on stereoscopic methods displaying sets of planar images of three-dimensional scenes taken from different viewpoints so as to provide stereoscopic depth and motion parallax.

At present, it is not clear at which performance level and to what extent stereoscopic methods have the potential to satisfy user demands on future image communications systems. Human factors studies intend to clarify this issue and to provide the basis to evaluate various technological solutions. This paper gives an over- view of relevant studies of the Human Factors Depart- ment, Heinrich-Hertz-Institut Berlin, and attempts to outline an approach to 3DTV currently under discussion.

0141-9382/93/030150-08 © 1993 Butterworth-Heinemann Ltd 150 Displays Volume 14 Number 3 1993

Human factors of 3D displays: S Pastoor

3D I M A G E C O M M U N I C A T I O N S

It is anticipated that 3DTV-similar to HDTV-wi l l eventually cover a wide range of domestic and pro- fessional applications like TV broadcasting, information retrieval, and conversational services (e.g. videotelephony and videoconferencing). As stereoscopic methods can only approximate but not replicate all aspects of natural binocular vision, it seems necessary to plumb the actual advantages of 3DTV and to determine the specific user requirements in various application fields.

Television broadcasting

Concerning television broadcasting, subjective evaluations of 3D versus 2D (HDTV) motion picture presentations were gathered from samples of typical TV viewers (i.e. non-experts) 3. For these evaluations a multi- dimensional assessment methodology was developed 4, since the traditional psychophysical procedures as well as the CCIR methods for quality and impairment evalu- ations cover only a narrow section of relevant psycho- logical effects of advanced TV systems. For instance, it was deemed necessary to use pictorial material bearing a close resemblance to ordinary (HD)TV programmes instead of still images or short sequences. Data on the immediate impact of specific shots were collected via simple input devices (e.g. joystick) during picture presentations. Afterwards bipolar rating scales were used for a still spontaneous, yet more differential appraisal of personal experiences. Finally, a questionnaire served to provoke well considered, reflective evaluations of actual experiences as well as some further generaliz- ation. Thus, for instance, items were related to a direct 2D/3D comparison, overall image quality, advantages and disadvantages of large-screen presentation for 3D and 2D, anticipated suitability of 3D for different types of TV programmes, and market acceptance. The picture material (a 40-minute regional news programme) was recorded and displayed with 35mm film equipment using polarizing filters for left/right image separation. To have equal viewing conditions, special polarizing glasses had to be worn for 2D presentations as well.

From a synopsis of the collected response data, a clear preference for 3D presentation emerged: the immediate impact was felt to be more intense and satisfying, identical scenarios were rated more appealing and inter- esting, and a final forced choice alternative led to a clear- cut decision (of 88% of the viewers) in favour of 3D. On the other hand, the study also revealed some limiting factors. For instance, despite the fact that special and presumably annoying 3D effects (e.g. things popping far out of the screen or exaggerated stereoscopic space) had been avoided, subjects indicated a significant amount of disagreeable 3D 'micro-events' (too much impact, loss of psychological distance, visual strain). Further studies are needed to examine whether or not this aspect can

60

.

40- .

30- .

20-

10-

Preference [%]

m I I i _ J m <= 2.0 2.5 - 3.0 3.5 - 4.0 4.5 =>

Figure 1 Preferred viewing distances (in multiples of the picture heigh0 for large screen (lm x 2.2 m) 3D (I-q) and 2D (1) motion picture presentation

be neglected in applications with a prolonged viewing time.

With respect to the viewing distance (a parameter directly influencing the required spatial resolution of the display) almost the same conditions were preferred for large-screen HDTV and 3DTV so that under this aspect both systems will be compatible. A distance of between three and four times the picture height seems to be advisable (see, for example, Figure !).

Videoconferencing

A reluctant uptake of the videoconferencing service might be due, among other things, to the fact that too many aspects of real face-to-face meetings are lost with conventional TV equipment. It is expected that this drawback can be reduced by employing techniques that allow the display of approximately life-sized three- dimensional images of the conferees and their surround- ings with adequate spatial resolution along with individual perspective adjustment and motion parallax. Displaying the proper perspective for each conferee ('view-per-person' principle 5) permits individual conferees on the other side to be addressed by non-verbal signals, e.g. by eye contact.

For subjective evaluation experiments a 3D display system has been set up in our laboratory. The final version will include individual stereoscopic channels with motion parallax for two subjects. This set-up provides four separate display channels by means of a combination of polarizing filters (two separate views for subject 1 and subject 2, simultaneously displayed on a rear projection screen) and shutter glasses (to separate left- and right-eye views for both subjects). In order to reproduce motion parallax, head-tracking systems determine the individual viewing positions and control the viewpoints of the stereoscopic cameras accordingly.

Displays Volume 14 Number 3 1993 151

Human factors of 3D displays: S Pastoor

(This experimental set-up is certainly not intended for real videoconferencing.)

3D versions are currently being tested and assessed in comparison with 2D-(HD)TV versions by samples of users performing tasks which are considered to be representative of future videoconferencing situations (collaborative decision making, negotiating, maintenance and repair, joint editing).

First results, based on data from questionnaires and interviews, showed that the telepresence effect (i.e. the impression of sharing space with the conferees at the remote site) is substantially enhanced by stereoscopic display 6. The experiment also underlined the importance of life-sized presentation of the conferees, which has some bearing on the display size needed for videoconferencing.

BASIC REQUIREMENTS

When picture quality is improved in advanced display systems by removing the bulk of conventional reproduc- tion errors, users may soon discover shortcomings that have gone unnoticed so far. When, for example, depth reproduction is improved by stereoscopic methods, any discrepancy between the apparent size of displayed objects and their familiar size may irritate the viewers. This effect is expected to occur due to the increased stimulation of the visual size-constancy mechanism. We have investigated whether size-constancy had an influence on the screen size required for the realistic representation of a natural s c e n e 7.

Picture size

Under normal viewing conditions the perceived size of an object remains fairly constant irrespective of its distance and the visual angle that it subtends. The basic mechanism evidently exploits the real-life fact that any change in distance results in a corresponding change in angular size. Consequently, it is to be expected that a

displayed object will tend to appear unnaturally large or small, if its angular size and its distance do not correspond as usual. In contrast to conventional 2D presentations, where the 'distance' is fixed by the screen distance, for 3D displays both parameters 'angular size' and 'distance' may be brought into correspondence by adjusting the stereoscopic depth accordingly (Figure 2; ideally, the angular size and stereo distance should match real-life conditions).

In our experiments, we used a 21in 3D display which we expected to be representative of future multimedia desktop applications. Typical videophone scenes (e.g. portrait views) were captured at various camera distances and displayed at short and long viewing distances (0.7 versus 2 m, for more details see Reference 7). The stereoscopic parameters were set either to match real-life conditions or to reduce screen parallaxes to a minimum.

The second condition is usually applied for stereo displays in order to keep disparities in a moderate range, but it makes the stereoscopically seen object appear close to the screen distance. Subjective evaluations showed that the displayed scenes do appear miniaturized under this condition, in cases where the camera distance is greater than the screen distance. On the other hand, these size distortions were ruled out as expected when the stereo parameters matched real-life conditions.

Unfortunately one cannot conclude from this result that even small-sized screens are suited to display objects without perceptual size distortions. Evidence was obtained that viewing comfort is substantially reduced by the excessive disparities that may result from the adaptation of stereoscopic distances to real-life distances. Under this condition, the strong mismatch of accom- modation and convergence of the lines of sight, when fixating distant objects, as well as the excessive depth discontinuity at the border of the screen put unacceptable stress upon the visual system. Hence, we have concluded that larger screens are required for videophone displays so that portrait views can be presented without substantial scale reduction. Follow-up experiments will include a variation of the screen size and the picture contents

~ g ~ e 2

Eyes Screen

"~:

< > < >

Stereo distance Stereo distance Expected effect of the horizontal parallax between the left- and right-eye views on the apparent size of a stereoscopically displayed object

152 Displays Volume 14 Number 3 1993

Human factors of 3D displays: S Pastoor

CCIR Impairment rating

imperceptible f

perceptible, but not annoying I

slightly annoying / _ _ ~ , 1 LI

annoying 3 dB

very annoying :~s ~o 3'5 4b 4'5 go

Signal-to-noise ratio [dB]

Figure 3 Impairment ratings of visual noise in (O) 2D displays and (O) 3D displays (uncorrelated left- and right-view noise signals)

(including picture material having close resemblance to TV programmes) in order to derive conclusions for a broader range of prospective applications.

VISUAL N O I S E R E Q U I R E M E N T S

TV signals are generally affected by noise interference resulting from a variety of sources including camera targets and amplifiers, recording media, as well as coding procedures applied for storage and transmission. Conse- quently, the prevention of visual noise ranks among the classical problems of television engineering. Regarding 3DTV, an exploratory study s has demonstrated that the subjective effect of visual noise is dependent on the degree of statistical correlation between the left- and right-view noise pattern (decreasing effect of noise with decreasing correlation). Obviously, the visual system is able to filter out random discrepancies between left- and right-eye images in order to concentrate on the meaning- ful information. We found that the signal-to-noise ratio of a stereo image w~th uncorrelated noise patterns can be approximately 3 dB lower than that of an iso-quality

Threshold peak-signal-to-noise ratio [dB]

30" a correlated " - ~ - - - ' " " ' ~ ]b , / ' - - " . . . . •

............ : 20 monocular ".. uncorrelated

°o . ° ° • . •

Ibrig ht side I 15 i i I I i

-15 -10 -5 0 5 10 15 Distance from edge [minarc]

Rgnre 4 Masking of a dynamic noise bar as a function of its distance from a steep luminance change. The visibility thresholds shown are those peak-signal-to-noise ratios inducing a level of 75% correct responses in a forced choice task. The diagram was obtained when the noise was present in the (a) dark and Co) bright side of the luminance step

two-dimensional television picture (Figure 3). This con- clusion has been supported by a recent experiment focus- ing on visual masking effects caused by steep luminance changes (i.e. edges) in stereo images. The experiment showed that similarly shaped masking curves are obtained for correlated, uncorrelated and monocular noise (noise present in only one image of a stereo pair). Overall, evi- dence was obtained that monocular noise is less visible than binocular noise, and that binocularly uncorrelated noise is less visible than correlated noise (Figure 4).

Disparity range

Under natural viewing conditions the range of disparity values, for which binocular fusion is possible, corresponds fairly well to the depth-of-focus range of the human eye. As a result, excessive retinal disparities generally appear in conjunction with blurred retinal images, an effect that helps to suppress annoying double images. On the other hand, current 3D display technologies require the viewer to accommodate on the screen surface in order to perceive a sharp image. This way, screen parallaxes may produce excessive disparities that are imaged on the retinas without blur. To avoid eyestrain the displayed disparity gamut has to be limited, e.g. by controlling the optogeometrical camera parameters (focal length, stereo convergence and interaxial separation) or by reducing parallaxes with signal processing methods. The range of 'comfortably viewable' disparities is currently being investigated, taking into account the influence of par- ameters such as the depth-of-focus of the pictures (range and position), screen brightness (affecting the depth- of-focus of the viewer's eyes), and viewing distance.

In a preliminary experiment 9 subjects assessed viewing comfort when watching a series of stereoscopic present- ations with a wide range of disparity and depth-of-focus

CCIR Impairment rating

imperceptible

perceptible, but not annoying

slightly annoying

annoying

very annoying

1 c/deg

2.7 e / d ~

1.5 c/deg 5.6 c/deg 3 c/deg

I I l l

0 35 70 105 140 Disparity [minarc]

Figme 5 Subjective assessments of stereoscopic pictures with different magnitudes of disparity at five levels of depth-of-focus (cut- off frequencies in cycles per degree)

Displays Volume 14 Number 3 1993 153

Human factors of 3D displays: S Pastoor

conditions. The subjects were asked to fixate a real target mounted in the centre part of the screen, while slides showing a flat background (a geometric style painting) were rear projected with a stereo projector. By mounting pairs of slides with different horizontal offsets in stereo slide frames, the background appeared at different depth positions. Thus disparity levels ranging from 0 to 140 min(arc) were realized when fixating the target. Each of the disparity levels were realized for five levels of picture sharpness (cutoff frequencies of 1 to 23 cycles per degree).

The traditional 70 min(arc) rule-of-thumb was con- firmed by this experiment, as larger disparities led to a marked drop in comfort ratings (Figure 5). Only in conditions with heavy blurring of the background (i.e. in conditions where the resulting disparate retinal images were extremely unsharp) did disparities of up to 140 min(arc) not deteriorate viewing comfort.

D i s p a r i t y re so lu t ion

The human ability to perceive spatiotemporal changes in binocular depth is strongly limited compared to the resolving power for visual pattern in a frontoparallel plane (contrast sensitivity function). This effect can be exploited by stereo video-codecs in order to reduce the data rate required for storage and transmission of the disparity signal. In Figure 6, the range in which binocular depth perception is possible, is schematically outlined as a function of the magnitude of the disparity signal and its spatial and temporal frequency.

Our recent experiments have been focusing on the maximal frequencies allowing depth perception (absolute upper frequency limits, see Figure 6). We have found that the upper temporal frequency limit is approximately 6 Hz (measured with sinusoidal random-dot disparity gratings at a disparity magnitude of 12 min(arc)). The

Temnnml o=ol,,_t,.,.n=,=l, freqt

[~-

• ~ trequency Ic/oeg]

Figure 6 Outline of the disparity range allowing perception of binocular depth and motion-in-depth. ( ~ ) , Threshold measure- ments carried out in our recent experiments

upper spatial frequency limit was found to be at about 4 cycles per degree and a disparity magnitude of 4 min(arc), a value about ten times smaller than the visual cutoff frequency for frontoparallel patterns.

In another experiment we found that the disparity signal may be linearly quantized with a step size corresponding to 0.8 rain(arc), without causing visible quantization errors in the reconstructed stereo pair 1°.

M O T I O N P A R A L L A X

Conventional stereoscopic systems provide a fixed pair of views corresponding to the fixed viewpoint of the stereo camera. Consequently, motion parallax information is missing and, more dramatically, the perceived shape of the stereoscopic space (and of any displayed object) is distorted if the viewing position deviates from the centres of perspective of the displayed stereo pair. Lateral head motion makes object points in front of the screen apparently follow the viewer, whereas points behind the screen shift in the opposite direction. To avoid these unnatural visual effects multiview displays are required that reproduce proper perspective views corresponding to the observer's actual viewpoint. It is to be expected that such a display must correctly react upon any lateral, vertical and frontal head motion 1. Approaches to multi- view displays currently include autostereoscopic systems simultaneously providing a limited set of (e.g. six to 24) views with spatially fixed centres of perspective 1~ (this means that viewers have to position their heads within fixed viewing zones in front of the display); alternatively, head-tracking systems have been proposed that sense the viewpoint of the observer(s) and control image output accordingly. Head-tracking systems can be designed for operation with and without eyeglasses as well.

V i e w p o i n t s a m p l i n g

Our human factors studies on multiview systems have shown that a comparatively high density of different

CCIR Qual i ty rating

excellent -

good

fair

poor

bad ~/

o'.s ~ ~ ~ 1'o 2t) Maximal paral lax shift [minarc]

Plgure 7 Quality rating of multiview 3D displays as a function of parallax shifts at view changes

154 Displays Volume 14 Number 3 1993

CCIR Impairment rating imperceptible original stereo pair

perceptible, but not annoying ~ ] Iocall

[asymmetry at rest; [ ~, slightly annoying ~ ° b a l asymmetry I N

I locally confined , ] ~ I asymmetry m m°ti°nl ~ annoying

very annoying

o12 o'.s ; ~ ~ t'o 2'0 Left-view-to-right-view luminance ratio [dB]

Figure S Impairment ratings for local and global luminance asymmetries in a stereo pair

perspective views is required to produce smooth motion parallax. We found that lateral motion-parallax quantiz- ation errors should be kept below a threshold value of approximately 1 rain(arc) (see Figure 7, for more details see Reference 12). Thus, when the reproduced depth volume subtends, for example, 20 min(arc) disparity for a 65 mm eye point separation (which is a rather moderate value) about 20 different views per interocular distance must be reproduced to have smooth motion parallax. Simultaneous projection of such a large number of views is unrealistic for technical reasons. Head-tracking systems can meet this requirement more economically.

Brightness constancy

Another point in favour of head-tracking methods is that, with fixed-viewpoint systems, head motion leads to (moving) local or global changes in perceived picture brightness. Generally, these artefacts are different in both eyes causing binocular rivalry. Visual sensitivity for such imbalanced picture brightnesses has been found

ii! i

Human factors of 3D displays: S Pastoor

Lentieular screen

Projel wer

Figure 9 Addressing of the viewer's eye(s) in head-tracking autostereoscopic displays can be accomplished by a moving projector

to be very critical ~3'~4. Empirical results showed that the luminance transfer functions of the left- and right-eye views may differ by 3 to 6 dB, when the resulting lumin- ance asymmetry pattern is stationary, while even a 0.2dB difference is perceptible when the pattern is moving (see Figure 8). Any attempt to reduce brightness changes in fixed-viewpoint systems produces increased crosstalk between adjacent views.

Registration tolerances

A conceivable approach to head-tracking autostereo- scopic systems could utilize a rear projection screen with lenticular sheets on both the front and back sides; any head motion of the viewer would then require a corre- sponding (lateral or frontal) shift of the stereo projector (see Figure 9). Hence, picture registration of left- and right-eye views must be guaranteed for a moving pro- jector arrangement. Tolerances for registration errors in stereo pairs have been established in psycho-optical experiments, indicating that vertical registration errors (errors of height) should not exceed a limit of 3 min(arc) and horizontal errors should be kept below 10 min(arc) 15.

Figure 10 Stereo pair and interpolated intermediate view for multiview 3D displays

Displays Volume 14 Number 3 1993 155

Human factors of 3D displays: S Pastoor

Perspective interpolation

In many projective 3D applications it will be impossible or undesirable to shift the camera position mechanically according to viewpoint changes of individual viewers (e.g. in broadcast TV). In such applications image-processing techniques are needed to enable motion parallax. This approach requires a high-precision stereo analysis of the spatial structure of the scene (based on the images captured by a fixed set of, for example, three to five cameras) followed by the reconstruction of images for any virtual viewpoint within the real-camera array (see Figure 10; methods developed at our laboratory are presented in References 16-18).

Perspective interpolation techniques are complicated by the fact that parts of the scene are half-occluded in the original set of camera images because of the different camera standpoints. Stereo analysis algorithms generally fail to provide a correct depth map for half-occluded regions as there is no stereo correspondence. Hence, there is the problem of how to define these parts in the reconstruction process. Our search for adequate algor- ithms is guided by results of current human factors studies on the visual perception of half-occluded regions.

Another problem is the reconstruction of smooth edges. Random errors in the stereo analysis process may create irregular image distortions which the observer will notice as edge-jitter. The visibility threshold for such impairments has been found to be in the order of 1 min(arc). The threshold increases with decreasing con- trast and decreasing spatial frequency of the impaired image pattern 19. Smoothing the edges requires complex morphological filters eliminating outliers but preserving shape.

As mentioned above, noise requirements on stereo- scopic images are generally reduced compared with flat images. However, noise in intermediate views differs from the noise signals produced by different cameras in that it may be highly correlated due to the reconstruction process. Therefore, noise requirements in multiview systems are expected to be the same as in conventional 2D systems.

Perspective interpolation techniques are also being developed with regard to multipoint videoeonferenelng. For instance, an individual eye-contact facility can be realized by (dynamically!) positioning the virtual camera standpoints close to the individual eye positions of the displayed conferees. Moreover, 3D-image analysis is necessary for a telepointing facility in 3D video- conferencing, if occlusions of the telepointer by opaque objects are to be simulated.

C O N C L U S I O N S

3D techniques will have a broad application spectrum. When used in future image communications systems, specific requirements must be met in order to compete

156 Displays Volume 14 Number 3 1993

successfully with advanced 2D systems like high- definition TV. It seems necessary to define evaluation criteria appropriate to the anticipated application scenarios and to judge candidate systems against these criteria. In contrast to applications where 3D techniques help to improve task performance (e.g. fault diagnosis, remote control and visual inspection tasks), objective measures (such as the time needed to solve the task and the error rate) are not as important as subjective criteria. For example, subjective picture quality and viewing comfort should by no means fall behind standards set by competitive 2D systems.

When current concepts for 3D displays are assessed under this premise, the result sometimes sharply contra- dicts the enthusiastic statements of their inventors. So, for example, autostereoscopic concepts forcing the viewer to find and keep a fixed head position for undistorted viewing are certainly not suitable for advanced image communications. The same applies to multiview auto- stereoscopic systems that offer, on one hand, an extended viewing zone (i.e. some freedom for head movements) by a multitude of simultaneously displayed views but, on the other hand, introduce severe degradations of picture quality such as viewpoint-dependent crosstalk and brightness artefacts.

The technological progress required to develop a display that provides practically unimpaired stereoscopic space perception including motion parallax is a strong challenge. For the medium term, autostereoscopic techniques using double-sided lenticular rear projection screens seem most promising. In this approach, optical head-tracking devices would control the process of optically addressing the viewer's eyes by a corresponding shift of the stereo projection unit. By arranging several independently movable projection units, this system could in principle be used simultaneously by more than one viewer. Certainly, for the longer term, research should strive for display techniques without moving mechanical parts.

A C K N O W L E D G E M E N T S

The work reported here has been supported by grants of the Research and Technology Department (BMFT): of the Federal Republic of Germany and partly by the: Commission of the European Communities (RACE Project 2045 DISTIMA). An earlier version of this article was presented at the IEE Colloquium on Stereoscopic Television, 15 October 1992, and has been published in the IEE Digest 1992 p. 173. The author is solely responsible for the contents of this paper.

R E FE R E N C E S

The papers cited below refer exclusively to human factors studies carried out at the Heinrich Hertz Institute. Further literature on stereoscopic vision is mentioned in the cited papers.

Human factors of 3D displays: S Pastoor

1 Boerger, G 'Human factors problems with 'VisionLike TeleVision'. (VLTV)' Proc. 12th Int. Display Research Conference (Japan Display "92), Hiroshima, Japan (12-14 October 1992) pp 13-16

2 Pastoor, S '3D-television: a survey of recent research results on subjective requirements'. Signal Processing - Image Commun. 1991, 4(I), 21-32

3 W6pking, M 'Subjective assessments of 3D vs. 2D presentation of motion pictures for large screen home television.' Proc. Ist Int. Symposium on 3-D Images, Paris, France (26-28 September 1991) lap 51-58 (published by Festival 3D, 41 rue Barrault, 75013 Paris, France)

4 W6pking, M 'Towards a comprehensive assessment methodology for psychological effects of high-resolution wide-field displays'. Proc. 13th Int. Symposium on Human Factors in Telecommun- ications, Torino, Italy (10-14 September 1990) pp 579-591

5 Miihlbach, L, B6cker, M and Prussog, A 'Communicative presence in videoconferencing'. Proc. 4th Int. Symposium on Human Factors in Telecommunications, Darmstadt, Germany (11-14 May 1993) Decker, Heidelberg, pp 213-224

6 Prussog, A, Mfflalbaeh, L and B6cker, M 'Telepresence in stereo- scopic videoconfereneing'. Proc. ddh Int. Symposium on Human Factors in Telecommunications Darmstadt, Germany (11-14 May 1993) Decker, Heidelberg, pp 201-211

7 Komatsu, T and Pastoor, S 'Subjective evaluation of the puppet theater effect with TV sized stereoscopic displays'. Heinrich Hertz Institute, Internal Report (1992)

8 Pastoor, S and Beldie, I P 'Subjective assessments of dynamic visual noise interference in 3D TV pictures'. Proc. SID 1989, 30(3), 211-215

9 W6pking, M 'Viewing comfort with stereoscopic pictures-a study on the subjective effects of disparity magnitude and depth- of-focus'. Heinrich Hertz Institute, Internal Report (1993)

10 Kost, B and Pastoor, S 'Visibility thresholds for disparity quantization errors in stereoscopic displays'. Prec. SID 1991, 32(2), 165-170

11 B6rner, R 'Autostereoscopic 3D imaging by front and rear projection and on fiat panel displays'. Displays 1993, 14, 39-46

12 Pastoor, S and Sehenke, K 'Subjective assessments of the resolu- tion of viewing directions in a multi-viewpoint 3D TV System'. Prec. SID 1989, 30(3), 217-223

13 Beldie, I P and Kost, B 'Luminance asymmetry in stereo TV images'. Prec. SPIE 1991, 1457, 242-247

14 Beldie, I P 'Luminance asymmetry in stereo TV images lI'. In Parameters of Picture Quality for HDTV and 3DTV (ed S Pastoor) Final Report, Heinrich Hertz Institute Berlin, Berlin, Germany, 1991, pp 146-156

15 Beldie, I P and Baumgart, H 'The effect of geometric distortions in stereoscopic images on viewing comfort'. Prec. Les Jour~es de Rennes: La t~l~vision en relief, Soci6t6 des Electrieiens et des Electroniciens Publications, Paris, 1990 (in French)

16 Skerjanc, R and Liu, J 'A three camera approach for calculating disparity and synthesizing intermediate pictures'. Signal Processing -Image Commun. 1991, 4(0, 55-64

17 Liu, J and Skerjanc, R 'Construction of intermediate pictures for a multiview 3-D system'. Proc. SPIE 1992, 1669, 10-19

18 Liu, J and Huang, S 'Using topological information of images to improve Stereo Matching'. IEEE Conference on Computer Vision & Pattern Recognition, New York (15-17 June 1993) (poster presentation)

19 W6pking, M, Beldie, I P and Pastoor, S 'Subjective effects of displacement errors in electronically processed stereo-television pictures'. Spat. Vis. 1988, 3(1), 45-72

Displays Volume 14 Number 3 1993 157