Super-Resolution Composition in Multi-ProjectorDisplays

Christopher Jaynes and Divya Ramakrishnan

Metaverse Lab, Dept. of Computer ScienceUniversity of Kentucky

jaynes@metaverselab.org

AbstractRecent advances in multi-projector research have madelarge-format tiled display walls feasible. Resolution of thetiled display is defined by the resolution of the componentprojectors and the tiling. In domains where display surfacesize is constrained or the number of pixels per unit area onthe display must be increased, traditional tiling may not befeasible. We propose a multi-projector method that exploitsoverlapping regions to achieve increased resolution andcontrast on the display surface.

Given a set of overlapping projectors and a target image,component images are derived and then rendered by eachprojector resulting in an image on the display surfaceexceeding the resolution limits of the individual projectors.The relative viewing geometry between the projectors i staken into consideration while computing the componentimages that should be projected. Sub-pixel shifts betweenthe displays are viewed as integer disparities in the high-resolution target image. Results demonstrate that for avariety of target images, there is perceptible resolutionimprovement on the display surface. Super-resolution imagecomposites have a significantly higher sampling rate on adisplay surface without a corresponding increase in displayarea.

1. Introduction

This work introduces the Super-Resolution Compositionproblem for multi-projector display and presents an initialapproach to achieving Traditional multi-projector high-resolution is achieved by tiling component projectorstogether, calibrating their relative geometry [4,8,19,21],potentially their radiometric differences [15,24], andrendering a coherent image across all projectors. Due tosignificant advances in automatic calibration, tiled displayis feasible, very large-format displays are fairlystraightforward to deploy in unconstrained environments,and novel tiled configurations are being explored [25].

This work focuses on the super-resolution composition(SRC) of multiple overlapping and calibrated projectorfrustums. In traditional multi-projector displays,accidental overlap occurs quite frequently and SRC can beused to increase resolution and display contrast in theseregions. For displays that require high-resolution but

have insufficient surface area to support tiling, projectorscan be fully overlapped to exploit both the increasedresolution and contrast that SRC affords.

Recent research advances in multi-projector displays bothfrom the computer vision and graphics communitiesinspire this work [5,9,10,11,12,14,26]. Efforts havefocused on building large-scale displays from clusters ofprojectors that automatically calibrate [4,12,20,22], andcorrectly render a single, coherent tiled image[5,10,12,14,19]. The work here is an alternate approach toachieving higher resolution display from multi-projectors.

Multi-projector displays are loosely configured and mayhave significant regions of overlap. Several researchershave addressed these overlapping regions to provideintensity blending and color uniformity [15,19,24].Recently, these regions have been exploited to activelyremove shadows and provide increased contrast ratio[3,16].

Indeed, regions of the display surface illuminated by morethan one projector can be viewed as an OpticalFramebuffer in which operations may take place byappropriately adjusting images in the different projectors.This Framebuffer Optique, and its potential use forintensity and contrast enhancement, as well as cooperativerendering of linear computer graphics operations (e.g. lightsource compositing) was first noted by [16]. This paperdemonstrates that the Framebuffer Optique can beexploited to produce high-resolution images bycompositing low-resolution components.

Solution to the SRC problem has direct utility in multi-projector display environments as is demonstrated here.Images that exceed the resolution limits of a singleprojector can be more accurately visualized using multipleprojectors and the SRC method without increasing displaysize. This dramatically increases the sampling rate of therendered images on the display surface to betterapproximate the fidelity of the human eye. High-resolution insets in a projected display are also madefeasible through this technique.

The technique requires at least two overlapping projectorswhose view frustums illuminate a planar display surface.

During an initial calibration phase, a camera is used torecover a homography between each projector and a baseprojector frame. A base projector frame is aligned withthe high-resolution space and only differs in its resolution.Sub-pixel overlap defines pixels in the high-resolutiontarget frame. Figure 1 depicts this situation.

Because each projector is related to the target frame via ageneral homography, the relative two-dimensional shiftand sampling rate will change across the display. Thecomponent homographies are approximated by a set oftwo-dimensional sub-pixel shifts (referred to as the linearshift matrix) that represents the sub-pixel disparity of oneprojector with respect to the target image reference frame,for a given image sub-region.

Component images are then estimated in the frequencydomain where the target image is sub-sampled and phaseshifted according to the sampling rate and shift matrix foreach component. The resulting amplitudes and phasevalues are then optimized according to a cost function thatmeasures the difference between the target image and theimage that results from adding the multiple sub-sampled,shifted components together.

2. Previous and Related Work

Super-resolution composition is directly related to asignificant research history on super-resolutionreconstruction from image sequences. Many of thetechniques described here are related to the super-resolution reconstruction problem and are closely relatedto the frequency domain approach [2,7,18,23]. For aconcise overview of the more common approaches tosuper-resolution reconstruction, the reader is referred to[2,6].

Although multi-projector display systems are the focus ofan active research community, there has been relativelylittle work related to exploiting projector overlap. Initial

research focused on computing overlap regionsautomatically from camera-based calibration techniques[4,12,19,22] and monitoring active displays for reactivecalibration [21]. Other methods seek to address overlapregions through intensity blending [9,19], geometriccorrection [4,12,14,19,22], and color balancing [15,24].These techniques are important in that loosely configured,automatically calibrated tiled displays are almost certain tocontain regions of projector overlap that must beaddressed. In contrast to these efforts, we exploit theutility of overlapping regions rather than seek to removeor attenuate their effects.

Only recently, methods for exploiting projector overlaphave been introduced. Detection of radiometric changeson the display surface can be used to intensify or attenuatedifferent projectors to actively remove shadows on aninteractive display [3,13]. Welch. et. al. demonstratedthat the rendering problem can be decomposed into severalcomponents and added together on the display surface tosupport distributed computation of lighting and renderingeffects [16]. The Framebuffer Optique is described in thesame work as a virtual Framebuffer that can be accessedvia manipulation of the component images and addedtogether through image superposition. We demonstratehow the Framebuffer Optique can be exploited to producehigh-resolution image overlay by solving the super-resolution composition problem.

3. Image Decomposition and Compositing

There are three steps in deriving an appropriate set ofcomponent images. First, each projector must becalibrated to the target reference frame, in order todetermine the relative sub-pixel shift between eachprojector and that frame. Once calibrated, an initial set ofcomponent images is derived via the known shifts andsub-sampling rates for each component. This phase ofthe algorithm can be performed in parallel for eachprojector. Given these initial estimates, a globaloptimization step minimizes the difference between thesum of the components and the target image.

3.1 Calibration

The goal of calibration is to derive an accurate mappingfrom each projector’s framebuffer coordinates to the high-resolution target frame. This mapping must be accurate toless than a pixel and presents a significant challenge inpractice. In order to compute the sub-pixel disparitybetween projectors and a target frame, pixelcorrespondences between any two projectors must becomputed to far less than a pixel accuracy and nonlineareffects due to both camera and projector optics must beknown and corrected.

The epipolar relationship between pairs of projectors and aknown surface model is sufficient to compute pixelcorrespondences for every pixel in the Framebuffer. In

(Dx, Dy)

Figure 1: Multiple projectors illuminate a planar surface.A single projector defines the Framebuffer Optique,whose high-resolution pixels (shown in circle) aredefined by the overlapping sub-pixel shiftedcomponents.

the results shown here, however, the display surface isconstrained to be planar and the full projective relationshipbetween any two devices, i and j can be modeled as a 3x3homography matrix that maps pixels in projector j directlyto pixels in projector i, through the display plane. Thehomography can be automatically computed given asufficient set of matchpoints between the two projectors inquestion.

Prior to estimating the calibration matrix, the radialdistortion coefficients, k1 and k2, of the camera and allprojectors are independently estimated in an offlineprocess. Prior to estimating the linear homographybetween each camera and projector, points in each deviceare unwarped using the recovered distortion coefficients tocompute point positions in an undistorted frame ofreference. Recovered homographies, then, map pointsfrom one undistorted frame to another.

Matchpoints are then computed between a camera thatobserves the display surface, and each projector in thedisplay. For each projector, a set of Gaussian targetfiducials centered at randomly selected framebuffer pixelsare iteratively displayed. The target is captured in thecamera and a match-point pair, using the undistortedcoordinates, is stored. The sub-pixel location of the targetin the camera is computed through an optimization routinethat estimates seven parameters of a distortinghomography that is specific to each matchpoint pair. Wehave shown that this technique can recover match-points atclose to 1/5th pixel accuracy and the reader is encouragedto read [20] for more details and analysis of the subpixelmatchpoint finding algorithm.

Given a sufficient number of correspondences (25 for theresults shown here), a homography between each projectorp and the display camera c, Hp

c, is then computed usinglinear least squares estimation. These homographiesrepresent the mapping between each projector and thecamera used to calibrate the display.

Each projector to camera matrix is then converted into ahomography between each projector and the target imagereference frame. An arbitrary projector Framebuffer isselected as the base projector reference frame. Althoughthe target frame and base projector Framebuffer are ofdifferent resolutions (the target is significantly higher),this projector defines the target space up to an unknownscale by assuming that the base Framebuffer is axis-aligned with, and shares the origin of the target imagespace. Therefore, the relationship between any projectorand this target image frame, Hb

i, can be written as acomposite homography from the projector (i) to thecamera (c), and then to the base projector frame (b):

†

H ib = Hb

c( )-1

H ic (1)

This situation is depicted in Figure 1. Projectors overlapon the display surface and are each shifted with respect tothe base frame. Shifted overlapping pixels give rise to thehigher resolution space where sub-pixel shifts define

integer pixels in the high-resolution target frame. Forresults shown here we assume the sub-pixel calibrationaccuracy up to 1/4th of a pixel resulting in resolutions inthe target frame that are 4-times those of componentprojectors. We assume a constant sampling rate from oneprojector to the target frame. Although generalconfiguration of projectors will lead to non-uniformsampling rates (particularly in cases of extreme off-axisprojection and skew), a mean sampling rate is a goodapproximation for common projector setups.

In order to demonstrate the calibration accuracy that can beachieved under controlled conditions using accurate,undistorted camera matchpoints, a homography wasderived between two overlapping frustums. A high-resolution image was then captured very close to thedisplay surface using a digital still camera. Figure 2shows two 4x4 pixel grids drawn on the high-resolutionimage corresponding to the two overlapping projectors.The white grid, corresponding to projector 1 pixelboundaries, was drawn by hand for demonstrationpurposes while the black grid was defined by thehomography from projector two’s frame to the frame ofprojector 1. The white circle on the image depicts thecenter pixel p in the framebuffer of projector 2. This userestimated center can then be compared to the sub-pixeldisparity predicted by the calibrated homography (centersof the black squares). In this case, calibration predicted asubpixel shift of p with respect to q of (0.09,-.12) whileinspection reveals a shift of projector 2 with respect toprojector 1 of (0.13,.11).

Figure 2: Calibration accuracy for overlapping projectorfrustums. White lines show pixel boundaries for baseprojector. Circles correspond to pixel centers estimated byhuman subject while center of actual grid predicted byhomography (shown as black lines) is approximately off by1/5th of a pixel.

According to our formulation, the low-resolutioncomponent images are modeled as sub-sampled versionsof the image target with a uniform two-dimensional shiftand sampling rate with respect to the target frame ofreference. Clearly, the projective warp between twoprojectors describes a more general displacement thanuniform shift. In order to derive an appropriatecomponent image, without the undue computation that aper-pixel solution would require, the homography can beapproximated by a set of two-dimensional shift vectorsbetween a projector and the target frame.

The homography between projector i and the base frame isdecomposed into a linear shift matrix that represents two-dimensional offsets between the projector and the basereference frame. Each entry in the shift matrixcorresponds to a region in the target reference frame forwhich the two-dimensional offset is assumed to beconstant. Once computed, the shift matrix replaces themore general homography and regions in the componentframe are related to the target frame through a constant,two-dimensional offset. We present a method forconstructing a linear shift matrix Si

B from a givenhomography Hi

B given target error tolerance e.

The two-dimensional disparity D , between a componentprojector reference frame r and target frame b is written asthe difference between the locations of a pixel in frame rthe same pixel in frame b (given by the knownhomography).

†

D = p - Hrb p (2)

The disparity in x and y directions (Dx,D y) isindependently given by:

†

D x = px -H1px

H3 px

D y = py -H2 py

H3 py

(3)

where Hk is the vector formed by the kth row of the matrixHr

b. If we assume that D x and D y are independent, thedisparity value is a linear function of the pixel position inthe component frame.

As x ranges from zero in the component projector to xmax,the resolution of projector in the x direction, the disparityvalues will vary in accordance with the line equation givenby Equation 3. This line is divided into k equal regionssuch that the disparity values in the region are all withinex of one another. Conceptually, these k regions arecolumns in the component image that will use the sameapproximate x-shift values, ~Dx, for the purposes ofderiving the component image corresponding to pixelscontained in that column.

Given the line equation for independent disparities in the ydirection (Equation 3), a similar process divides thecomponent frame into rows of uniform y-disparity witherror tolerance ey. These regions are combined to produceregions in the component image containing approximatevalues for two-dimensional shifts that are within

†

e £ ex2 + ey

2 of the values represented in the actual

homography. Therefore, for a given error tolerance e (fixedto be .2 pixels for the results shown here), thehomography can be decomposed into areas of uniformdisparity. These region-based two-dimensionalapproximate shifts, (~Dx,~Dy), and the corresponding

offset of the region itself, (Ox,Oy), are used to derive the acomponent image for a single projector. It should benoted, that, as error tolerance approaches zero, the region-based two-dimensional shifts simple become the twodimensional shifts given on a per-pixel basis by thehomography itself and no accuracy is lost. We select anerror tolerance that reduces computation while stillretaining a perceptually good result.

Figure 3 shows ten regions corresponding to a 5x2 shiftmatrix, computed from the homography between aprojector to the base frame for a two-projector setup.Given a fixed error tolerance, the number of regionscomputed by this process is related to the amount ofprojective (off-axis) distortion induced by the viewinggeometry.

3.2 Component Image Estimation

Component image estimation is performed in thefrequency domain where initial images are first estimatedand then optimized. We do not construct componentimages in the spatial domain because overlapping pixelintensities and the constraints they represent are difficultto characterize. We are strongly motivated to formalizethe problem in the frequency domain because, aperceptually good image must take into account models ofthe human visual system (HVS). Successful perceptualmodels, in particular those that assign importance toparticular aspects of an image, are defined as weightedfrequency filters. Because the problem is ill-posed (i.e.we can only approximate a target image with a givenprojector setup), these perceptual models must beincorporated into an objective function during theoptimization phase of reconstruction. Although, we donot yet incorporate an HVS model, we expect thatimproved results will require an appropriate model ofperception.

A given target image is first converted to the resolution ofthe Framebuffer Optique, defined by the sub-pixel shiftpattern recovered in the calibration phase. This targetimage I(x,y), is converted to a corresponding discreteFourier transform (DFT) FT(u,v).

A component image for a particular projector is estimatedin two phases. First, sub-components for the n regions ofuniform disparity are derived. Following that, sub-components are combined to form a single componentimage for the projector.

Figure 3: Automatic decomposition of a homography intoregions of uniform two-dimensional disparity. White linesdenote boundaries of uniform 2D displacement.

The target image DFT is sub-sampled at a rate of Rx andRy based on mean sampling-rate derived from thecalibration phase for a particular component projector withrespect to the target frame. The DFT of the target imageFT(u,v), and the sampled component image, Fs(u,v), arerelated via aliasing:

†

FS[u,v] = a FTk

MRx

+ pfsx,l

NRy

+ qfsy

Ê

Ë Á Á

ˆ

¯ ˜ ˜

q= 0

M -1

Âp= 0

N-1

Â(4)

where fsx=1/Rx and fsy=1/Ry are the two-dimensionalsampling rates in x and y directions. Again, non-uniformsampling rates can be supported by this framework andcan be estimated on a per-region basis in a manner similarto that used to estimate two-dimensional disparity.Future work will address this issue.

The sub-sampled DFT signal is shifted, for each region r,by the corresponding entry in the Linear Shift Matrix,(~Dx

r,~Dyr), plus the offset (Ox

r,Oyr) for that region. The

shifting property of the Fourier transform relates spatialdomain translations to a phase shift in the frequencydomain as:

†

FD r[u,v] = e j2p ((~D xr +Ox

r )u+(~D yr +Oy

r )v)FS (u,v)(5)

Equation 5 holds only for a stationary signal and shiftingthe signal for a finite region of the target image may resultin invalid frequency coefficients due to boundaryproblems. In practice, a region of size w x h is extendedby the magnitude of the phase shift to include neighboringpixels. For regions on the image edge, the edge values ofthe image are copied into a border whose size is definedby the magnitude of the phase shift. The resulting Fourierseries

†

FD r[u,v] is the frequency space representation of

the sub-sampled, shifted image for a specific region in thecomponent image. Each of the n regions is then composedinto a single Fourier Series using the distributive propertyof the Fourier transform over addition. This results in asingle Fourier series for a component projector, p:

†

FDp[u,v] = FD r[u,v]

r= 0

N

 (6)

3.3 Global Optimization

Once an initial set of component images have beenestimated, they must be adjusted to more closelyapproximate the contents of the target image. There are anumber of ways to formulate the problem of determining

the component image contents given their relative positionto the target reference frame and a target image. Figure 4depicts the theoretical problem to be solved. Pixel valuesfor projector 1, shown in black, must be determined withrespect to the constraint that the total energy supplied byall projectors sums to the expected value for the highresolution pixel under consideration, shown as k1 in theimage. We are exploring several different approachesincluding linear programming and direct optimization ofthe free parameters (see Conclusions). Here we present aheuristic-based iterative relaxation method used for theresults here.

Figure 4: Ideal two-projector composite case. Pixel values a1

and b1 must be determined so that k1=a1+b1. Because thevalue of b1 also influences the value of k2, k3, and k4, the pixelvalues for a single component are not independent.

Component images, computed for each overlappingprojector, are derived independently. These initialcomponents do not take the additive nature of thesuperposition process into account and must be adjustedin a global optimization phase to achieve increasedcontrast. In order to do so, the intensity values containedin the target image at each pixel are multiplied by thenumber of projectors that overlap that pixel. This definesa new target of increased contrast that is approximated bythe projector components during the optimization phase.

Each derived component image for each of the kprojectors,

†

FDp , is treated as the initial estimate for an

iterative algorithm that seeks to minimize the differencebetween the sum of the different components and theimage target. We model the superposition process on thedisplay surface as additive and minimize:

†

FT [u,v]- FDi [u,v]

i= 0

k

ÂÊ

Ë Á

ˆ

¯ ˜

2

(7)

In future work, Equation 7 will be biased by a perceptualmodel that assigns importance to particular frequencyevents in the reconstructed image. The unbiased errormetric, must be augmented by additional constraints if weare to converge to component images at are reasonable.Several constraints are available and we are still exploring

how the choice of constraints at this phase of thealgorithm influences composite image quality. Currently,the outermost projector boundary pixels are fixed to beexactly x/k of the intensity of the high-resolution pixel itoverlaps (For example, pixel a1 in Figure 4) where x is thetarget value and k the number of projectors that overlap thesuper-resolution pixel. This defines a set of pixel valuesthat can no longer be adjusted.

The remaining component image pixels, p, are thenvisited in random order in each image, the largestdifference between the high-resolution target valuescontained within component pixel p and their currentvalues (defined by the sum of the component pixel and itsoverlaps) is corrected.

Correction can either involve directly adjusting the low-resolution component pixel to match the target pixel valueof the largest error exactly or may involve a weightedadjustment of the component pixel intensity. The pixelvalue is adjusted to exactly match the desired value for thetarget pixel if, in doing so, the maximum error of theother high-resolution pixels contained within p do notexceed the previously measured maximum error.

If direct correct if pixel p yields an error that is greaterthan the maximum error contained at p prior toadjustment, the intensity of the component pixel inquestion is adjusted according to, DI = lDt – (1- l)Dt+1.Where, lambda is scalar weight that can dampen localchanges in component images and will ultimately controlthe amount of variation in the component images from theinitial estimates. Dt and Dt+1 are the maximum pixeldifferences between the intensity of the component pixeland any high-resolution pixel it overlaps both before thevalue is adjusted and after.

Starting from the periphery, pixels are iteratively adjustedaccording to the algorithm in all component images thatcan influence the value of a high-resolution target. Thisalgorithm is applied iteratively to all images until eitheruntil it converges and no pixel values are adjusted in asingle pass over each image component or the global errorfunction is smaller than a predefined threshold.Determining this threshold for a given calibration accuracyand target image is a matter of current work and inpractice, the global error measure is observed by a humanoperator who determines when to terminate the algorithmwhen it appears to be no longer improving.

4. Results

The technique was evaluated for two and three projectorscenarios where the projectors were configured to havesignificant overlap and each projector’s optic axis wasoriented to be within approximately 15-degreesperpendicular to the display plane. Distance from theprojectors varied for different test cases but ranged from 6feet to 12 feet from the display. The technique was testedfor 1024x768, 800x600, and 640x480 componentprojector resolutions.

Projectors were calibrated using 25 matchpoints perprojector and the projector to target image homographywas recovered in each test case. Shift matrices, and thecorresponding image regions were derived from thecomponent homographies to within 0.25 pixel error(estimated using a technique similar to the one discussedin Section 3.1).

For each projector, an initial component image wasderived using the techniques described in Section 3.2.Each of these images (still in the frequency domain) wasthen provided to a single machine for global optimization(see Section 3.3). The resulting Fourier series for eachprojector was then converted to a spatial signal by each ofthe k different machines to produce a component imagethat was loaded into the Framebuffer for display.

The time required to derive initial component images isnegligible compared to the time spent during the globaloptimization phase. Optimization time is directly relatedto the number of parameters being optimized (defined bythe number and resolution of the components) as well asthe complexity of the search space defined by the targetimage. Current times preclude using the SRC techniquefor video-rate imagery on a multi-projector display. Weexpect that a closed-form approach to determining theoptimal components given the setup is feasible and is thesubject of future work.

Since the resulting image is only present on the displaysurface and is not represented digitally, quantitativecomparison of the SRC result against the target image isnot possible. Instead we present several examples ofreconstructed images by capturing a high-resolution imageof the display surface (sometimes at very close range) todemonstrate the perceptual result of the technique.

Figure 5 depicts a close-up image of the “pinky finger”taken from the Human Body Dataset. A 2048x2048resolution still image of a human torso (shown fully inFigure 3). The SRC image shown in Figure5b containsgreater contrast, less aliasing at boundaries, whileenhancing high-frequency features such as the point tip onthe fingernail.

(a) (b)

Figure 5: Close-up of pinky finger from 2048x2048resolution image (see entire image in Figure 3). (a) Singleprojector at 1024x768 resolution. (b) Two composite imagescomputed using SRC method. Note that the surface pattern i smore visible in image (b) due to increased contrast andresolution. The sharp tip structure on the fingernail i spresent in (b) and not in (a).

Composite results are perhaps best observed at the pixellevel resulting images on the display surface. Figure 6shows an extreme close-up of a curve from the VisibleHuman dataset. The image corresponds to a vein from animage that is 2148x968 pixels in resolution.

A computer desktop containing text and images at aresolution of 1280x1024, was rendered using a singleprojector at 1024x768 and reconstructed using the SRCmethod. Results are shown in Figure 7.

A close-up view of a section taken from a 2048x2048resolution aerial image dataset shows the improved imagequality that SRC affords. Figure 8 shows a close-up of acurving sidewalk region and several pedestrians taken fromthe “Aerial” dataset experiment. Previously unseenstructure (such as the white shirt) is visible in the SRCimage.

5. Conclusion

We have introduced the Super-Resolution Compositionreconstruction problem and demonstrated how an accuratedecomposition of a high-resolution target image into itsoverlapping, sub-pixel, shifted components can be used toimprove the resolution of a projected display. Resultsshow perceptible improvement for two and three-projectoroverlapping displays given a variety of target images.

In the near future, we expect to implement the SRCsolution in hardware (particularly the Fourier transforms),and devise a more efficient optimization method usingbetter initial estimates. We are also exploring directmethods to recover composite images. Under certainconstrained configurations, the constraints on componentimages can lead to a large and sparse system of linearequations that can be solved faster than the iterativemethod described here. In this way, video and interactiveapplications can be supported on a super-resolutiondisplay. In addition, we are working on faster sub-pixelcalibration techniques that allow users to “steer” projectorsto regions of imagery where high-resolution insets aredesired.

In longer-term research, we are exploring other operationsthat may be able to decompose the target into a set ofcomponent images that are ultimately overlaid in theFramebuffer Optique. Examples include distributedrendering and blending of color and intensity.

6. Bibliography

[1] S. Baker and T. Kanade, “Limits on super-resolution andhow to break them”, Int. Conf. Computer Vision & PatternRecognition, volume 2, pp. 372--379, 2000.

[2] S. Borman and R. L. Stevenson, “Super Resolution fromImage Sequences - A Review”, Proceedings of the 1998Midwest Symposium on Circuits and Systems, Notre Dame, IN,1998.

[3] H. Chen, R. Sukthankar, G. Wallace and K. Li. “ScalableAlignment of Large-Format Multi-Projector Displays UsingCamera Homography Trees”, IEEE Visualization (Vis2002),Oct. 2002.

[4] C. Cruz-Neira, D. Sandin, and T. Defanti, “Surround-screenprojection based virtual reality: The design andimplementation of the CAVE”, Proc. Of ACM Siggrgaph,1993.

[5] G. Demoment, "Image reconstruction and restoration:overview of common estimation structures and problems",IEEE Transactions on Signal Processing, vol. 37, no. 12, pp.2024-2036, December 1989.

[6] M. Elad and A. Feuer, “Restoration of Single Super-Resolution Image From Several Blurred, Noisy and Undersampled Measured Images”, IEEE Transactions on ImageProcessing, vol. 6, no. 12, pp. 1646-58, December 1997.

(a) (b)

(c) (d)

Figure 7: Close-up views of text projected on display surface.(a) Single projector at 1024x768. Sampling rate is too low anddetails (top of asterisk, for example) are lost. (b) Super-resolution composite. (c) Single projector at 1024x768 close-up of word “matrix”. Compare circle smoothness andcompleteness in letters “r” and “i”.

(a) (b)

Figure 6: Close-up view of line (veins) segments from alarger image from the human-body dataset (see Figure 3).Target resolution is 2148x2148. (a) Single projector viewof image. Aliasing of line-segment is a result of downsampling in the projector. (b) Super-resolutionreconstruction.

[7] H. Foroosh (Shekarforoush), J. Zerubia and M. Berthod,``Extension of Phase Correlation to Sub-pixel Registration'',IEEE Trans. Image Processing, vol. 11, Issue 3, pp. 188-200,2002.

[8] M. Hereld, I. R. Judson, R. L. Stevens, “Introduction toBuilding Projector-Based Tiled Display Systems”, IEEEComputer Graphics and Applications, pp. 22-28, 2000.

[9] G. Humphreys, M. Houston, Ren Ng, R. Frank, S. Ahern, P.D. Kirchner and J. T. Klosowski, “Chromium: A StreamProcessing Framework for Interactive Rendering on Clusters”,ACM Transactions on Graphics, vol 21(3), pp. 693-702,Proceedings of ACM Siggraph, July 2002, San Antonio, TX.

[10] G. Humphreys and P. Hanrahan, “A Distributed GraphicsSystem for Large Tiled Displays,” IEEE Visualization, 1999.

[11] C. Jaynes, S. Webb, and R. M. Steele, “A ScalableFramework for High-Resolution Immersive Displays”,International Journal of the IETE, Special Issue on VisualMedia Processing, Vol. 48, No. 3&4, pp. 273-285.

[12] C. Jaynes, S. Webb, R.M. Steele, M. Brown, and W.B.Seales, "Dynamic Shadow Removal from Front-ProjectionDisplays", IEEE Visualization, San Diego, 2001.

[13] Z. Li and A. Varshney, “A Real-Time Seamless TiledDisplay for 3D Graphics”, Proceedings, Seventh AnnualSymposium on Immersive Projection Technology, March 24 -5, 2002, Orlando, FL

[14] A. Majumder, Z. He, H. Towles, and G. Welch, “AchievingColor Uniformity Across Multi-Projector Displays”, 11thAnn. IEEE Visualization Conference, 2000.

[15] A. Majumder and G. Welch, "Computer GraphicsOptique: Optical Superposition of Projected ComputerGraphics", Fifth Immersive Projection Technology Workshop,Seventh Eurographics Workshop on Virtual Environments,Stuttgart, Germany, Springer-Verlag, May 16-18, 2001.

[16] S. McCanne and M. Vetterli and V. Jacobson", "Low-complexity Video Coding for Receiver-driven LayeredMulticast", IEEE Journal on Selected Areas inCommunication, 1999.

[17] Nguyen, N., P. Milanfar, G.H. Golub, “A ComputationallyEfficient Image Super-resolution Algorithm”, IEEETransactions on Image Processing, vol. 10, no. 4, pp. 573-583, April 2001.

[18] R. Raskar, G. Welch, and H. Fuchs (1998), “SeamlessProjection Overlaps Using Warping and Intensity Blending”,4th Intr. Conf. on Virtual Systems and Multimedia (VSMM),November 18-20, Gifu, Japan.

[19] M. Steele and C. Jaynes, "Parametric SubpixelMatchpoint Recovery with Uncertainty Estimation: AStatistical Approach", IEEE Workshop on Statistical Analysisin Computer Vision in Conjunction with CVPR 2003

[20] R.M. Steele, S. Webb, and C. Jaynes, "Monitoring andCorrection of Geometric Distortion in Projected Displays",International Conference in Central Europe on ComputerGraphics, Visualization and Computer Vision, Plzen, CzechRepublic, 2002.

[21] R. Surati, “Scalable Self-Calibrating Display Technologyfor Seamless Large-Scale Displays”, MIT, 1999.

[22] B. C. Tom and A. K. Katsaggelos, "Reconstruction of aHigh-Resolution Image by Simultaneous Registration,Restoration, and Interpolation of Low-Resolution Images'',Proc. 1995 IEEE International Conf. on Image Processing,pp. II-539-542, Washington, DC, Oct. 1995.

[23] G. Wallace, H. Chen, and K. Li, “Color Gamut Matchingfor Tiled Display Walls”, In Proc. Immersive ProjectionTechnology and Virtual Environments, pp. 293-302, 2003.

[24] S. Webb and C. Jaynes, “Design and Implementation ofthe Oracle: A High-resolution Object Visualization Device”,Technical Report, Dept. of Computer Science, University ofKentucky, 2003.

[25] G. Welch, H. Fuchs, R. Raskar, M. Brown, and H. Towles(2000), “Projected Imagery In Your Office in the Future”, IEEEComputer Graphics and Applications,

(a) (b)

Figure 8: Result of super-resolution composition on the aerial image dataset. Close-up view of pedestrians on a sidewalk. (a)Single projector at 800x600 resolution. Note aliasing problems on sidewalk in (a) and pedestrian’s torso at left is not fullyresolved in the imagery due to insufficient sampling. (b) Two-projector superposition result. Compare with results in (a).