View
11
Download
0
Category
Preview:
Citation preview
This document contains the draft version of the following paper: T. Peng and S.K. Gupta. Algorithms for generating adaptive projection patterns for 3-D shape measurement. ASME Journal of Computing and Information Science in Engineering, 8(3), 2008. Readers are encouraged to get the official version from the journal’s web site or by contacting Dr. S.K. Gupta (skgupta@umd.edu).
Algorithms for Generating Adaptive Projection Patterns
for 3-D Shape Measurement
Tao Peng ∗ Satyandra K. Gupta †
Abstract
Point cloud construction using digital fringe projection (PCCDFP) is a non-contact
technique for acquiring dense point clouds to represent the 3-D shapes of objects.
Most existing PCCDFP systems use projection patterns consisting of straight fringes
with fixed fringe pitches. In certain situations, such patterns do not give the best
results. In our earlier work, we have shown that for surfaces with large range of
normal directions, patterns that use curved fringes with spatial pitch variation can
significantly improve the process of constructing point clouds. This paper describes
algorithms for automatically generating adaptive projection patterns that use curved
fringes with spatial pitch variation to provide improved results for an object being
measured. We also describe the supporting algorithms that are needed for utilizing
adaptive projection patterns. Both simulation and physical experiments show that
adaptive patterns are able to achieve improved performance, in terms of measurement
accuracy and coverage, as compared to fixed-pitch straight fringe patterns.
Keywords: 3-D shape measurement; point cloud; digital fringe projection; adaptive pro-
jection pattern.
∗Department of Mechanical Engineering, University of Maryland, College Park, MD 20742. Email:pengtao@umd.edu
†Department of Mechanical Engineering and Institute for Systems Research, University of Maryland,College Park, MD 20742. Email: skgupta@eng.umd.edu
1
1 Introduction
Accurate and rapid measurement of the 3-D shapes of objects is needed in many applications.
Representative applications include reverse engineering, 3-D digital replication, and part
inspection. In these applications, dense point clouds (often containing hundreds of thousands
of points) that represent the shape of object need to be acquired for surface reconstruction
and error analysis. Existing methods for point cloud acquisition can be divided into two
categories: contact methods and non-contact methods. Examples of contact methods include
Coordinate Measuring Machine (CMM), articulated coordinate measuring arm, and laser
tracking system. These methods require touch probes to make physical contact with the
objects being measured, which leads to the following limitations: slow measuring speed,
difficulty in dealing with surfaces that are soft or fragile, and the presence of normal offsets
in the measured data (i.e. the measured 3-D points are not the contact points with the surface
but the center of the touch probe’s spherical tip). Non-contact measurement methods, on
the other hand, have eliminated these limitations. Some popularly used methods in industry
include laser scanning [1], photogrammetry [2, 3, 4], and structured light [5, 6].
In the community of Computer-Aided Design (CAD), intensive research has been done on
topics related to the processing of massive point cloud data acquired by varied means. Some
of the most popular topics include: representation of point cloud data [7, 8, 9], registration
and merging of multiple point clouds [10], surface reconstruction (from point clouds) [11,
12, 13, 14], extraction of surface features [15], and scan planning [16, 17]. In this paper, we
focus on the acquisition of point cloud data instead of discussing the above topics.
Point cloud construction using digital fringe projection (PCCDFP) is a non-contact
measurement technique based on structured light [18]. Compared to laser scanning and
photogrammetry, the most significant advantage of PCCDFP technique is its high data
rate. Depending on camera resolution and hardware synchronization, PCCDFP systems can
achieve data rates in the order of million points per second. While laser scanners need to be
2
Figure 1: Schematic diagram of a PCCDFP system with one projector and one camera
mounted on CMM or similar devices to perform area scan, due to their line scanning nature,
PCCDFP systems can perform area scans by themselves. For photogrammetry based tech-
niques, it is generally needed to place reflective targets on surfaces being measured in order
to achieve good accuracy. There is no such need for PCCDFP technique. For these reasons,
PCCDFP is the preferable technique for many industrial applications where an accurate,
fast, and inexpesive shape measurement tool is needed.
The simplest PCCDFP system contains one digital projector and one camera (as shown
in Fig. 1). The basic principle behind the approach is to project known patterns on the
object using the projector, by which the object’s surface is encoded with phase values. The
camera is used to take images of the object with the known patterns imposed on it. By
using techniques such as phase-shifting to design the projection patterns, the phase map of
the surface can be acquired accurately from the resulting images. A dense point cloud that
represents the surface being measured can then be constructed using triangulation method.
Readers are referred to Ref. [19, 20, 21] for more details on PCCDFP.
PCCDFP technique uses a set of projection patterns to perform a measurement. Different
designs of patterns could result in different measurement accuracy and coverage [22]. For
applications that demand high measurement speed, e.g. 100% on-line inspection of parts,
the number of projection patterns that can be used in a measurement is constrained by the
speed of system hardware, e.g. the speed of the projector and the camera, as well as the
sychronization between the two. In such a case, an interesting research issue is how to design
3
(a) Sinusoidal fringe pattern(shown with uniformly increasedfringe pitch than actual)
(b) Image (portion view) of thesphere under the projection ofthe pattern
Figure 2: Measurement of a sphere using straight sinusoidal fringes with fixed pitch
projection patterns such that measurement accuracy and coverage can be optimized when
the number of patterns to be used is limited.
A popularly used projection pattern for PCCDFP is sinusoidal fringe pattern (as shown
in Fig. 2(a)), in which the fringes are straight and the fringe pitch is fixed. Here, term “fixed
fringe pitch” refers to the pitch of fringes in a single pattern. For systems using temporal
phase unwrapping techniques [23, 24], multiple fringe patterns with different fringe pitches
are used. However, the fringe pitch in each pattern is still a constant, although it may vary
from pattern to pattern. In certain situations, fixed-pitch fringe patterns do not give ideal
results. Fig. 2(b) shows the image (portion view) of a sphere under the projection of a
sinusoidal fringe pattern. As can be seen, at the left side of the sphere fringes are crowded
together and indistinguishable. This will result in an unresolvable area in the constructed
point cloud. This is mainly caused by the change of surface normal direction from the left
side of the sphere to the right. On the other hand, while using a carefully designed pattern
with variable fringe pitch, the resulting image shows appropriately spaced fringes and the
accuracy of measurement can be possibly improved (see Fig. 3(a) and 3(b)). We should
note that the problem with crowding of fringes in the image cannot be simply solved by
using sinusoidal fringe pattern with a much larger fringe pitch, because that will reduce the
4
(a) Adaptive pattern (shownwith uniformly increased fringepitch than actual)
(b) Image (portion view) of thesphere under the projection ofthe pattern
Figure 3: Measurement of a sphere using curved fringes with spatially varying pitch
measurement accuracy in the middle portion of the sphere.
Digital projectors based on Digital Micro-mirror Device (DMD) or Liquid Crystal Display
(LCD) are capable of generating geometrically complex projection patterns. They provide
the hardware foundation for using variable-pitch fringe patterns in PCCDFP technique.
However, the potential improvement in measurement performance that can be achieved by
using adaptive projection patterns has not been utilized so far.
In our prior work [21], we have described a mathematical framework that can use ge-
ometrically complex projection patterns. The traditional phase-shifting method has been
generalized to handle patterns with spatially variable fringe pitches. A new triangulation
algorithm was developed to convert phase maps to point clouds. By utilizing a reference
phase map, this algorithm is able to use generalized projection patterns in a seamless man-
ner and at the same time handles the projector’s lens distortions automatically. Since the
new triangulation algorithm was derived strictly from the optical geometry model of the
PCCDFP system, it provides high accuracy in point cloud constructions.
In this paper, we describe algorithms for automatically generating adaptive projection
patterns to achieve improved accuracy and coverage in point cloud acquisition. The algo-
rithm utilizes knowledge on the rough shape of the object, which can be acquired from a
5
preliminary measurement. For applications such as inspection of parts on the production
line, this method involves negligible overhead, because the adaptive pattern needs to be
constructed only once and can be used for all successive parts of approximately the same
shape.
It is worth mentioning that, the accuracy and coverage in point cloud acquisition is a
complicated issue and the possible improvement that can be achieved by using adaptive
patterns is constrained by many factors, e.g. specifications of the hardware and the shape
of the object. It is possible that in certain cases adaptive patterns do not provide signif-
icant advantage over traditional fixed-pitch fringe patterns. The use of adaptive patterns,
nevertheless, provides a new tool for achieving better measurement performance.
2 Background
When fixed-pitch fringe patterns are used, the measurement of an object using PCCDFP
includes the following steps (see Fig. 4):
• Projects phase-shifted straight fringe patterns on the object and records the images.
• Constructs an absolute phase map of the object from the images acquired.
• Constructs a point cloud from the absolute phase map of the object.
Using adaptive projection patterns requires modifications to the above procedure and
introduces additional steps. The primary new step is generation of adaptive projection
pattern based on the shape of the object being measured. A schematic diagram of the new
procedure is shown in Fig. 5. A detailed explanation of it is given below.
• Generation of adaptive projection pattern: If objects have different shapes or
they are placed at different positions, the adaptive projection patterns that could give
optimal measurement performance may be different. Hence, if the shape or position of
6
Figure 4: Schematic diagram of the measurement workflow when fixed-pitch fringe patternis used
Figure 5: Schematic diagram of the measurement workflow when adaptive projection patternis used
7
the object being measured has changed, the adaptive projection pattern needs to be
regenerated. The algorithm for the generation of adaptive patterns requires knowledge
of the approximate shape of the object, as well as its position and orientation w.r.t.
the sensor. These information can either be provided to the system in the form of
a geometric model of the object, or acquired by a preliminary measurement. In the
latter case, the measurement can be conducted using a fixed-pitch fringe pattern with a
medium number of fringes to avoid any possible unresolvable area. This measurement
is not intended to give accurate result of the object’s shape; however it is appropriate
for the generation of adaptive patterns. A detailed description of the algorithm for
generating adaptive projection patterns is given in Section 3.
• Construction of new reference phase map: The point cloud construction algo-
rithm we have used requires a reference phase map, which is an absolute phase map of a
flat plane under the projection pattern used for measurement. When a new projection
pattern is generated and is to be used in future measurements, a new reference phase
map needs to be constructed. The construction of the new reference phase map can
be done through an interpolation based approach which takes the adaptive projection
pattern as input as well as two “regular” reference phase maps that are obtained using
fixed-pitch fringe patterns. A detailed description of this algorithm is presented in
Section 4.
• Measurement of the objects: Once the adaptive projection pattern has been gen-
erated and the corresponding reference phase map has been acquired, the procedure
of measurement by using the adaptive pattern is the same as with fixed-pitch fringe
patterns. In other words, a set of phase-shifted adaptive patterns are projected on
the object and the corresponding images are recorded. From these images an absolute
phase map of the object, and hence a point cloud that represents the object’s surface,
can be constructed.
8
To use adaptive projection patterns, we have generalized the conventional phase-shifting
technique [21]. The following two step approach is used to generate fringe patterns:
• Step 1: A phase function Φ(ξ, η) is defined in the image plane of the projector, i.e.
the ξ–η plane. Φ(ξ, η) must be continuous in the ξ–η plane and monotonic in either
ξ or η direction, depending on the position of the camera w.r.t. the projector.
• Step 2: The phase function Φ(ξ, η) is converted to light projection pattern(s) by some
encoding method. Light properties that can be used for encoding include intensity,
color, etc. In practice, light intensity, without the involvement of color, is most pop-
ularly used since the intensity of light can be measured accurately by photo sensors
such as CCD. A widely used modulation function for converting phase values to light
intensities is the sinusoidal function. For digital projectors, in which light intensity is
presented in gray-levels, the sinusoidal modulation can be described using the following
equation:
I(P )(ξ, η) =I
(P )max
2[1 + sin(Φ(ξ, η))] (1)
where I(P )(ξ, η) is the gray-level in the projection pattern and I(P )max is the maximum
gray-level of the projector. Fig. 6 shows an example phase function Φ (gray-levels
represent phase values) and the corresponding projection pattern constructed using
Eqn. 1.
3 Algorithm for Automated Generation of Adaptive
Projection Patterns
In PCCDFP techniques, projection patterns are used to acquire phase maps of surfaces.
The accuracy and coverage of the phase maps acquired determine the accuracy and coverage
of the final measurement result. The main idea behind adaptive projection patterns for
9
100 200 300 400 500
50
100
150
200
250
300
350
400
450
500
(a) Phase function Φ(ξ, η) (gray-levels represent phase values)
(b) Projection pattern con-structed, I(P )(ξ, η)
Figure 6: An example of the construction of generalized projection pattern
measuring a particular surface is to manipulate the local fringe pitch in the pattern such
that the accuracy and coverage of the phase map can be optimized.
Considering the presence of error sources such as image noise, the accuracy of the phase
maps can be improved by two approaches: maintaining high fringe contrast in the images,
and using the smallest fringe pitch possible in the projection pattern. There are inherent
limitations on how small a fringe pitch and high a contrast can be used. In addition, these
two approaches may lead to conflicting parameter settings.
The smallest fringe pitch that can be used in projection patterns is limited by the res-
olution of the digital projector. Most commercial digital projectors have relatively large
apertures and hence small depth of focus. The light projection in the measurement volume
of a PCCDFP system generally experiences some degree of defocusing [25]. When the fringe
pitch in the projection pattern is smaller than a certain value, the defocusing issue can cause
a significant drop of the projected fringe contrast and hence a poor fringe contrast in the
images. Based on our experience with three different DMD projectors, we found that 8 pixels
is the practical minimum for the fringe pitch to be used in projection patterns.
The finite resolution of the digital camera also limits the fringe pitch that could be used
in projection patterns. As demonstrated earlier, the local fringe pitch in the images depends
10
on the surface shape. For fixed fringe pitch in the pattern, the local fringe pitch in certain
areas of the images might be too small such that fringe crowding problem would occur and
result in unresolvable areas in the constructed phase map.
Based on the above guidelines for designing projection patterns, we developed an algo-
rithm to automatically generate adaptive fringe patterns for point cloud acquisition. The
basic idea behind the algorithm is to always use the smallest local fringe pitch in the pattern
which could provide good projection contrast and at the same time not to result in fringe
crowding in the images. A detailed description of the algorithm is as follows:
1. Acquisition of two absolute phase maps, Φ(V ) and Φ(H), of the surface being
measured:
Two absolute phase maps of the surface being measured are acquired by using vertical
and horizontal fixed-pitch fringe patterns respectively. Both patterns use large fringe
pitches to avoid possible unresolvable areas in the phase maps (due to fringe crowding
in the images). Let p(P )fc denote the smallest fringe pitch in the pattern that could
provide satisfactory projection contrast. In practice, we choose p(P )fc to be 8 pixels.
The fringe pitch used in the vertical fringe pattern can be written as cp0 · p(P )fc , where
cp0 is a coefficient that is generally selected to be greater than 2.
Let Φ(V )P (ξ, η) and Φ
(H)P (ξ, η) denote the phase distributions in the vertical and the
horizontal fringe patterns respectively. Φ(V )P and Φ
(H)P can be expressed using the
following equations:
Φ(V )P (ξ, η) =
[
2π/(cp0 · p(P )fc )
]
· ξ
Φ(H)P (ξ, η) =
(
2π/p(H)P
)
· η(2)
where p(H)P is the fringe pitch used in the horizontal fringe pattern and is generally
selected to be close to the value of cp0 · p(P )fc .
Let Φ(V )(u, v) and Φ(H)(u, v) denote the phase maps of the surface obtained by using
the patterns given by Eqn. 2. With these two phase maps, each point (u1, v1) in the
11
image can be associated with two phase values, Φ(V )(u1, v1) and Φ(H)(u1, v1). On the
other hand, a point in the projection pattern can be located uniquely by the same
phase value pair. If (ξ1, η1) denote this point, then the following equations hold:
Φ(V )P (ξ1, η1) = Φ(V )(u1, v1)
Φ(H)P (ξ1, η1) = Φ(H)(u1, v1)
(3)
By combining Eqns. 2 and 3, the coordinates (ξ1, η1) can be calculated as follows:
ξ1 =[
cp0 · p(P )fc /(2π)
]
· Φ(V )(u1, v1)
η1 =[
p(H)P /(2π)
]
· Φ(H)(u1, v1)(4)
2. Construction of the gradient field of the adaptive pattern, [∇Φ](A)ξ :
Let Φ(A)P (s, t) denote the phase distribution of the adaptive projection pattern to be
built, which is a 2-D array defined in the ξ-η space. (s, t) are indices to elements in
Φ(A)P . Let [∇Φ]
(A)ξ (s, t) be the gradients of Φ
(A)P (s, t) along ξ-axis, i.e.
[∇Φ](A)ξ (s, t) = Φ
(A)P (s, t + 1) − Φ
(A)P (s, t)
for s = 1, · · · , S and t = 1, · · · , (T − 1)
(5)
where S ×T has the dimensions of Φ(A)P . To build the adaptive projection pattern, the
gradient field [∇Φ](A)ξ needs to be constructed using the following approach:
• Initialize all elements in [∇Φ](A)ξ to 2π/p
(P )fc , where p
(P )fc is the critical fringe pitch
of projection pattern as defined earlier.
• Let p(I)fc denote the smallest local fringe pitch in the image that can provide
satisfactory phase accuracy. Let [∇Φ]u(i, j) denote the gradients of phase map
Φ(V )(u, v) along u-axis. Create a bitmap mask, M (I), for [∇Φ]u(i, j) to mark the
12
pixels whose phase gradients are larger than a given threshold. In other words,
for all pixels (i, j) in the phase map of the surface that satisfy
[∇Φ]u(i, j) >2π
cp0 · p(I)fc
(6)
mark the corresponding pixels in M (I) as 1s and all others as 0s. where cp0
is the coefficient defined in Eqn. 2. The 1-pixels in M (I) form a number of
connected components, which can be isolated and labeled by using region growing
algorithm [26].
• Let R(I)M (l) (l = 1, . . . , L) denote the connected components in M (I). Create a
bitmap mask M (A) for [∇Φ](A)ξ to mark the pixels whose gradients need to be
adjusted. This is done by the following procedure. For each region R(I)M (l) in
M (I), find its boundary B(I)M (l); map this boundary to M (A) using Eqn. 4; fill the
mapped boundary in M (A) to form a region R(A)M (l), which corresponds to the
region R(I)M (l) in M (I).
• Consider a marked region R(I)M (l) (l = 1, . . . , L) in M (I). For a pixel (i1, j1) in the
region, compute its corresponding coordinates in the projection pattern by using
Eqn. 4. Let (s1, t1) denote the calculated coordinates. Set the ξ-gradients of this
pixel in the adaptive projection pattern as follows:
[∇Φ](A)ξ (s1, t1) =
4π2
cp0 p(I)fc p
(P )fc [∇Φ]u(i1, j1)
(7)
Perform this operation for all pixels in region R(I)M (l). Depending on the mapping
from R(I)M (l) to R
(A)M (l), there might be missed pixels in R
(A)M (l) whose gradients
were not updated. Hence, a search in R(A)M (l) for missed pixels needs to be done
as the final step and the values of these pixels can be updated by means of
interpolation.
13
The above process is conducted for all marked regions R(I)M (l). The result of the
process is that, if the local fringe pitch in an area of the image is too small, the
corresponding area in the adaptive projection pattern is set with larger fringe
pitches.
• Smoothen [∇Φ](A)ξ to avoid steep changes in gradient values. This is done by
performing a Gaussian filtering over [∇Φ](A)ξ .
3. Construction of the phase distribution of the adaptive pattern, Φ(A)P :
Once the gradient field [∇Φ](A)ξ is constructed, the phase distribution of the adaptive
projection pattern, Φ(A)P , can be calculated from discrete integration of [∇Φ]
(A)ξ as
follows:
Φ(A)P (s, t) = Φ
(A)P (s, t − 1) + [∇Φ]
(A)ξ (s, t − 1) ,
for s = 1, · · · , S and t = 2, · · · , T
(8)
This process requires the initialization of the first column of Φ(A)P , i.e. Φ
(A)P (s, 1) (s =
1, · · · , S), which is generally set to zeros.
4. Building the adaptive pattern:
The adaptive projection pattern is built from its phase distribution, Φ(A)P , by applying
a sinusoidal modulation. The intensity distribution of the pattern can be expressed
using the following equation:
I(P )(s, t) =I
(P )max
2
[
1 + sin(
Φ(A)P (s, t)
)]
(9)
where I(P )(s, t) is the intensity of pixel (s, t) and I(P )max is the maximum intensity of the
projection pattern.
The process of generating an adaptive projection pattern for the measurement of a sphere
is demonstrated in Fig. 7. Fig. 7(a) shows the phase map of the sphere Φ(V ) (shown as iso-
phase value contours), which is acquired by using the vertical fringe pattern Φ(V )P . The
14
100 200 300 400 500
50
100
150
200
250
300
350
400
450
500
(a) Contours of the sphere’s phasemap, Φ(V ) (pixels with large phasegradients, [∇Φ]u, are highlighted)
100 200 300 400 500
50
100
150
200
250
300
350
400
450
500
(b) Pixels in the projection pat-tern whose phase gradients, [∇Φ]ξ,need to be set smaller than theirinitial values
100 200 300 400 500
50
100
150
200
250
300
350
400
450
500
(c) Contours of the constructedphase distribution of the adaptive
pattern, Φ(A)P
(d) Adaptive pattern generatedat the end (shown with uni-formly increased fringe pitchthan actual)
Figure 7: Generation of an adaptive projection pattern for the measurement of a sphere
15
highlighted region in the phase map indicates the 1-pixels in the mask M (I). The corre-
sponding region in the projection pattern, represented by mask M (A), is shown in Fig. 7(b).
Fig. 7(c) shows the constructed phase distribution of the adaptive pattern Φ(A)P in the form
of contours. The adaptive projection pattern generated at the end is shown in Fig. 7(d).
4 Algorithm for Construction of New Reference Phase
Map
Recall that the construction of point clouds requires a reference phase map which depends on
the projection pattern used for measurements (see Section 2 and Fig. 5). When a newly gen-
erated projection pattern has to be used, the reference phase map needs to be reconstructed.
The algorithm for construction of new reference phase map requires two absolute phase maps
of the reference plane as well as the generated adaptive pattern. Let Φ(V )P (ξ, η) and Φ
(H)P (ξ, η)
denote the phase distributions of the projection patterns used to obtain the required phase
maps. Φ(V )P (ξ, η) and Φ
(H)P (ξ, η) can be expressed using the following equations:
Φ(V )P (ξ, η) = cV · ξ
Φ(H)P (ξ, η) = cH · η
(10)
where cV and cH are constants. Φ(V )P is a vertical fringe pattern and Φ
(H)P is a horizontal
fringe pattern. Both patterns are of fixed fringe pitch.
Let Φ(V )R (u, v) and Φ
(H)R (u, v) be the corresponding phase maps of the reference plane
acquired by using fringe patterns Φ(V )P (ξ, η) and Φ
(H)P (ξ, η) respectively. Let Φ
(A)P (ξ, η) denote
the phase distribution of the adaptive projection pattern. Notice that when the reference
plane is fixed w.r.t. the PCCDFP system, each point (u, v) in the phase map has a unique
corresponding point in the projection pattern, (ξ, η), which shares same phase value as (u, v)
no matter what pattern is projected. Utilizing this fact, the construction of the new reference
16
phase map Φ(A)R can be completed by the following procedure:
1. For each pixel (u1, v1) in Φ(A)R (u, v), compute its corresponding point (ξ1, η1) in the
projection pattern in the following manner:
ξ1 = (1/cV ) · Φ(V )R (u1, v1)
η1 = (1/cH) · Φ(H)R (u1, v1)
(11)
where cV and cH are constants defined in Eqn. 10.
2. Using the calculated coordinates (ξ1, η1), compute the phase value Φ(A)P (ξ1, η1) by in-
terpolating over the phase distribution of the generated adaptive pattern Φ(A)P (ξ, η).
3. The calculated phase value Φ(A)P (ξ1, η1) is then assigned to Φ
(A)R (u1, v1).
An example of constructing new reference phase map for an adaptive projection pattern
is shown in Fig. 8. Fig. 8(a) and 8(b) are the contour plots of the phase maps Φ(V )R and Φ
(H)R
respectively. The blue crosses drawn in the figures are example pixels in the u-v space, i.e.
pixels (u1, v1) referred in the description of the algorithm above. Fig. 8(c) shows the phase
distribution of the adaptive pattern Φ(A)P in contours. The red crosses drawn in the figure are
points in the ξ-η space that correspond to the example pixels. Fig. 8(d) shows the contour
plot of the new reference phase map Φ(A)R that is constructed.
5 Results and Discussion
The two algorithms presented in this paper, namely the algorithm for automated generation
of adaptive patterns and the algorithm for construction of new reference phase map, were
implemented in Matlab. Using a PCCDFP system (hardware and software) we have devel-
oped in our prior work [21], a number of experiments were performed on objects with various
shapes. For each object, several measurements were made by using automatically generated
17
100 200 300 400 500
50
100
150
200
250
300
350
400
450
500
(a) Contours of phase map Φ(V )R
(blue crosses are example pixels inthe (u, v) space)
100 200 300 400 500
50
100
150
200
250
300
350
400
450
500
(b) Contours of phase map Φ(H)R
(example pixels drawn in bluecrosses)
100 200 300 400 500
50
100
150
200
250
300
350
400
450
500
(c) Phase distribution of the adap-tive pattern shown in contours,
Φ(A)P (red crosses are points in the
(ξ, η) space that correspond to theexample pixels shown in (a), (b)and (d))
100 200 300 400 500
50
100
150
200
250
300
350
400
450
500
(d) Contours of the new reference
phase map computed, Φ(A)R (exam-
ple pixels drawn in blue crosses)
Figure 8: Construction of new reference phase map for an adaptive pattern
18
adaptive patterns as well as traditional sinusoidal patterns with various fringe pitches. The
measurement accuracy and coverage were then compared as indicators of the performance of
different patterns. Two types of experiments were conducted. The first type of experiments
were on CAD models using a simulation software. The second type of experiments were
conducted on physical objects using a physical PCCDFP system.
5.1 Performance Tests Using Measurements on CAD Models
We have developed a simulation software for PCCDFP system. The software is basically a
scene rendering engine which generates images (from the camera’s view) for given object(s),
projection pattern, and scene description (e.g. projector and camera parameters, ambient
light condition, etc.). The projector and the camera are modeled using the pinhole camera
model with Brown’s lens distortion model [27]. The scene rendering is done by ray tracing
with the consideration of the pixel integration effect of CCD/CMOS cameras. The ACIS
geometric kernel is used to facilitate the ray tracing process so that the software can handle
objects of virtually any shapes provided their ACIS models are given. The classic Phong
model is used to model the surface reflection of the objects.
To simulate the influence of error sources, noise was added to the images of objects which
were obtained under the projection of fringe patterns. For all simulation runs, noises were
generated with a uniform distribution and a magnitude of 2.5% (plus and minus) of the
maximum image intensity.
The developed simulator allows us to quickly conduct experiments on virtual objects (i.e.,
CAD models). Four representative experiments are reported in this section: a block with a
cone-shaped hole (shown in Fig. 9), a part with a sawtooth profile (shown in Fig. 13(a)), a
spherical surface (shown in Fig. 14(a)), and a randomly generated spline surface (shown in
Fig. 15(a)). The X-Y dimensions of the parts are equal to or bigger than 200 mm×200 mm.
The Z-dimensions of the parts are as follows: the cone-shaped hole, 160 mm; the part with
19
Figure 9: Part with a cone-shaped hole
Figure 10: One of the images acquired (portion view) by using fixed-pitch fringe pattern,nF = 100
sawtooth profile, 27 mm; and the spline surface, 55.554 mm. All parts have diffuse surfaces.
The automatically generated adaptive projection patterns for the measurements of differ-
ent parts are shown in Fig. 11(a), 13(b), 14(b) and 15(b). When fixed-pitch fringe patterns
(e.g. nF = 100) were used, the images of all the parts have certain degree of fringe over-
crowding in some portions of the images. As a result, the construction of point clouds failed
at these regions. Take the measurement of the cone-shaped hole as an example. As can be
seen from Fig. 10, there is a severe fringe overcrowding in the region that corresponds to
the center-right part of the cone. In the constructed point cloud as shown in Fig.12(a), that
part of the cone is missing. The same problem was observed in the measurements of other
parts as well. However, the use of adaptive projection patterns eliminated this problem (see
Fig. 11(b)). This is because of the fact that, in the middle region of the projection pattern
(see Fig. 11(a)), which is projected on the right slope of the cone, the fringe pitch has been
20
(a) Adaptive fringe pattern gen-erated (shown in larger fringepitch than actual)
(b) One of the images acquired (por-tion view)
Figure 11: Simulated measurement of a cone-shaped hole using adaptive fringe pattern
(a) Point cloud obtainedby using fixed-pitchfringe pattern, nF = 100(pseudo-color representsz-coordinate)
(b) Point cloud obtained byusing the adaptive fringepattern
0 1000 2000 3000 4000 50000.03
0.035
0.04
0.045
0.05
0.055
0.06
0.065
Size of Unresolvable Area (pixels)
RM
S o
f Mea
sure
men
t Err
or (
mm
) Fixed−pitch frg. ptn., θPart
=0o
Fixed−pitch frg. ptn., θPart
=7o
Adaptive fringe pattern
49
64
81
100
121
49
64
81 100121
(c) Measurement performance:adaptive fringe pattern vs. fixed-pitch fringe patterns (nF representsfringe number)
Figure 12: Results of the measurement of a cone-shaped hole
21
extended dramatically so that a reasonable fringe pitch is forged when observed from the
camera’s angle.
The measurement performance achieved by using adaptive and fixed-pitch fringe patterns
are shown in Fig. 12(c), 13(d), 14(c) and 15(c). The horizontal axes of the plots represent
the number of pixels that are unable to be resolved in a measurement. The total number of
pixels is 512 × 512 = 262, 144. The vertical axes represent the RMS value of measurement
error. nF is the number of fringes in the fixed-pitch fringe patterns. As can be seen from the
plots, when fixed-pitch fringe patterns (with varied fringe numbers) were used, any single
measurement could not achieve good accuracy and good coverage at the same time. A high
fringe number (e.g. 121) is able to achieve very good measurement accuracy but the surface
area it fails to measure is quite large; On the other hand, a low fringe number (e.g. 49) has
excellent measurement coverage but the accuracy of measurement is fairly poor. With the
use of adaptive patterns, a compromise can be achieved. In all measurements conducted, the
adaptive patterns achieved an accuracy that is comparable to a fixed-pitch fringe pattern
with a fringe number between 100 to 121. At the same time, it achieved a measurement
coverage that is comparable to (a fixed-pitch fringe pattern with) a fringe number between
49 to 64.
In the measurements of the cone-shaped hole by using fixed-pitch fringe patterns, the
part was placed at a few slightly different tilt angles around Y (W )-axis. Fig. 12(c) shows the
results of the part at two different tilt angles, θPart = 00 and θPart = 70, where θPart = 00
corresponds to the orientation that the part’s top surface is parallel to the X(W ) − Y (W )
plane. The measurement results indicate that, different tilt angles of the part yield different
measurement accuracies and coverages for the same fringe pattern. However, it only changes
the balance of the measurement performance between accuracy and coverage but makes no
improvement to the measurement performance as a whole.
22
(a) Part with a sawtooth profile (b) Adaptive fringe patterngenerated (shown using largerfringe pitch than actual)
(c) Point cloud obtained by usingthe adaptive fringe pattern (pseudo-color represents z-coordinate)
0 500 1000 1500 2000 2500 3000 35000.044
0.046
0.048
0.05
0.052
0.054
0.056
0.058
0.06
Size of Unresolvable Area (pixels)
RM
S o
f Mea
sure
men
t Err
or (
mm
) Fixed−pitch fringe patternAdaptive fringe pattern
nF=49
nF=64
nF=81
nF=100
nF=121
(d) Measurement performance:adaptive fringe pattern vs. fixed-pitch fringe patterns
Figure 13: Simulated measurements of a part with a sawtooth profile
(a) Spherical surface (b) Adaptive fringe patterngenerated (shown in largerfringe pitch than actual)
0 500 1000 1500 2000 25000.03
0.035
0.04
0.045
0.05
Size of Unresolvable Area (pixels)
RM
S o
f Mea
sure
men
t Err
or (
mm
) Fixed−pitch fringe patternAdaptive fringe pattern
nF=49
nF=64
nF=81
nF=100
nF=121
(c) Measurement performance:adaptive fringe pattern vs.fixed-pitch fringe patterns
Figure 14: Simulated measurements of a spherical surface
23
(a) Randomly generatedspline surface
(b) Adaptive fringe patterngenerated (shown in largerfringe pitch than actual)
0 100 200 300 400 500 6000.036
0.038
0.04
0.042
0.044
0.046
0.048
0.05
0.052
Size of Unresolvable Area (pixels)
RM
S o
f Mea
sure
men
t Err
or (
mm
) Fixed−pitch fringe patternAdaptive fringe pattern
nF=121
nF=64
nF=49
nF=100
nF=81
(c) Measurement performance:adaptive fringe pattern vs.fixed-pitch fringe patterns
Figure 15: Simulated measurements of a spline surface
5.2 Performance Tests Using Measurements on Physical Parts
The performance test of adaptive projection patterns was also conducted on physical parts,
using a PCCDFP hardware we have built. The major parameters of the hardware are as
follows: the angle between the optical axis of the projector and the optical axis of the camera
is 270; the distance between the sensor and the center of the measurement volume is around
600 mm; the projector’s field-of-view angles are 43.60 (Horizontal) and 33.40 (Vertical); the
camera’s field-of-view angles are 22.280 (Horizontal) and 16.770 (Vertical); the resolution of
the projector is 1024× 768 pixels; the resolution of the camera is 640× 480 pixels; and both
the projector and the camera have a gray-depth of 8 bits. All measurements were conducted
under regular indoor lighting conditions.
Fig. 16 shows the experiment results of a plastic flowerpot, which has a maximum di-
ameter of 198 mm and a depth of 149 mm. The measurement coverages achieved by using
adaptive projection pattern and fixed-pitch fringe patterns are shown in Fig. 16(c). The
horizontal axis of the plot represents the number of fringes in the projection pattern (nF ),
and the vertical axis represents the number of pixels that could not be resolved in the mea-
surement. For fixed-pitch fringe patterns, the size of unresolvable area increases as the fringe
number increases. This is caused by fringe overcrowding as the fringe pitch in the projection
24
(a) Photograph of theflowerpot
(b) Adaptive fringe pattern gen-erated
0 30 60 90 120 150 1800
5
10
15
20
Number of Fringes in Projection Pattern
Siz
e of
Unr
esol
vabl
e A
rea
(x10
3 pix
els) Fixed−pitch fringe pattern
Adaptive fringe pattern
(c) Measurement coverageachieved: adaptive pattern vs.fixed-pitch fringe patterns
Figure 16: Measurements of a plastic flowerpot
pattern gets smaller. In the case of adaptive pattern, a measurement coverage comparable to
a (fixed-pitch fringe pattern with a) fringe number of 50 was achieved, although the adaptive
pattern has an equivalent fringe number of 118. Since the accurate shape of the flowerpot is
not known, the study on measurement accuracy was not possible.
Fig. 17 shows the experiment results of a plastic tube, which has a diameter of 127.34 mm
and a height of 95 mm. The adaptive pattern generated for the measurement of the tube
is shown in Fig. 17(b). In order to evaluate the measurement accuracy, the point cloud
acquired in a measurement was fitted to a cylinder and the residual deviation was analyzed.
Fig. 17(c) shows the measurement coverage and measurement accuracy achieved by using
different projection patterns. The horizontal axis of the plot represents the number of pixels
that could not be resolved in the measurement, and the vertical axis represents the RMS value
of the divergence of the point cloud from a perfect cylinder. As can be seen, the adaptive
projection pattern achieved a better overall measurement performance than fixed-pitch fringe
patterns. Also notice that, for fix-pitch fringe patterns, the RMS value of measurement error
increased as the fringe number (nF ) grew from 100 to 144, which is different from the test
results with CAD models (see Sec. 5.1). This is because, as the fringe number gets larger
than around 100, the defocusing issue of the projector [25] causes a significant drop in fringe
contrast, which in turn causes an increase in measurement error. However, this phenomenon
25
(a) Photograph of thetube
(b) Adaptive fringe pattern gen-erated
0 5 10 150.06
0.065
0.07
0.075
0.08
0.085
0.09
0.095
0.1
Size of Unresolvable Area (x103 pixels)
RM
S o
f Mea
sure
men
t Err
or (
mm
) Fixed−pitch fringe patternAdaptive fringe patternn
F=32
nF=64
nF=81
nF=100
nF=125
nF=144
(c) Measurement performance:adaptive fringe pattern vs.fixed-pitch fringe patterns
Figure 17: Measurements of a plastic tube
was not captured in the tests with CAD models since the defocusing issue of projector was
not considered in simulations.
Similiar tests were also conducted on a variety of parts. The developed algorithms worked
well in generating adaptive projection patterns to eliminate fringe overcrowdings in images.
Fig. 18 shows the result of measurements made on a plastic human face model. When fixed-
pitch fringe patterns were used, as shown in Fig. 18(a), the left boundary of the face and
the left side of the nose have fringe overcrowding problems. The automatically generated
adaptive pattern, shown in Fig. 18(b), has increased fringe pitches in these regions. The
resulting image (shown in Fig. 18(c)) obtained from adaptive pattern has no such problem.
Similar improvement was also observed in the measurements of a plastic base of a telephone
handset (results shown in Fig. 19).
As a summary of the above, when the number of fringes in the projection pattern is
fixed, e.g. in order to maintain a certain measurement accuracy, adaptive projection pattern
is able to achieve better measurement coverage than fixed-pitch fringe pattern.
26
(a) Image (trimmed) acquiredusing fixed-pitch fringe pattern
(b) Adaptive fringe pattern gener-ated (shown in larger fringe pitchthan actual)
(c) Image (trimmed) acquiredusing adaptive pattern
Figure 18: Measurements of a plastic human face model
6 Conclusions
Most existing PCCDFP systems use projection patterns consisting of straight fringes with
fixed fringe pitches. When measuring objects with a large range of surface normal directions,
it is hard to achieve high measurement coverage and good measurement accuracy at the
same time by using fixed-pitch straight fringe patterns. Patterns that use curved fringes
with spatial pitch variation can significantly improve the point cloud acquisition process in
such cases. Hence, a possible way to achieve good measurement accuracy and coverage is
to use projection patterns that have been customized for the object being measured. Such
customization involves curving fringes at certain locations and spatially varying fringe pitch.
This paper describes algorithms for automatically generating adaptive projection patterns
that use curved fringes with spatial pitch variation to provide improved measurement results
for an object being measured. In addition, we also describe the algorithm for constructing the
reference phase map. Both simulation and physical experiments show that adaptive patterns
are able to achieve improved performance. Specifically, adaptive projection patterns provide
27
(a) Image (trimmed) acquiredusing fixed-pitch fringe pattern
(b) Adaptive fringe pattern gener-ated (shown in larger fringe pitchthan actual)
(c) Image (trimmed) acquiredusing adaptive pattern
Figure 19: Measurements of a plastic base of a telephone handset
better overall measurement performance (coverage and accuracy) than fixed-pitch fringe
patterns, especially when the object being measured has a large range of surface normal
directions.
In applications such as on-line parts inspection doing batch processes, e.g. sheet metal
stamping, parts being measured are of virtually the same shape and are placed approxi-
mately at the same position and orientation with respect to the measuring device. Hence,
the adaptive patterns optimized for measuring these parts need to be generated only once
and can be repeatedly used thereafter. The overhead of using adaptive patterns (i.e. the
procedure to generate adaptive patterns) is negligible, which makes it an ideal solution for
such applications.
As a part of the future work, an in-depth understanding of the relationship between
the fringe pitch in projection pattern and the resulting measurement performance needs
to be established. Furthermore, the influence of other measurement factors need to be
characterized.
28
7 Acknowledgments
This work has been supported by NSF grant DMI-0093142. However, the opinions expressed
here are those of the authors and do not necessarily reflect that of the sponsor.
References
[1] Z. Ji and M. C. Leu. Design of optical triangulation devices. Optics and Laser Tech-
nology, 21(5):335–338, 1989.
[2] C. S. Fraser and D. C. Brown. Industrial photogrammetry - new developments and
recent applications. Photogrammetric Record, 12(68):197–217, 1986.
[3] S. N. Lane, T. D. James, and M. D. Crowell. Application of digital photogramme-
try to complex topography for geomorphological research. Photogrammetric Record,
16(95):793–821, 2000.
[4] L. M. Galantucci, R. Ferrandes, and G. Percoco. Digital photogrammetry for facial
recognition. Journal of Computing and Information Science in Engineering, 6(4):390–
396, 2006.
[5] B. Breuckmann, F. Halbauer, E. Klaas, and M. Kube. 3D-metrologies for industrial
applications. In Proceedings of SPIE 3102, pages 20–29, 1997.
[6] F. Chen, G. M. Brown, and M. Song. Overview of three-dimensional shape measurement
using optical methods. Optical Engineering, 39(1):10–22, 2000.
[7] P. Benko, R. R. Martin, and T. Varady. Algorithms for reverse engineering boundary
representation models. Computer-Aided Design, 33(11):839–851, 2001.
[8] W. Sun, C. Bradley, Y. F. Zhang, and H. T. Loh. Cloud data modelling employing a
unified, non-redundant triangular mesh. Computer-Aided Design, 33(2):183–193, 2001.
29
[9] J. Huang and C. H. Menq. Combinatorial manifold mesh reconstruction and opti-
mization from unorganized points with arbitrary topology. Computer-Aided Design,
34(2):149–165, 2002.
[10] J. Jamshidi, G. W. Owen, and A. R. Mileham. A new data fusion method for scanned
models. Journal of Computing and Information Science in Engineering, 6(4):340–348,
2006.
[11] J. Barhak and A. Fischer. Parameterization and reconstruction from 3D scattered points
based on neural network and PDE techniques. IEEE Transactions of Visualization and
Computer Graphics, 7(1):1–16, 2001.
[12] S. Azernikov and A. Fischer. A new volume warping method for surface reconstruction.
Journal of Computing and Information Science in Engineering, 6(4):355–363, 2006.
[13] G. Moroni and M. Rasella. Application of regression spline to reverse modeling. Journal
of Computing and Information Science in Engineering, 7(1):95–101, 2007.
[14] R. J. Urbanic, W. H. ElMaraghy, and H. A. ElMaraghy. An integrated systematic design
recovery framework. Journal of Computing and Information Science in Engineering,
6(4):318–330, 2006.
[15] K. Schreve, C. L. Goussard, A. H. Basson, and D. Dimitrov. Interactive feature modeling
for reverse engineering. Journal of Computing and Information Science in Engineering,
6(4):422–424, 2006.
[16] C. Mehdi-Souzani, F. Thiebaut, and C. Lartigue. Scan planning strategy for a gen-
eral digitized surface. Journal of Computing and Information Science in Engineering,
6(4):331–339, 2006.
30
[17] P. Singh, Y. Wu, R. Kaucic, J. Chen, and F. Little. Multimodal industrial inspection
and analysis. Journal of Computing and Information Science in Engineering, 7(1):102–
107, 2007.
[18] S. Toyooka and Y. Iwaasa. Automatic profilometry of 3-D diffuse objects by spatial
phase detection. Applied Optics, 25(10):1630–1633, 1986.
[19] R. Sitnik, M. Kujawinska, and J. Woznicki. Digital fringe projection system for large-
volume 360-deg shape measurement. Optical Engineering, 41(2):443–449, 2002.
[20] R. Legarda-Saenz, T. Bothe, and W. P. Juptner. Accurate procedure for the calibration
of a structured light system. Optical Engineering, 43(2):464–471, 2004.
[21] T. Peng and S. K. Gupta. Model and algorithms for point cloud construction using digi-
tal projection patterns. Journal of Computing and Information Science in Engineering,
7(4):372–381, 2007.
[22] E. U. Wagemann, M. Schonleber, and H. J. Tiziani. Grazing holographic projection
of object-adapted fringes for shape measurements with enhanced sensitivity. Optics
Letters, 23(20):1621–1623, 1998.
[23] H. Zhao, W. Chen, and Y. Tan. Phase-unwrapping algorithm for the measurement of
three-dimensional object shapes. Applied Optics, 33(20):4497–4500, 1994.
[24] W. Nadeborn, P. Andra, and W. Osten. A robust procedure for absolute phase mea-
surement. Optics and Lasers in Engineering, 24(2–3):245–260, 1996.
[25] C. R. Coggrave and J. M. Huntley. Optimization of a shape measurement system based
on spatial light modulators. Optical Engineering, 39(1):91–98, 2000.
[26] Kenneth R. Castleman. Digital Image Processing. Prentice Hall, second edition, 1995.
31
[27] D. C. Brown. Close-range camera calibration. Photogrammetric Engineering, 37(8):855–
866, 1971.
32
Recommended