Upload
truongnhu
View
213
Download
0
Embed Size (px)
Citation preview
Interactive Virtual Environments
Modelling 3D Geometric and Elastic Properties
Emil M. Petriu, Dr. Eng., FIEEEProfessor, School of Information Technology and Engineering
University of Ottawa, Ottawa, ON, Canadahttp://www.site.uottawa.ca/~petriu
May 2008
Interactive Model-Based Hapto-Visual Teleoperation - a human operator equipped with hapticHCI can telemanipulate physical objects with the help of a robotic equipped with haptic sensors.
……
Tactile Sensor Interface
HapticRobot
InterfaceROBOT(k)
OBJ(i)
Robot Arm Controller
CyberGrasp ™ CyberTouch ™
Haptic Human I nterfaceUSER (k)
NETWORK NETWORK
…
{ [ 3D(j) & F(k,j) ], t }
AVATAR HAND ( k )
. . .
. . .
OBJ (N) OBJ (1)
3D Geometric & Elastic
Composite Model of Object{( x
p, y
p, z
p,E
p) | p = 1,..,P }
OBJ ( j )
Virtual Operation TheaterVirtual Operation Theater
Composite Haptic Interaction Vector between User (k) and Object (j)
ApplicationSpecific
InteractiveAction
Scenario
…
……
Tactile Sensor Interface
HapticRobot
InterfaceROBOT(k)
OBJ(i)
Robot Arm Controller
Tactile Sensor Interface
HapticRobot
InterfaceROBOT(k)
OBJ(i) OBJ(i)
Robot Arm Controller
CyberGrasp ™ CyberTouch ™
Haptic Human I nterfaceUSER (k)
CyberGrasp ™ CyberTouch ™
Haptic Human I nterfaceUSER (k)
NETWORK NETWORK
…
{ [ 3D(j) & F(k,j) ], t }
AVATAR HAND ( k )
. . .
. . .
OBJ (N) OBJ (1)
3D Geometric & Elastic
Composite Model of Object{( x
p, y
p, z
p,E
p) | p = 1,..,P }
OBJ ( j )
Virtual Operation TheaterVirtual Operation Theater
Composite Haptic Interaction Vector between User (k) and Object (j)
ApplicationSpecific
InteractiveAction
Scenario
{ [ 3D(j) & F(k,j) ], t }
AVATAR HAND ( k )
. . .
. . .
OBJ (N) OBJ (1)OBJ (1)
3D Geometric & Elastic
Composite Model of Object{( x
p, y
p, z
p,E
p) | p = 1,..,P }
OBJ ( j )
3D Geometric & Elastic
Composite Model of Object{( x
p, y
p, z
p,E
p) | p = 1,..,P }
OBJ ( j )
3D Geometric & Elastic3D Geometric & Elastic
Composite Model of Object{( x
p, y
p, z
p,E
p) | p = 1,..,P }{( x
p, y
p, z
p,E
p) | p = 1,..,P }
OBJ ( j )
Virtual Operation TheaterVirtual Operation Theater
Composite Haptic Interaction Vector between User (k) and Object (j)
ApplicationSpecific
InteractiveAction
Scenario
…
Model-based approach, based on the kinematics and dynamics of the object handled with the fingertips, provides a convenient representation of the dexterous manipulation. Quoting Salisbury et al.’s recent survey of haptic rendering, “improved accuracy and richness in object modeling and haptic rendering will require advances in our understanding of how to represent and render psychophysically and cognitively germane attributes of objects, as well as algorithms and perhaps specialty hardware (such as haptic or physics engines) to perform real-time computations”.----------------K. Salisbury, F. Conti, F. Barbagli, “Haptic Rendering: Introductory Concepts, ”IEEE Computer Graphics and Applications, Vol. 24, No. 2, pp. 24 – 32, 2004.
Modelling allows to simulate the behavior of a system for a varietyof initial conditions,excitations and systems configurations
The quality and the degree of the approximation of the model can be determined only by a validation against experimental measurements.
The convenience of the model means that it is capable of performing extensive parametric studies, in which independent parameters describing the model can be varied over a specified range in order to gain a global understanding of the response.
E.M. Petriu, "Neural Networks for Measurement and Instrumentation in Virtual Environments,” in Neural Networks for Instrumentation, Measurement and Related Industrial Applications , S. Ablameyko, L. Goras, M. Gori, V. Piuri - Eds.), NATO Science Series, Series III: Computer and System Sciences Vol. 185, pp.273-290, IOS Press, 2003
Discreet vs. Continuous Modelling of Physical Objects and Processes
DISCREET MODEL•sampling => INTERPOLATION COST
y(j) = y(A) + [ x(j)-x(B)] . [ y(B)-x(A)] / [x(A)-x(B)]
x
y
x(j)
y(j) = ? A
B
x
y
CONTINUOUS MODEL• NO sampling =>
NO INTEPPOLATION COST
Analog Computer vs. Neural Network Modelling of Continuous Physical Objects and Processes
Both the Analog Computers and the Neural Networks are continuous modelling devices.
The Analog Computer (AC) allows to solve the linear or nonlinear differential and/or integral equations representing mathematical model of a given physical process. The coefficients of these equations must be exactly known as they are used to program/adjust the coefficient-potentiometers of the AC’s computing-elements (OpAmps). The AC doesn’t follow a sequential computation, all its computing elements perform simultaneously and continuously. An interesting note, “because of the difficulties inherent in analog differentiation the [differential] equation is rearranged so that it can solved by integration rather than differentiation.” [A.S. Jackson, Analog Computation, McGraw-Hill Book Co., 1960].
The Neural Network (NN) doesn’t require a prior mathematical model. A learning algorithm is used to adjust, sequentially by trial and error during the learning phase, the synaptic-weights/coefficient-potentiometers of the neurons/computing-elements.
Similarly to the analog computer, a NN doesn’t follow a sequential computation algorithm, all its neurons performing simultaneously and continuously. The neurons are also integrative-type processing elements.
Compare the performance of three NN architectures used for 3D object shape modelling:
• Multilayer Feedforward (MLFF ) • Self-Organizing Map (SOM ) • Neural Gas Network
A.-M. Cretu, E.M. Petriu, G.G. Patry, “Neural-Network-Based Models of 3-D Objects for Virtualized Reality: A Comparative Study,” IEEE Trans. Instrum. Meas.," Vol. 55, No. 1, pp.99-111, 2006.
NN Modelling of 3D Object Shape
The surface of an object is described by a set of zeros in the output response of the NN, while the interior or exterior regions of the object are described by negative or positive value, respectively.
Initial 3D pointcloudof sample
points representing the object
{( xi, yi, zi) | i = 1,...,N}
Neural Network model
of the object
{( xp, yp, zp) | p = 1,...,P}
Starting from a pointcloud of sample points capturing the shape of the object to be modeled, the NN produces a volumetric representation of the object.
The NN model provides a continuous representationof the object surface and permits an extensive study not only of the modeled object surface, but of theentire object volume. Thus it inherits the advantages of the polygonal and surface models, without inheriting their drawbacks.
It can be used to perform simple operations such as • object morphing, • set operations, and • object collision detection
Initial 3D pointcloud of samplepoints representing the object
{( xi, yi, zi) | i = 1,..., N}
xi, yi, zi xp, yp, zp
Transformed (translated, rotated,scaled, bent, tapered, twisted)
object{( Xi, Yi, Zi) | i = 1,..., N}
Neural Network Architecturefor 3D Object Representation
MLFF SOM Neural Gas
Xi, Yi, Zi
MLFF
Neural-Network modelof the object
{( xp, yp, zp) | p = 1,..., P}
Transformation Module:translation, rotation, scaling, and deformations (bending, tapering, twisting)
MLFF Neural Network
Pointcloud of sample pointsrepresenting the objectO{( xi, yi, zi) | i = 1,..., N}
xi, yi zi
Transformed object pointcloud{(X i, Yi, Z i) | i =1,..., N }
Xi, Yi ,Zi
The transformation module implements: * rotation, * translation, * scaling, and * deformations such as:
- bending, - tapering, and - twisting.
Transformation Module - Generation Mode
Original
Tapering
Rotation
Translation,Rotation,Scaling
Twisting
Bending
The transformation module has no hidden layer and the outputs have linear activation functions.
Knowing the equations, the network weights will be set such thatthe desired transformation is performed
The translation, rotation, scaling, and tapering will be encoded in the weights implementing the generalized 3D transformation matrix.
When provided with a set of points from a given reference model as inputs and those of a transformed model as targets, the NNtransformation module can learn and store in its weights information on how these points have been transformed and/or deformed.
Rotation and Translation
X
Z
Y
x
y
z
m1
m2
m3
n1n2
n3
q1
q2
q3t z
t y
t x
ψϕψϕ
ϕψθψϕθψθψϕθ
ϕθψθψϕθψθψϕθ
ϕθ
coscossincos
sin)sincoscossin(sin)coscossinsin(sin
cossin)sinsincossin(cos)cossinsinsin(cos
coscos
33
32
31
23
22
21
13
12
11
aqaqaqanananamamam
==
−=−=−=
=+=−=
=
Rotation
19080 points, 10-5-1, 5 extra surfaces, d=0.055, 1200 epochs, 2.8 hrs.
MLFF Representation - Results
250 points, 6-3-1, 1 extra surface, d=0.055, 550 epochs, 7 min.
7440 points, 8-4-1, 5 extra surfaces, d=0.055, 1100 epochs, 1 hr
51096 points, 20-10-1, 5 extra surfaces, d=0.055, 2000 epochs, 5.2 hrs.
2500 points, 12-6-1, 2 extra surfaces, d=0.06, 1020 epochs, 45 min.
MLFF Representation - Results19000 points, 14-7-1, 4 extra surfaces, d=0.055, 1100 epochs, 3.3 hrs
W12(i)
(x, y, z) points
X
Z
YReference
weight matrixW1
linear interpolation(n steps,i =1,..., n)betweenW1 andW2
X
Z
YTarget
weight matrixW2
O1
O2
(xm, ym, zm)of the morphedobjecti
x
z
y
MLFF Representation – Applications Object Morphing
MLFF Representation – Applications Set Operations
O1<0ANDO2<0
(x, y, z) belongs to intersection
O1<0ANDO2>0
(x, y, z) belongs to difference
X
Z
YNN
object1
O1
X
Z
YNN
object2
O2
( x, y, z )
O1<0ORO2<0
(x, y, z) belongs to union
97%
0%
2.3%
X
Z
YNN
object2
O
( x1, y1, z1 )( x2, y2, z2 )...( xn, yn, zn )
sampled pointsobject 1
Oi < 0
collision betweenobject 1 andobject 2
i=1,.., n
MLFF Representation – Applications Object Collision Detection
96.56%
MLFF Representation – Applications Object Recognition
W
alignmentinformation
RepresentationModule
X
Z
Y TransformationModule
points of thereference modeliin the database
sampledalignedpointsof transformedobject
Degree ofbelonging to a referencemodel i
points on thetransformedobject
The model-based recognitionis done by first aligning the transformed given object with the reference model stored the database.
The NN of the transformationmodule is trained using the reference model points as training points and the points of the given object as targets.In this way, the transformation information is stored in the NN weights.
After the alignment, a set of points is sampled from the transformed given object and then tested to see the degree of belonging to the reference model.
MLFF Representation – Applications Object Recognition
2.4% 4.9% 1.6% 99.1%
93.3% 3.3% 91.7% 3.1%
95% 99.1% 5.78% 98.3%
91.7% 6.6% 1.6% 92.5%
• simple and compact architecture• continuous volumetric model (though trained with
surface)• information about the entire object space• provides desired accuracy• represents objects of varied complexity• preserves details• morphing, set operations, recognition, collision
detection
• computationally intensive (for both learning and rendering)
• lack of local control of the object
MLFF Neural Network Modelling – Summary
Advantages Disadvantages
SOM and Neural Gas - Compressed Representation Models
Compressed NN model of the 3D object
{( xp, yp, zp) | p = 1,2, …, P }, where P<N
SOM / Neural GasPointcloud of sample pointsrepresenting the object O{( xi, yi, zi) | i = 1,..., N}
xi, yi zi
xp, yp zp
SOM Representation – NN Architecture
• Activation Function– soft competition
• Learning– unsupervised
input layer ...
[ x i, yi , zi ]
wji
yj K=2
K=1
winningneuron
A SOM has a input layer, where vector containing the x,y,z coordinates of points will be presented. Computations are feedforward in the first layer and bidirectional in the connection (output layer). The purpose of the lateral connection (represented by a Gaussian) is to reinforce the activation values of strong neurons and decrease the activation of weak ones. The output neurons are arranged on a grid, usually 2D. The network is based on competition - soft competition. The winning neurons is the one closest to theinput vector. Its value and the value of a given neighbourhood are updated during learning. After learning, two vectors belonging to the same cluster will be projected on two close neurons in the output space.
• Activation Functions:– soft competition– neighbourhood
ranking• Learning
– unsupervised
Neural Gas Representation – NN Architecture
The NG releases the neurons from the output grid. There are no connections between units in the connection layer. The nodes move independently over the data space. It also exploits the idea of soft competition, but the neurons to be updated are not selected according to topological relation, but rather according to rank. The rank is the rank the neurons have in an ordered list of distances between their weights and the input vector. NG converges quickly and to a lower distortion error.
Initial pointcloud
Neural Gas
SOM
19080 points 14914 points 13759 points
1125 points, 42 min.
1125 points, 26 min.
875 points,11 min.
875 points,24.5 min.
875 points22 min.
875 points, 10 min.
er= 0.0098
er= 0.0125
SOM and Neural Gas Modelling - Results
SOM and Neural Gas Modelling – Applications Object Morphing
SOM and Neural Gas Modelling – Applications Segmentation
• simple and compact (weights)• compressed• less memory usage• desired accuracy• objects of varied complexity• details • morphing, motion detection,
segmentation
• computational expensive for high accuracy
• no information about the object space
• no direct surface representation
SOM and Neural Gas Modelling – Summary
Advantages Disadvantages
MLF
FNe
ural
Gas
SOM
3.3 hrs 1 hrs
42 min.
26 min.
9 min.
3 min.
MLFF, SOM, and Natural Gas Modelling – Performance Comparison Training Time
0
50
100
150
200
250
300
350
Co
nst
ruct
ion
tim
e (
min
)
hand pliers face statue
Models
MLFFNN SOM Neural Gas
MLFNNcomputational time = construction time + generation time+rendering
SOM and Neural Gascomputational time =construction time + rendering
MLFF, SOM, and Natural Gas Modelling – Performance Comparison
As it can be seen, the MLFFNN is the most computational expensive, followed by NG. This time is only the construction time. However, the MLFFNN requires a generation time as well to reconstruct the model given the weights and self-organizing architectures require the rendering time. For the point based rendering technique, the self-organizing architectures are still less computationally expensive than the MLFFNN.
0
200
400
600
800
1000
1200
Sto
rage s
pace
(K
b)
hand face pliers statue
Models
Point CloudMLFFNNSOMNeural Gas
MLFF, SOM, and Natural Gas Modelling – Performance Comparison Compactness
The use of neural network modeling is advantageous from the point of view simplicity and compactness.
MLFNN – provide continuous models, information on the entire object space, convenient for many applications, however they are time consuming.
SOM and Neural Gas – provide compressed models while maintaining the properties of the objects, have very good accuracy, and they are less time consuming
The use of any specific techniques depends on the application requirements.
MLFF, SOM, and Natural Gas Modelling of 3D Objects
CONCLUSIONS
Neural Network Adaptive Sampling of 3D Object Elastic Properties
A.-M. Cretu, E.M. Petriu, “Neural-Network –Based Adaptive Sampling of Three-Dimensional-Object Surface Elastic Properties,” IEEE Trans. Instrum. Meas.," Vol. 55, No. 2, pp. 483-492, 2006.
Recovery of the elastic material properties requires touching each point of
interest on the explored object surface and then conducting a strain-stress
relation measurement on each point.
< = ≤≤ ⋅=
0
maxmax
max
pppp
ppppp
ififE
εεσσεεεσ
The elastic behaviour at any given
point (xp, yp, zp) on the object surface
is described by the Hooke’s law:
where Ep is the modulus of elasticity ,
s p is the stress, and e p is the strain
on the normal direction.
Tactile probing is a time consuming Sequential operation
Find fast sampling procedures able to minimize the number of the sampling points by selecting only those points that are relevant to the elastic characteristics.
non-uniform adaptive sampling algorithm of the object’s surface,
which exploits the SOM (self-organizing map) ability to find optimal finite quantization of the input space.
Initial 3D geometricmodel of the object's surface
{( xi, yi, zi) | i = 1,...,N}SOM / Neural Gas
Adaptive-sampled3D geometricmodel of the object surface{( xp, yp, zp) | p = 1,...,P}
RoboticTactileProbing
Adaptive-sampled3D geometric&
elastic composite modelof object's surface
{( xp, yp, zp, Ep) | p = 1,...,P}
xi, yi, zi
xp, yp, zp
Ep
Adaptive Sampling Control of the Robotic Tactile Probing of Elastic Properties of 3D Object Surfaces
SOM (Self Organizing Map) and Neural Gas NN architectures are both used to build compressed model of the 3D object originally defined as a point-cloud.
The weight vector will consist of the 3D coordinates of the object’s points.
During the learning procedure, the model will contract asymptotically towards the points in the input space, respecting their density and thus taking the shape of the object encoded in the point-cloud.
Data point-clouds obtained with a range scanner are used to train the network. Normalization is employed to remove redundant information from a data set, by a linear rescaling of the input vectors such that their variance is 1.
In order to evaluate the quality of the models, a straightforward measure of the precision is used. The precision is estimated as the average distance between each data vector and its winning neuron .
a) b) c)
d) e) f)
g) h) i)
(a) Training data setof 3721points
(b) Neural Gas network, error=0.0112,
(c) SOM, error=0.0133(d) Noisy data set,
random 0 – 0.1,e) Neural Gas network
error=0.0383, (f) SOM, error=0.0266(g) Noisy data set,
random 0 – 0.04, (h) Neural Gas network
error=0.0224, (i) SOM, error=0.0241
Robustness to noisy training data
Training Neural Gas networkwith a map size of 25×45 for 20 epochs, for α0 = 0.5, λ0 = number of neurons/2 and a SOM, with the initial neighborhood radius σ0 =5, and a map size of 25×45, trained for 100 epochs, with data corrupted by different levels of noise.
The initial set of 3721 points is reduced to 1125 points.
It takes approximately 250s for the SOM to build a model of a sphere, while it takes approximately double for the Neural Gas NN. However, even for a larger number of training epochs (5 times more) the SOM does not reach the same accuracy as the Neural Gas NN does, for data that is not very noisy (a random noise level below 0.1).
SOM suffers from the boundary problem. The models obtained look as if they contain cavities.
0
0.02
0.04
0.06
0.08
0.1
0.12
0.14
0.16
0 0 - 0.04 0 - 0.05 0 - 0.1 0 - 0.2 0 - 1
random noise level
rela
tive
erro
r
Neural Gas SOM
For low levels of noise the Neural Gas network performs better than SOM. For higher level of noise, SOM tends to smooth the effect of noise, while the Neural Gas network, which has high sensitivity,follows the noisy patterns.
• The first column presents three views of the originalpoint-cloud of 19,080 points representing a human face.
• The second column presents the compressed model of 1,152 points obtained using The Neural Gas network.
• The third column presents the compressed model of 1,152 points obtained using SOM.
Qualitative comparison between the Neural Gas and the SOM adaptive sampled models.
• The map sizes are equal for both networks.
• The first column represents the original point-cloud,
• The second column represents the Neural Gas model.
• The third column represents the SOM model.
For the 14,914 points of the original point-cloud model given in the first figure,it takes 24 min. to build the Neural Gas model shown in the second figure and 11 min. to build the SOM model shown in the third figure (for the same map size of 25x35 in both cases).
00.20.40.60.8
11.21.41.61.8
2
5 10 20 50 100 1000
number of training epochs
rela
tive
erro
r
Neural Gas SOM
For both, Neural Gas and SOM, networks the quality is improving with the number of training epochs
On the whole the quality of the Neural Gas models appears to be better. Because of the boundary problem, the SOM models are to be avoided for non-noisy data.
Neural Gas and SOM neural networks are both able to compress the initial model with the desired degree of accuracy.
The number of points can be further reduced by reducing the map size. However, there is a compromise to be made between the quality of the resulting compressed model and the map size.
Neural Gas networks are able to model an entire scene of objects while the SOM networks are not able of such a performance.
Starting from a 3D point-cloud, a neural gas NN yields a reduced set of points on the 3D object’s surface which are relevant for the tactile probing. The density of these tactile probing points is higher in the regions with more pronounced variations in the geometric shape. A feedforward NN is then employed to model the force/displacement behavior of selected sampled points that are probed simultaneously by a force/torque sensor and theactive range finder.
3D pointcloudof data
Samplepoints
Deformationprofiles
ForceMeasurements
FeedforwardNeural Network
F
profile(f0)profile(f1)profile(f2)profile(f3)
f0f1f2f3
Neural gasnetwork
Rangefinder
Force/Torquesensor
Neural Network Mapping an Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications
Sampling points selected withthe neural gas network.
Variable elasticity object usedfor experimentation.
(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006 ).
Sampling points selected withthe neural gas network for the ball.
Elastic ball usedfor experimentation.
(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006 ).
Different magnitudes of a normal force are applied successively on the selected sampling points using the probe attached on the force/torque sensor and a range profile is collected with the laser range finder for each force magnitude.
(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006 ).
There is no need to recover the explicit displacement information from the range profiles. Instead the NN models use the raw range data as a function of applied force, F, without explicitly defining values for the displacement. For each cluster of similar elasticity, a feed-forward NN with two input neurons (F and a), 45 hidden neurons (H1-H45) and one output neuron (Z), is used to learn the relation between forces and the corresponding geometric profiles provided by the range finder.
F
Z...
H1
H45
H2
α
The NN associated with each material were trained for 10,000 epochs using the Levenberg-Marquardt variation backpropagationalgorithm with the learning rate set to 0.1. The whole data set is used for training in order to provide enough samples. The training takes approximately 10 min. on a Pentium IV 1.3GHz machine with 512MB memory. For the rubber, the sum-squared error reached during training is 3.7 x10-3, for cardboard is 3.5 x10-2 while for the foam is 2.2x10-2. As expected, the error is lower for the rubber where data is more compact and less noisy, while it remains slightly higher for the cardboard and even higher for the foam. But in all cases, excellent convergence is achieved.
(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).
(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).
Deformation profiles for semi-stiff material (cardboard).
(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).
Deformation profiles for smooth material (foam).
Real and modeled deformation curves using neural network for semi-stiff material (cardboard) under a normal force of: a) F=0.1N, b) F=0.37N, and c) F=2.65N.
(a) (b) (c)
Real and modeled deformation curves using neural network for smooth material (foam) under a normal force of: a) F=0N, b) F=0.93N, and c) F=3.37N.
(a) (b) (c)
(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).
Real and modeled deformationcurves using neural network for rubber under a normal force of: a) F=0N, b) F=65.52N, and c) F=80.5N.
(a) (b)
(c)
(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).
(a) (b)
Real and modeled deformation curves using neural network for rubber under forces applied at different angles: a) F=65N, a1=10° and F=65N, a2=170°,b) F=36N, a1=25°, and F=36N, a2=155
(from .A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).
(from A.M. Cretu, E.M. Petriu, P.Payeur “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” submitted to IEEE Tr. Instr. Meas., Nov. 2006).
Real, modeled and estimated deformation profiles detail of estimated deformation profiles using neural network for rubber ball for increasing forces applied at 75-degree angle.
Hardware Neural NetworkArchitectures
ANNs / Neurocomputers =>architectures optimized for neuron model implementation
general-purpose, able to emulate a wide range of NN models;special-purpose, dedicated to a specific NN model.
Hardware NNs consisting of a collection of simple neuron circuits provide the massive computational parallelism allowing for a higher Model rendering speed.
ANN VLSI Architectures:• analog ==> compact,high speed,
asynchronous, no quantizationerrors, convenient weight “+”and “X”;
• digital ==> more efifcient VLSI technology,robust, convenient weight storage;
Pulse Data Representation:• Pulse Amplitude Modulation (PAM) -not satisfactory for NN processing;
• Pulse Width Modulation (PWM);• Pulse Frequency Modulation (PFM).
Number of nodes
0
103
106
109
1012
103 106 109 1012 Node complexity[VLSI area/node]
RAMsSpecial-purpose neurocomputers
General-purpose neurocomputersSystolic arrays
Computational arays
Conventional parallel computers
Sequential computers
[from P. Treleaven, M. Pacheco, M. Vellasco, “VLSI Architectures for Neural Networks,”IEEE Micro, Dec. 1989, pp. 8-27]
Pulse Stream ANNs: combination of different pulse data representation methodsand opportunistic use of both analog and digital implementation techniques.
HARDWARE NEURAL NETWORK ARCHITECTURES USING RANDOM-PULSE DATA REPRESENTATION
Looking for a model to prove that algebraic operations with analog variables can be performed by logical gates, von Neuman advanced in 1956 the idea of representing analog variables by the mean rate of random-pulse streams [J. von Neuman, “Probabilistic logics and the synthesis of reliable organisms from unreliable components,” in Automata Studies, (C.E. Shannon, Ed.), Princeton, NJ, Princeton University Press, 1956].
The “random-pulse machine” concept, [S.T. Ribeiro, “Random-pulse machines,” IEEE Trans. Electron. Comp., vol. EC-16, no. 3, pp. 261-276,1967], a.k.a. "noise computer“, "stochastic computing“, “dithering”deals with analog variables represented by the mean rate of random-pulse streams allowing to use digital circuits to perform arithmetic operations. This concept presents a good tradeoff between the electronic circuit complexity and the computational accuracy. The resulting neural network architecture has a high packing density and is well suited for very large scale integration (VLSI).
Interactive telemanipulation applications require real-time rendering of complex NN models
HARDWARE ANN USING MULTI-BIT RANDOM-DATA REPRESENTATION
Generalized b-bit analog/random-data conversion and its quantization characteristics
VRV
RVRQ
CLOCKCLK
VRP
b-BITQUANTIZER
X XQ
ANALOG RANDOMSIGNALGENERATOR
-D/2 0R
p(R)1/D
+D/2
++
.(k+0.5) D(k-0.5) D.
XQ
X
k
k+1
k-1
0
b D.
1/Dp.d.f.of VR
D/2D/2
b D. (1- b) D.
.V= (k- b) Dk D.
E.M. Petriu, L. Zhao, S.R. Das, V.Z. Groza, A. Cornell, “Instrumentation Applications of Multibit Random-Data Representation,” IEEE Trans. Instrum. Meas., Vol. 52, No. 1, pp. 175- 181, 2003.
32 266 5003.2
1
1.2
x2is
x2ditis
x2RQis4
2
dZis
dHis
dLis
MAVx2RQis
is
Moving Average ‘Random Pulse -to- Digital ” Conversion
2-bit random-data NN architectureof an auto-associative memory
P1, t1 P2, t2 P3, t3
Training set
30
P30x1
30x30
n
30x1
a
30x1W
)*hardlim( PWa =
Recovery of 30% occluded patterns
Model-based approach, based on the kinematics and dynamics of the object handled with the fingertips, provides a convenient representation of the dexterous manipulation. “Improved accuracy and richness in object modeling and haptic rendering will require advances in our understanding of how to represent and render psychophysically and cognitively germane attributes of objects, as well as algorithms and perhaps specialty hardware (such as haptic or physics engines) to perform real-time computations” [K. Salisbury, F. Conti, F. Barbagli, “Haptic Rendering: Introductory Concepts,” IEEE Computer Graphics and Applications, Vol. 24, No. 2, pp. 24 – 32, 2004].
Neural Networks which are able to learn nonlinear behaviors from a limited set of measurement data can provide efficient and compact multi-media object modeling solutions. Due to their continuous, analog-like, memory behavior, NNs are able to provide instantaneously an estimation of the output value for input values that were not part of the initial training set.
NNs consisting of a collection of simple neuron circuits provide the massive computational parallelism offering efficient storage, model transformation, and real-time rendering capabilities for large numbers of composite geometric & haptic object models involved in the model-based interactive telemanipulation.
Conclusions
S. Andrews, “Measurement-Based Modeling of Contact Forces and Textures for Haptic Rendering,” M.Sc. Thesis, 2007.
J.-C. Delannoy, “Modeling Articulated Bodies with Distributed Mass for Virtual Interactive Environments,” M.A.Sc. Thesis, 2006.
A.-M. Cretu, “Neural Network Modeling of 3D Objects for Virtualized Reality Applications,” M.A.Sc. Thesis, 2003.
A. Moica, "Coprocessor for Decoding Multi-Valued Pseudo-Random Sequence Patterns," M.A.Sc. Thesis, 2000.
L. Zhao, "Random Pulse Artificial Neural Network Architecture," M.A.Sc. Thesis, 1998.
S.S.K. Yeung, “Model-Based Tactile Object Recognition Using Pseudo-Random Encoding“, Ph.D. Thesis, 1996.
C. Archibald, “A Computational Model for Skills-Oriented Robot Programming,“ Ph.D. Thesis, 1995.
D.M. Colven, "Tactile Pattern Recognition Using Neural Networks," M.A.Sc. Thesis, 1993.
Ottawa “U” Research Group – Relevant Graduate Theses
J. Zhou, N.D. Georganas, E.M. Petriu, X. Shen, F. Marlic, “Modelling Contact Forces for 3D Interactive Peg-in-Hole Virtual Reality Operations,” Proc. I²MTC 2008 – IEEE Int. Instrum. Meas. Technol. Conf., pp. 1397-1402, Victoria, BC, Canada, May 2008.
E.M. Petriu, A.-M. Cretu, P. Payeur, “Neural Network Modeling Techniques for the Real-Time Rendering of the Geometry and Elasticity of 3D Objects,” Proc. SOFA 2007, 2nd IEEE Int. Workshop on Soft Computing Applications, pp. 11-16, Gyula , Hungary, and Oradea. Romania, Aug. 2007.
A.-M. Cretu, E.M. Petriu, “Neural-Network –Based Adaptive Sampling of Three Dimensional-Object Surface Elastic Properties,” IEEE Trans. Instrum. Meas.," Vol. 55, No. 2, pp. 483 - 492, 2006.
A.-M. Cretu, E.M. Petriu, G.G. Patry, “Neural-Network-Based Models of 3-D Objects for Virtualized Reality: A Comparative Study,” IEEE Trans. Instrum. Meas.," Vol. 55, No. 1, pp.99-111, 2006.
A.-M. Cretu, E.M. Petriu, P. Payeur, “Neural Network Mapping and Clustering of Elastic Behavior from Tactile and Range Imaging for Virtualized Reality Applications,” Proc. IST 2006, IEEE Intl. Workshop on Imagining Systems and Techniques, Minori, Italy, pp.17–22, April 2006.
A.-M. Cretu, J. Lang, E.M. Petriu, “A Composite Neural Gas-Elman Network that Captures Real-World Elastic Behavior of 3D Objects,” Proc. IMTC/2006, IEEE Instrum. Meas. Technol. Conf., pp. 1063-1068, Sorrento, Italy, April 2006.
E.M. Petriu, S.K.S. Yeung, S.R. Das, A.-M. Cretu, H.J.W. Spoelder, “Robotic Tactile Recognition of Pseudorandom Encoded Objects,” IEEE Trans. Instrum. Meas., Vol. 53, No. 5, pp. 1425-1432, 2004.
Ottawa “U” Research Group - Publications in Modelling3D Object Geometry and Elastic Properties