95

Development of Micro-manipulation System for Operationin Scanning Electron Microscope

H. Eda, L. Zhou, Y. Yamamoto, T. Ishikawa, T. Kawakami and J. ShimizuSystem Engineering Department, Ibaraki University, Hitachi-shi 4-12-1, Japan, 316-8511

Tel: +81-294-38-5188 Fax: +81-294-38-5188

Abstract

The ultimate goal of this project is to develop amanipulation system enabling unskilled operatorsto deal with objects of micron or sub-micron size ina scanning electron microscope, as easily as todeal with objects in usual size. Described in thispaper is the results achieved in the first phase ofthe research, in which the focusing point is given tothe conceptual design, the prototype developmentand the operability evaluation. The system ismodularized into the manipulation unit, the controlunit and the man-machine interface. Themanipulation unit is further comprised of a twin-arm manipulator mounted on a rotary table and aspecimen stage with four degrees of freedom linearalong X, Y and Z direction, and rotational aroundthe Z-axis. The manipulator is driven by PZTactuators with magnifier elements and able to coveran envelope as wide as 200µm for each axis of X, Yand Z.

Instead of doing a direct operation, the operatorsteers the manipulator via an user-friendlyinterface which is designed to absorb the opticaland mechanical variations. It allows the operatorto concentrate to the manipulation without payingmach attentions to the changes in magnification ofSEM or other conditions. The control unit mergesthe visual information of the SEM and themanipulation information from the user interfaceand derives the optimum locomotion of the arm forthe desired operation.

Keywords: micro-manipulator, micromachining,PZT actuator, scanning electron microscope,man-machine interface

1 Introduction

In the applications of precision, medical andbiological engineering, there are increasingdemands of implementing fabrication, assemble,inspection, modification and/or evaluation at themicron or sub-micron scale1) 2). Currently, most ofsuch operations are manually done by highly skilledoperators with the assistance of scanning electronmicroscopes (SEM). It is obviously subject to themagnification of the SEM. The limited visible areaat a high magnification of SEM makes it difficult

for the operator to find and trace the target. Theoperators have also to experience a long timetraining to familize the feeling of micro-scalemovements in order to achieve high repeatabilityand high accuracy. Yet, the operations are tediousand time consuming. It is, therefore, stronglyexpected to develop a manipulation system possiblefor unskilled operators to easily execute the desiredtasks.

As the first step of research, a manipulation system(including hardware and software) has beendesigned and developed to smoothly and

continuously execute a series of micro-operationssuch as cutting-grasping- transferring, at different

Controlmodule

Mechanismmodule Man-machine

interface module

Fig.1 System concept

Operationpanel

Man-machineinterface module

Vision　system

SEM

Controller moduleMechanism module

Manipulator

Fusi

onF u

sion

Fig.2 Main modules and key components

108

SEM magnifications. The scope of this paperincluded the conceptual design, the prototypedevelopment, field tests and the operabilityevaluation. Especially, a parallel plate type forcesensor has been newly developed and incorporatedinto the manipulator arms to deal with the microobjects from as soft as tissues to as hard as metals,which has been considered difficult to realize withthe conventional tools.

2 Conceptual design

The system has been developed with a basicconcept that the unskilled operators can intuitivelywork on the desired tasks without paying muchattention to the changes in working conditions likethe object size, SEM magnifications and etc..Instead of manipulating the object directly, thedeveloped system allowed the operator to have agood control of the manipulators via an interfacewith a great workability. The interface wasdesigned to absorb the mechanical and opticalvariations by using the visual information and sensethe tiny force exerting on the objects, to enhancethe repeatability and accuracy. With suchconfiguration, the developed system made itpossible for the operator to concentrate on the tasksbased on the visual information.

The conceptual layout of the system was shown inFig.1 and Fig.2. The system was comprised ofthree main modules; the manipulation mechanism,the controller and the man-machine interface.Each module was responsible for different function.The mechanism module was set inside the vacuumchamber of the SEM for the object manipulation aswell as the direct observation. The interfacemodule with a great workability provided the visualinformation to and retrieved the instructions fromthe operator. The other tasks including themeasurement, image processing and necessaryforce feedback were automatically executed by thecontrol module.

The mechanism module further incorporated atwin-arm manipulator, a movable specimen stageand a rotary table. Through the interface, theoperator was able to remotely move the left andright arms as if his own. To begin with, the visionsystem of the control module captured the imagevia a the microscope (SEM), abstracted anddisplayed the information necessary for thefeedback control and manipulation. Based on thevisual information, subsequently, the operator gaveinstruction to the control module. By fusing thevisual information and operational information thatthe operator gave, the control module derived thecommanding information for the movements of themanipulators. At the moment the manipulatorinteracted with the target, the force detected was

converted into the resistance of the joystick so thatthe operator could exactly feel the interferencebetween the manipulator and target. Afterward, theoperation was carried out at the continuous visualand force feedback.

3　　　　Module description

3.1　　　　Mechanism module

The detailed drawing of the mechanism modulewas schematically shown in Fig.3. Themanipulator had a twin-arm configured in the left-right symmetry. Each arm came with threeactuators superimposed one on the top of another inorder to move along the XYZ axial direction. Asshown in Fig.4, the actuator was driven by thepiezoelectric element and its displacement wasenlarged 20 times by a parallel plate structuredmagnifier. The working range was from 0.1µm to200µm for each direction of X, Y and Z axis.

Similar to the human hands, the manipulator took

twin-arm configuration. Each arm was equipped

PZT

Fig.4 PZT actuator and magnifier

Fig.5 Rotary table with the mechanism of rotating aroundany specific center

Fig. 3 Manipulation mechanism

Rotary table Twin arm manipulator

Specimenstage

Z

XY

Instant rotationalcenter

DCMotor

Roller pair

PZT actuator

109

with a parallel plate type force sensor to detect theforce ranging from 90µN to 30mN, which isresponse to the fabrication of soft/low strengthobjects and hard/high strength objects as well. Theindividual arm could play different role orcooperate each other for complicated tasks. Forexample, one arm held the object, while anotherarm executed the concrete operations such ascutting, peeling, removal and carrying.

The manipulator was mounted on a rotary table. Asillustrated in Fig.5, the table was supported by apair of rollers set perpendicularly and driven by DCmotors. The attitude of the roller pair wasadjustable within 3 degrees by the PZT actuatorstogether with parallel plate magnifiers. The pointwhere the lines perpendicular to the roller pairintersect defined the instant rotational center of thetable. It meant that the rotational center iscontrollable by the attitude of the roller pair, so thatthe table rotated around any specified center. Withassistance of such a rotary table, the system wasable to position the manipulator against the targetfrom any desired direction.

The specimen was seated on the specimen stagewhich is movable linearly at the X, Y and Zdirection, and rotationally around the Z-axis. The

movable range was ±3mm for each axis and 360°for rotation. The mechanism of the specimen stagewas detailed in Fig.6. Fig.7 showed themanipulator module assembled on the SEMchamber kit.

3.2　　　　Interface module

As described previously, the system developmentwas emphasized on the point of how to realize theoperations desired by the unskilled operators. This

consideration resulted in a concept of that theoperator, instead of doing a direct operation,handled manipulator via the user-friendly interfacewith great workability. Shown in Fig.8 was theoperation panel of the interface module. Joystickswere chosen as the operation tool for its highperformance. Out of four joysticks, two in mostright and left were used for the control of the rightand left arms. Their transverse and longitudinaltravels were respectively synchronized to themovements of arms in X- and Y-axial direction.

The arms moved in the Z axial direction when thejoysticks were twisted. The moving speed wassubject to the magnification of the SEM, so that themanipulator always traveled with a constant speedcross the image display, regardless of the change inSEM magnifications.

The movements of the rotary table and thespecimen stage were automatically run by thecontrol module, to visually captured the object, tolocate it within the scope and to absorb variousmechanical and optical variations. In case ofmanual operations required, the other two joystickswere available to provide the functions as same asmentioned above. This arrangement enabled theoperator to intuitively operate the manipulator withfeeling as same as he moves his own arms.

Unlike the conventional one-way command inputtools, the newly developed joysticks could

Z

YX

DC motor for Y

-axis

DC motor for Z-axis

DC motor for θ-axis

Fig.6 Specimen stage

Left armcontroller

Right armcontroller

Specimen stagecontroller

Rotary tablecontroller

Contactindicators

Potentialmeters

Fig.8 Man-machine interface

Fig.7 Manipulator assembled on SEMchamber kit

110

bidirectionally communicate with the manipulationmodule3) 4). It meant that the joysticks were able togive the instructions for movements whileretrieving information from the manipulator. Toremotely manipulate a tiny object in SEM, forexample, it was effective to utilize not only thevisual and position information but also theinformation of force exerted on the object. In thissystem, the strength of the working force wascontinuously measured by the strain gaugesattached in the manipulator tip. By feeding thedetected forces back to the joystick via a torquemotor which could generate a constant torqueproportional to the driving current, the user actuallysensed whether the contact was made to and howmuch the force was applied. By feeding thedetected forces back to the PZT actuator,additionally, the active force control has beenrealized to maintain a constant force between themanipulator and the object being manipulated.With force control, the contact point between themanipulator and object became “floating” innature5) so as not to undesirably damage the object,either to remove excessive material from the object.

3.3　　　　Control module

One major function of the control module was toautomatically track and locate an arbitrarily shapedobject when the visible area or the SEMmagnification is dynamically changed during theoperation. The current research has developed animage-processing algorithm to capture and trackany specified object. The target, it could be on thespecimen and/or the manipulator tip, was firstdefined or selected by simply clicking the mousebutton and related to the actuator (the specimenstage or the rotary table) where it sat on. Thedeveloped algorithm then digitized the imagecaptured and converted the pixel into the position

information. Subsequently, when the SEMmagnification changes, the control module drovethe specimen stage or the rotary table to keeptarget(s) always within the scope.

In addition to the auto-track function based on thevisual information, the control module alsointerpreted the instructions given to the manipulatorby the operator into the appropriate movement andspeed to match the current SEM magnification. Itoffered a flexible environment for the operator towork easily yet efficiently.

Shown in Fig.9 was the information flow in thecontrol module. The high performances carried outin the controller were all realized on a personalcomputer. Such environment made it possible foroperator to intuitively work out the tasks based onthe visual and force information, even withoutfeeling changes of the magnification.

4　　　　Field tests

In order to evaluate the capability of the developedsystem, several functional tests were carried toperform cutting, marking, transferring, moving,lifting and flipping-over. Two of them, cutting andpeeling/transferring, were demonstrated below.4.1　　　　Cutting

The object shown in Fig.10 (a) was a star-shape cellof goumi leaf. The manipulators were equippedwith sharp needles as their operation tools. The testwas designed to investigate the cooperativity of thetwin-arm manipulator and specimen stage. Therequired task was to “cut the cell into half”, whichis one of the operations most frequently performedto study the physical/chemical properties insidecells.

Twin armManipulator

SEM

Strainamp

Straingauge

XYZXYZXYZXYZ

XYZXYZXYZXYZ ActuatordriverPZT

A/Dboard

D/Aboard

CPU

D/Aboard

A/Dboard

TorquemotorAmp

JoystickRotary

table

AmpDC

motorωωωω

θθθθ ActuatordriverPZT

Specimenstage Amp

DCmotor

XYZXYZXYZXYZθθθθ Motordriverboard

Image processingboard MonitorVisual informationVisual informationVisual informationVisual information

XYZXYZXYZXYZ

XYXYXYXYθθθθ

XYZXYZXYZXYZ

XYZXYZXYZXYZθθθθ

Potential

meter

Potential

meter

Fig.9 Information flow

111

(a) Contact

(b) Stripping

(c) Lifting

(d) TransferingFig.11 Field test 2: Peeling/transferring

Epidermis

100µm(a) Tool tip and target

(b) Approaching while visual area changed

(c) Right arm holds while left arm cuts the target

(d) Target cut offFig.10 Field test 1: cutting

Cell cut-off

Left armRight arm

100µm

Star-shape cell

112

In the photo (b), the target cell was manuallymoved to the center of the scope and themanipulators were positioned at where theoperation was intent to start with. Once the targetand the relevant tool tip were specified andselected, they were always located within the SEMscope by the auto-track function. Regardless thechanges in the SEM magnification, the operator feltno visual lag and could immediately start thenecessary operation.

In the photo (c), the right arm came to hold the tophalf of target cell, while the left arm cut off theother half. The photo (d) showed the cell aftercompletion of the cutting.

4.2　　　　Peeling/transferring

The objective of the experiment shown in Fig.11was to “latch” the epidermis of the botanical tissue,“strip” it off the body and “transfer” it to a requiredplace. The task placed the importance oncooperation between arms and force feedback.

As shown in (a), the specimen stage was firstshifted to locate the point for operation. Then, theleft arm was made a contact to the specimen andlatched the targeted substance. Occasionally, thisoperation was possibly completed by the left armonly. For most cases, however, it requiredcooperative operation between the manipulators,rotary table and specimen stage to properly positionboth manipulators arms and specimen.

In the photo (b), the right arm came to help the leftarm to hold the target. The subsequent “striping”task was intuitively done by lifting both armstogether. It was very difficult for conventionalmanipulators to keep the relative position of tipsunchanged while moving arms in 3-D space. Forthis case shown in (c), the active force control wasutilized to sustain the object, with the newlydeveloped algorithm which was able to establish aconstant force for the interaction between right andleft arms.

After being separated from its body, the targetepidermis was transferred to another place. Duringthe transferring, as shown in (d), the relativeposition between the right arm and the left arm wasfrozen by the active force control. Also, the auto-track function made the target always visible whenthe arms as well as the target traveled a longdistance even exceeding the current scope of theSEM

5 Conclusion

This paper has described the micro-manipulationsystem developed for operations in SEMenvironment and its performance in the field test.The developed system united both observation and

manipulation together, which are normally done indifferent platform, in a single set-up. It allowed theoperator to perform necessary fabrications duringthe observation, also to monitor the manipulationon-site. The achievements were summarized asfollows;

(1) A twin-arm micromanipulation system, whichwas able to cooperatively manipulate theobjects from as hard as tissue to as soft asmetals, has been developed for SEMenvironment.

(2) A parallel plate type force has been developedand cooperated into the tool tip to sense theinterfering force between the tool andspecimen. The force feedback control not onlyprovided the actual resistance to the man-interface (joystick), but also actively controlledthe force being constant during the operation.

(3) The image processing algorithm has beendeveloped to countermeasure the missingtarget problem. The vision feedback systemcould automatically track any specified targetand locate it with in the scope of SEM. Theinstructions governing the manipulatormovement were also translated to anappropriate speed and distance based on thevisual information, to match the current SEMmagnification.

The field tests had shown that the basic concept ofthe developed system is applicable to themicromanipulation such as cutting, grasping,peeling and transferring. The future research willinclude skill analysis, knowledge baseestablishment, auto/manual mode development andman-machine interface evaluation6).

Acknowledge

This work was financially supported by theMinistry of Education, Science and Culture (Japan)under the Fundamental Research Type A(10305012).

References

1. Marc Madou: Fundamentals of MicroFabrication, CRC press, 1997

2. S T Smith and D G Gatwynd: Fundamentals ofUltra Precision Machine Design, Gordon andBreach Science Publishers, 1992

3. Dieter Vischer: Cooperating Robot with Visualand Tactile Skills, Proceedings of IEEEInternational Conference on Robot andAutomation , 1992

4. S Hirai, et. al: Integration of Task KnowledgeBase and a Cooperative Maneuvering System,IEEE International Workshop in IntelligentRobots and System, 1990

113

5. T L Graf: Deburring, Finishing and GrindingUsing Robots and Fixed Automation: Methodsand Applications, Technical paper, 3MAbrasive System Division,1993

6. H Fujita: Micro Machine World, KogyoChousakai,1993

114

95 95