6
Skill analysis of human in machine operation Satoshi Suzuki Kanda Branch of the 21st Century COE Project Office Tokyo Denki University 1-18-13, Soto-Kanda, Chiyoda-ku, Tokyo 101-0021, Japan E-mail: [email protected] Yaodong Pan Hatoyama Branch of the 21st Century COE Project Office Tokyo Denki University Ishizaka Hatoyama-chyo, Saitama 350-0394, Japan E-mail: [email protected] Abstract-A purpose of this paper is to discover catholic characteristic of human skill in machine manipulation for re- alization of a Human Adaptive Mechatronics(HAM). HAM is a novel concept of an intelligent mechanical systems that adapt themselves to the user's skill and assist to improve the user's skill. This paper reports experimental analyses of human characteristic in machine operation by utilizing a stabilization task. By using a force-feedback haptic device and a real-time computer graphics of a virtual pendulum, processes of subjects to be skilled were observed, and three kinds of analyses were applied. The first is probabiity distribution analysis of motion, the second is moni- toring of brain cortical by a near-infrared spectroscopy(NIRS), and the last is correlation analysis to estimate human control law. It could be confirmed that human who mastered the stabilization task sufficiently (i)utilizes sensory information of wide body part, (ii)reinforces on active visual sensing, and (iii)predicts velocity information by using dynamics model which is formed in brain. I. INTRODUCTION A. Proposal of 'Human Adaptive Mechatronics' Mechatronics is known as the discipline integrated by me- chanical, electrical and information technology and has been used to produce advanced artifacts used in modem society. Our modem life, which has been developed through the interdis- ciplinary studies of diverse engineering fields is surrounded and enhanced by gadgets of mechatronics products. It is expected to develop mechatronics systems which are capable of adapting themselves to the level of the skill or dexterity of humans who use the systems. We need to found a new mechatronics discipline considering human in the closed loop. Against the backdrop of these topics, our Tokyo Denki University proposed a new concept of a human-machine sys- tem called 'Human Adaptive Mechatronics(HAM)'[ 1]. HAM is defined as an "intelligent mechanical systems that adapt themselves to the user's skill under various environments, assist to improve the user's skill, and assist the human-machine system to achieve best performance". This research was se- lected as one of 'The 21st Century COE(Center of Excellence) Program', which is supported by the Japanese Ministry of Fumio Harashima Department of Electrical Engineering Tokyo Denki University 2-2 Kanda-Nishiki-cho, Chiyoda-ku, Tokyo 101-8457, Japan E-mail: f.harashima(ieee.org Katsuhisa Furuta Department of Computers and Systems Engineering Tokyo Denki University Ishizaka Hatoyama-chyo, Saitama 350-0394, Japan E-mail: furuta(k.dendai.acjp Education, Culture, Sports, Science and Technology with the aim to form the world highest level research education bases. B. Elucidation of human skill We think the following items are needed for HAM in the present stage, . definition and quantification of human skill . cognition method of human behavior from machine-side . assistance method for human by a machine . change of machine's function for total enhancement First of all, quantification of human skill is needed for the HAM. It is a natural idea that modeling of human controller is needed. In control engineering, research on modeling of human behavior had been continued from the early days. It is said that Tustin tried first to express a human control model in 1940s when classic control theories were systematized. He utilized a linear transfer function to model human action and proposed a linear servo control[2]. He also indicated that human control action contained considerable non-linearity and that a human has functions of an adaptation, prediction and optimization. In the 1960's, many models to express human control properties were introduced. Ragazzini[3] modeled a human as a PID controller and indicated that a human is time-variant system having randomness. He claimed that we should pay attention to differences among individuals. The work of Baron in 1970 showed good agreement between theory and experiment in a scheme of an optimal control for a VTOL aircraft[4]. Recently, thanks to collaboration with brain science and system engineering, a research on brain has become more active and human's cerebration logic is being elucidated. The feedback/feedforward model[5] and a Smith predictor[6] are well known models of brain's control strategy. The former model proposes that human brain tends to change from 'feedback' to 'feedforward' by learning. The latter model says that human brain forms models of delays caused by neural signal transmissions and recognition and utilizes them to make controllers in his(her) brain. These models are ad- equate to treat a process of human skill because they were 0-7803-9422-4/05/$20.00 ©2005 IEEE 1556

[IEEE 2005 International Conference on Neural Networks and Brain - Beijing, China (13-15 Oct. 2005)] 2005 International Conference on Neural Networks and Brain - Skill analysis of

  • Upload
    k

  • View
    213

  • Download
    0

Embed Size (px)

Citation preview

Skill analysis of human in machine operationSatoshi Suzuki

Kanda Branch of the 21st Century COE Project OfficeTokyo Denki University

1-18-13, Soto-Kanda, Chiyoda-ku, Tokyo 101-0021, JapanE-mail: [email protected]

Yaodong PanHatoyama Branch of the 21st Century COE Project Office

Tokyo Denki UniversityIshizaka Hatoyama-chyo, Saitama 350-0394, Japan

E-mail: [email protected]

Abstract-A purpose of this paper is to discover catholiccharacteristic of human skill in machine manipulation for re-alization of a Human Adaptive Mechatronics(HAM). HAM isa novel concept of an intelligent mechanical systems that adaptthemselves to the user's skill and assist to improve the user's skill.This paper reports experimental analyses ofhuman characteristicin machine operation by utilizing a stabilization task. By using aforce-feedback haptic device and a real-time computer graphicsof a virtual pendulum, processes of subjects to be skilled wereobserved, and three kinds of analyses were applied. The first isprobabiity distribution analysis of motion, the second is moni-toring of brain cortical by a near-infrared spectroscopy(NIRS),and the last is correlation analysis to estimate human control law.It could be confirmed that human who mastered the stabilizationtask sufficiently (i)utilizes sensory information of wide body part,(ii)reinforces on active visual sensing, and (iii)predicts velocityinformation by using dynamics model which is formed in brain.

I. INTRODUCTION

A. Proposal of 'Human Adaptive Mechatronics'Mechatronics is known as the discipline integrated by me-

chanical, electrical and information technology and has beenused to produce advanced artifacts used in modem society. Ourmodem life, which has been developed through the interdis-ciplinary studies of diverse engineering fields is surroundedand enhanced by gadgets of mechatronics products. It isexpected to develop mechatronics systems which are capableof adapting themselves to the level of the skill or dexterityof humans who use the systems. We need to found a newmechatronics discipline considering human in the closed loop.

Against the backdrop of these topics, our Tokyo DenkiUniversity proposed a new concept of a human-machine sys-tem called 'Human Adaptive Mechatronics(HAM)'[ 1]. HAMis defined as an "intelligent mechanical systems that adaptthemselves to the user's skill under various environments,assist to improve the user's skill, and assist the human-machinesystem to achieve best performance". This research was se-lected as one of 'The 21st Century COE(Center of Excellence)Program', which is supported by the Japanese Ministry of

Fumio HarashimaDepartment of Electrical Engineering

Tokyo Denki University2-2 Kanda-Nishiki-cho, Chiyoda-ku, Tokyo 101-8457, Japan

E-mail: f.harashima(ieee.org

Katsuhisa FurutaDepartment of Computers and Systems Engineering

Tokyo Denki UniversityIshizaka Hatoyama-chyo, Saitama 350-0394, Japan

E-mail: furuta(k.dendai.acjp

Education, Culture, Sports, Science and Technology with theaim to form the world highest level research education bases.

B. Elucidation of human skillWe think the following items are needed for HAM in the

present stage,. definition and quantification of human skill. cognition method of human behavior from machine-side. assistance method for human by a machine. change of machine's function for total enhancement

First of all, quantification of human skill is needed for theHAM. It is a natural idea that modeling of human controlleris needed. In control engineering, research on modeling ofhuman behavior had been continued from the early days. It issaid that Tustin tried first to express a human control modelin 1940s when classic control theories were systematized.He utilized a linear transfer function to model human actionand proposed a linear servo control[2]. He also indicated thathuman control action contained considerable non-linearity andthat a human has functions of an adaptation, prediction andoptimization. In the 1960's, many models to express humancontrol properties were introduced. Ragazzini[3] modeled ahuman as a PID controller and indicated that a human istime-variant system having randomness. He claimed that weshould pay attention to differences among individuals. Thework of Baron in 1970 showed good agreement betweentheory and experiment in a scheme of an optimal controlfor a VTOL aircraft[4]. Recently, thanks to collaboration withbrain science and system engineering, a research on brain hasbecome more active and human's cerebration logic is beingelucidated. The feedback/feedforward model[5] and a Smithpredictor[6] are well known models of brain's control strategy.The former model proposes that human brain tends to changefrom 'feedback' to 'feedforward' by learning. The latter modelsays that human brain forms models of delays caused byneural signal transmissions and recognition and utilizes themto make controllers in his(her) brain. These models are ad-equate to treat a process of human skill because they were

0-7803-9422-4/05/$20.00 ©2005 IEEE1556

proposed with assumption that human characteristics changes.We can find several studies on skill[7], but there is not muchreport that focuses on both elucidation of 'skill' axiom andmachine operation. Especially, most approaches that treat skillin machine operation limit their scope to each work task. Forinstance, in an assist control of a crane reported in reference of[8], the support was executed by utilizing adequate referencepatterns and/or of expert's models. The result is good, but theconclusion has less generality for elucidation of human skill.A purpose of this paper is to discover catholic characteristic

of human skill on machine manipulation for realization ofa HAM. Especially, a brain monitoring by a near-infraredspectroscopy(NIRS) was used. As a task of a machine manip-ulation to investigate skill, a balancing control was adopted byreferring Cabrera's studies[9], [10].

This paper is organized as follows. Section II explains adeveloped haptic system to investigate skill and a measurementsystem of brain activation. Section III explains three methodsof analyses, and shows their results. The first is probabilitydistribution analysis of motion controlled by human, thesecond is an analysis of brain activation, and the last is anestimation of human controller by using a correlation analysisagainst actual logged data. Section IV presents the conclusion.

II. SET-UP OF MEASUREMENT SYSTEM FOR SKILLEVALUATION

A. Skill in balancing taskStabilization is basic action in a human-machine system. For

instance, general manipulation of a machine can be resolvedinto segments of simple motion controls, and their controlscan be translated into stabilization problem. And a balancingcontrol of an inverted pendulum is an adequate exampleto investigate human control property including a voluntarymotion and a visual process. Cabrera measured behavior ofhuman balancing a stick on his/her finger tip, and confirmedinteresting facts: (i) a human in balancing control adjustsposition of the hand more rapidly than time-delay of visual-voluntary-motion process, and (ii)there are two power lawregimes in a power spectrum on the fluctuation of stick'sincline angle. He concluded that a human utilizes physicalcharacteristics such that parametric noise can stabilize cyclicmotion. Additionally, he found that the expert has nonlinearcharacteristic, that is human tends to move his/her handquickly when a stick slants heavily. Cabrera's study is veryinteresting because he extracted inherent characteristic of skillfrom human behavior. If similar tendency can be found onmanipulation of a machine, his approach can be utilized as acatholic method for skill evaluation. Based on this idea, webuilt a haptic test system using a balancing task with a virtualpendulum, and experimental analyses were executed.

B. Experimental system setupVisual information is essential for most of human-machine

systems, and is a standard way to transmit information froma machine and the environment to human high-dimensionalfunction. Force sense is also important information to increase

maneuverability and is adopted in a part of physical man-machine interfaces. Therefore, we developed a haptic testsystem treating these kinds of information. A left photo ofFig. 1 shows the test system.The system composes from threesub-units: (i)a real-time monitor of a virtual pendulum model,(ii)a slider controlled by a virtual internal model and (iii) abrain monitor NIRS.

Fig. 1. Experimental scene(left) and head probe of NIRS(right)

The unit-(i) displays computer graphics(CG) of a pendulum.The real-time CG is made by Direct 3D library, and isdisplayed by a projector and was magnified in order to givefull-size image to the operator. The motion obeys the physicallaw of the dynamrics model, and changes by operator's forcethat is measured by actual force sensor. Dynamic equations ofa standard liner-type pendulum are given as

(j ± MP12)O + MplCo.,~ = -bO + mpglsO =: hi (1)mplc9O ± (MC ± rnP)xC = -Cc.iC + MPlS962 + fh =: 2

(2)

where J, mp,, m~,b, c~,21 and g are an inertia of the pendulumaround the center of gravity, masses of a pendulum-link anda cart, viscous coefficients of the pendulum and the cart,total length of the link, and gravity acceleration, respectively.Variables 0, xc and fh are an angle of the pendulum's link,a position of the cart, and a virtual exogenous force that iscomputed from measured force fh added by human. Nota-tions of cO and sO mean c 0 and s 0. The parameters ofpendulum are chosen as m, = mp= 1.6[g], 1 = 1[m], b =5.- i05, c = 0.0275, and simulates a light pendulum. Anoperator can feel as 100[g] pendulum in his(her) feeling byan effect of a scale factor a~= 60 between the slider andvirtual model's coordinates.The unit-(ii) is a physical interface, i.e. haptic device. The

device is made of one degree-of-freedom slider driven by alinear DD-motor. The operator manipulates a grip attachedto the slider as he(she) looks at projected CG of a virtualpendulum. The force added by the operator is detected by asensor attached to the grip. In order to move a slider as the cartof the virtual pendulum, a virtual internal model control[1I 1]is utilized. The slider is controlled so that it behaves like avirtual model having specified properties of mass, stiffness,and damping. The block diagram of these control scheme isshown in Fig. 2.

1557

,Lv_i5kDW15n .ICG-pendulumk- e

Fig. 2. Brock diagram of haptic system with virtual CG model

The unit-(iii), i.e. NIRS, can detect cortical changes inoxygenated/deoxygenated hemoglobin. NIRS has less accu-racy/slower time response than a fM, however NIRS isfeasible way. to measure the brain activity of the humanwho manipulates maehine[12]. This task needs processing ofvisual-voluntary motion, and it can be considered that variousareas in brain have relation to complete this complex task.Functions of brain are differentiated into local area roughly,and Brodmann's brain map is known as explanation of thelocalization(left of Fig.3). According to the map, some areasconcerning to motor control exist around central sulcus. Espe-cially, primary motor cortex(MsI) is most important area forvoluntary motion, and motions ofbody's muscle are controlledby local region in the cortex. The relation map is known as amotor homunculus(right of Fig.3). In primary somatosensorycortex(SmI), which is an opposite side of MsI against thecentral sulcus, sensor homunculus exists similarly like motorhomunculus. By considering these facts, to measure wholearea covering MsI and SmI, we prepared a narrow probe capby combining 3x34x4-3x3 sensor units as shown in a rightof Fig. 1. International 10-20 electrode system, which is anapplication method for electrodes of brain wave monitoringand based on the distance between the nasion and the inionof the scalps, was used to decide the position of the probe.Changes in the concentration of oxy-, deoxy-Hb, and totalHb(sum of oxy- and deoxy-Hb) were measured by the ETG-4000 system(Hitachi Medical Corporation, Tokyo, Japan) thatutilizes two wavelength 695nm and 830nm. The samplinginterval is lOOms.

motor area (SpA)primary motor cortex (36icentralsulcus

frontal yefeldfie IdsrimaryMot

W somatosens oy

corotexx(SmI.l

CortexJ

parietbc

::w,: :::aI S .: iAL: :ac

littlerang

middle

indexbrow

nek

eyelid & eyeballface

late rIentricie

toWg

inn ow

Fig. 3. Lateral surface of the cerebral hemisphere(left) and motor homuncu-1usQeft)

III. ANALYSIS AND RESULTSubjects were five males(age-22-23) and four fe-

males(age=21,22,52 and 55) with no history of neurologicaldeficits. Written consent and ethical approval were obtained

before the examinations. Subject sat in a chair and manipulatedthe slider with his(her) left backhand keeping touch to theslider. 10-time trials a day are imposed to each subject and90 seconds rest was given between trials. During rest time,the subject was said to be close his(her) eyes and so as notto receive any stimulation. A purpose of the investigation iscomparison of status of progress of skill learning. Hence, wechose three representatives after all trials were finished: high-skilled status(HS, female, age=22), middle-skilled status(MS,male, age=22) and low skilled status(LS, female, age=55).Figure 4 shows results of evolution of the three subjects. Thex-axis is the trial days, and the dots in y-axis direction showthe duration of stabilization of a balancing stick. The dottedline is the average of each day, and the solid line is changeof maximum duration. HS could get cue on 5th day, and shecould master rapidly the stabilization. MS was getting usedto manipulation gradually, but there was dispersion in theduration in even fourth day. We cannot find bigger progressin LS than the others.

Change of stabilized duration

4001

300

X 200

100 _ ...

0

(b)MS

.1 ..3

|||Xll;|Xtll8~~~ I &|U.-I slll.. ..i1 2 3 4

.. ...

..... ... .,.. .....

6

day

Fig. 4. Evolution of stabilized duration of three subjects

A. Probability Distribution of motion velocityNext analysis is an investigation of probability of cart's

velocity, and this analysis also was proposed by Cabrera [10].Making histograms of its velocity from logging-data of a cartgives Fig. 5, that are obtained by normalization of histogramsby bin= 0.05m/s. LS's distribution looks super-Gaussian andconcentrates to the center(low-velocity) range. In case of MS,however, we can find wide spread to higher velocity range.The spread becomes clearer in HS than in MS. This fact issame to Cabrera's direct hand balancing control. Hence, theoperator HS can be considered as sufficient skilled person.

B. Analysis of brain activation

48 ch data measured by the NIRS were analyzed by a prin-cipal component analysis(PCA). For n-multivariable sampled

1558

strength ofdecomposed modes

Zv

15 -(b)MS

O'0

O 50

/ 11

0 - 200 0 3 3 4 45 5100 150 200 250 300 350 400 450 S00

0v [mls]

Fig. 5. Comparison of probability distribution in cart's velocity

data sequence x(t) E R', an interval covariance matrix V iscomputed by

N

v = 1 X(t_i+l)_x(t-i+l)Ti=l

(c)LS 1stlO- A\ (.. 2nd

5L i~~~~~~ / ~~~~~A~morethan3rd

0 50 100 150 200 250 300 350 400 450tis [s]

Fig. 6. Intensity variation of modes decomposed by PCA

(3)

with N-sequence data from the current time t. Singular valuedecomposition(SVD) of V is computed as

Vo = UoEouoT, So = diag(a4(o.*' (0where ( > ... > o( . If unitary matrix Uo is denoted asU= [u ... u(O ], Vo can be decomposed as

VO = O'(0 *U(O U(o T + or(O *uU(O U( T + . .+ v( . U(UU(oV f1 U1 U1 ±2 U2 U2 + n.±T nu$un

Next, computing next SVD for V1 := Vo- u( u( Tgives

V1 = U1 1U[= [u(} ** u$}Jdiag(a(r('..$))u(T..u$T]T,

and oP is second principal component. Later, same manneris repeated, then a strength sequence of decomposed mode{ ,*(-.. }, and the corresponding distribution vectors{ u1 ,( , } are obtained. Hence visualization of thecomponents of v into a brain geometric map gives topographymap of decomposed mode.

To compare steady state of each subject, each longestduration trial of their last day were chosen as an analysistarget ofPCA. The moving interval was specified as N _ 100.Figure 6 shows each absolute value of the decomposed modesoI iI (i = 1 - 48) and their change. We can find that almostonly first mode of each subject is dominant. And HS's levelis very stronger than the others.From the former result, we judged that only first mode is

sufficient to investigate active status in the brain, hence weanalyzed a brain topography of first mode at the time before

Fig. 7. Topography image of first-mode by PCA

10 seconds from the end of each duration. The topographyimages on t = 2040(HS), t = 374(MS) and t = 388(LS) aredrawn in Fig. 7. The gray pattern expresses relative strengthlevel in decomposed distribution map. White area shows strongarea and black shows weak one. Note that the gray leveldoes not indicate common absolute strength in three images.Explanations of brain functional area and some regions byhomunculus are denoted in their images. Note that the relationbetween the region in brain and part at body is opposite inleft and right and that the homunculus map gives only roughinformation since brain does not be active always accordingto the homunculus map. On the later analysis, same index

1559

20,

Probability distribution of veloc-ity

U.Fb _ \

alphabet characters are used in sentences and figures forconvenience.

At first, activation circumstance of the HS was analyzed,and the following tendencies were found.

(a) Activation at wide range(knee, hip, trunk, shoulder,elbow and hand) in primary somatosensory cor-tex(SmI) is strong.

(b) There was strong activation in left and right premotorcortex(PMC: Brodmann no.6) and in wide supple-mentary motor area(SMA).

(c) Activation in right and left 'eyeball' regions of motorhomunculus in primary motor cortex(MsI) can berecognized.

(d) Activation in left 'arm' region in a MsI homunculusis stronger than the other right region, and theabsolute level is weaker than whole area.

Fact-(a) means that HS utilizes sensor information from widearea in body though it seems that she moves only hand mainlywith her body on a chair. We can guess that enhancementof sensory sensitivity to own body's change is important forskilled manipulation. It is known that a PMC has relation withvisual motion, and hence we can also guess from fact-(b) thatactivation in brain had been customized to visual feedback op-eration in order to observe sensitively the pendulum's motionby eyes. However, it is said that monitoring ofPMC is difficultsince a bone of parietal region is thick. Hence the descriptionof (b) may be doubtful.

Next, the MS and LS were analyzed by taking facts(a)-(d)into consideration.

(SmI) In both case ofMS and LS, there is no wide activatedregion. Only local activated region exists(comparedwith (a))

(PMC) Both left .and right sides of PMC in cases of LSand MS are not activated. Only left hemisphere ofMS and right hemisphere of LS are active.

(eyeball in MsI) MS is active but not so strong(c'), andLS is not active(c"i).

(arm in MsI) Region of 'right-arm' in MsI of MS isactive(d'). In case of LS, regions of both 'arm's inMsI and SmI are active(d").

Comparing facts of an expert HS and additional facts of non-experts MS/LS leads the following summaries.

. High skilled operator has high activity at sensory wideregion and at eyeball area in motor region, but a motorcontrol region is not so rather strong.

. Low skilled operator has tendency of activation mainlyon sensor region in SmI and motor region in MsI, i.e.,this kind of person tends to concentrate only on controlof arm.

C. Estimation ofhuman controllerFrom results of NIRS analysis, we find that observation of

motion is important for skill. To estimate how the expert couldacquire the control strategy, we investigated correlation analy-sis between pendulum's motion and force that is generated by

human. Since human has strong nonlinearity including time-delay, discontinuous, and time-varying, an identification ofhuman controller is difficult generally. To succeed in reliableidentification, it is necessary to decide structure of humancontroller. A purpose of analyses mentioned in this subsectionis to derive some information to identify human control law.

It can be guessed that pendulum's angle information isutilized for the stabilization mainly. And it is known that thereis time-delay in visual-voluntary motion process[6]. Hence weassumes that simplest human controller is approximated asfollows.

fh(t) a19(t-A1) b1, (4)where a, corresponds to a proportional gain against pendu-lum's angle, A1 is the time-delay, and b, is a constant tocompensate a bias existing in an actual system. By usingactual logging data sequence { 0(t), fh(t) } ofwhen HS couldsucceed in stabilizing for 230 seconds, parameters a1 and biwere estimated by a method of least squares. The estimationwas done against each A1 that is changed from t = -3[s]to t = 3[s] at 10[ms] intervals. The results are shown inFig. 8. On each graph, al, b1, correlation coefficient r, andidentification error el are shown in order. el is a mean valueof error of fh(t)-(alO(t-Al) bl) with identified parametersa, and bl. A minimum point of el and a maximum point ofri are same at A1 = 0.24[s]. This fact means proportionalcontrol against the pendulum angle includes 240 ms delay.It is said that a delay of human visual-voluntary motion is0.2 ,- 0.4, hence this analytic result are considered reasonable.The correlation coefficient r, at same point is maximum as0.53, but is not sufficiently close to 1. Therefore we can guessthat this proportional control is not dominant factor.

6

42 .......

-3 -2 -1 1 2 30.1

"5- 0 . ..

-0.1-3 2 -1 0 1 2 3

~0.5 A0-3 -2 -1 0 1 2 3

Al [s]

Fig. 8. Correlation analysis of human proportional control

1560

Next, similar analysis was applied to the following controllaw assumed.

fh(t) = a2O(t- A2) + b2, (5)

where a2 corresponds to a derivative gain, and A2 is the time-delay. Figure 9 shows results of this case. From the graphs,the error is small at A2 - -0.03[s]. A negative sign of A2means that the experts generates her force by predicting ofthe velocity information. In other words, the expert forms adynamics of behavior model of pendulum inside her brain.Maximum of the correlation coefficient max(r2) - 0.93 isbigger than max(rl) = 0.53, hence the expert utilizes angularvelocity information strongly than simple angle info-rmation.We can conclude that (a)control based on a predicted velocityinformation by using an virtual pendulum model which isformed inside her brain, and (b)visual observation that isneeded for the prediction, play an important role in thismachine manipulation.

6

4

-3 2 - 2 3

I. I-0.1

-3 -2 -1 0 1 2 3

2

A2 [SI

Fig. 9. Correlation analysis of human derivative control

IV. CONCLUSIONTo reveal human skill on manipulation of a machine, several

kinds of analyses were applied to the balancing task of a virtualpendulum. From an investigation of probability distribution onvelocity of human control, an additional wide spread to highervelocity range to Gaussian distribution could be observed inskilled person also in case of man-machine system. Analysisof brain activation by a NIRS gave several important guessof skill. I.e., (i) utilization of sensory information of widebody part, (ii)reinforcement on an active visual sensing inPMC, and (iii)decrease of ratio of paying attention to motorcontrol of hand, may be needed for skill acquisition. Thecorrelation analysis of pendulum's motion and human force

gave us a guess such that an expert stabilizes system by usinga predicted velocity information based on an virtual pendulummodel which is formed inside her brain.

Additional analyses for catholic facts should be applied tomany and various kinds of persons. We would like to reportfurther in near future work.

AcknowledgmentsThis research is supported by the Grant-in-Aid for 21st

Century COE(Center of Excellence) Program in Ministryof Education, Culture, Sports, Science and Technology. Theauthors are grateful to the Ministry for supporting this work.

REFERENCES[1] Katsuhisa Furuta, "Control of Pendulum: From Super Mechano-System

to Human Adaptive Mechatronics," in proc. of the 42nd IEEE Confon Decision and Control, Maui, Hawaii USA, Plenary, pp.1498-1507,2003.

[2] A. Tustin, "The Nature of the Operator's Response in Manual Controland its Implications for Controller Design," J. of Inst. of Electrical Eng.,vol.94, Part.IIA, pp.190-202, 1947.

[3] Ragazzini,J.R., "Engineering Aspects of the Human Being as a Servo-Mechanism," Meeting ofthe American Psychological Association, 1948.

[4] S. Baron, D.L. Kleinman, and W.H. Levison, "An Optimal ControlModel of Human Response Part II: Prediction of Human Performancein a Complex Task," Automatica, vol.6, pp.371-383, 1970

[5] D.M. Wolpert, M.Kawato, "Multiple paired forward and inverse modelsfor motor control," Neural Networks, vol.11, pp.11317-1329, 1998.

[6] R.C.Miall, D.J.Weir, D.M.Wolpert and J.F.Stein, "Is the Cerebellum aSmith Predictor ?," Journal ofMotor Behavior, vol.25, no.3, pp.203-216,1993.

[7] B.P. Virginia, and D. Julien Doyon, "Dynamic Cortical and SubcorticalNetworks in Learning and Delayed Recall of Timed Motor Sequences,"The Journal of Neuroscience, vol.22, no.4, pp.1397-1406, 2002.

[8] M. Yoneda, F. Arai, T. Fukuda, K. Miyata, and T. Naito, "OperationalAssistance for Straight-line Operation of Rough Terrain Crane," in proc.ofIEEE Int. Conf on Systems, Man, and Cybernetics(SMCJ99), vol.IVpp.1V27-IV32, 1999.

[9] Juan L. Cabrera and John G. Milton, "On-Off Intermittency in a HumanBalancing Task," Physical Review Letters, vol.89, no.15, pp.158702/1-4,2002.

[10] Juan L. Cabrera and John G. Milton, "Human stick balancing: TuningLe'vy flights to improve balancing control," Chaos, vol.14, no.3, pp.691-698, 2004.

[11] K.Kurihara, S.Suzuki, F.Harashima and K.Furuta, "Human AdaptiveMechatronics(HAM) for Haptic System,"' in proc. of the 30th AnnualConference of the IEEE Industrial Electronics Society(IECON'04), Bu-san, Korea, in CD-ROM, 2004.

[12] Ichiro Miyai, Hajime Yagura, et.al, "Premotor Cortex is Involved inRestoration of Gait in Stroke," Annals of Neurology, vol.52, no.2,pp.188-194, 2002.

[13] M.C. Nechyba and Y. Xu, "Human Control Strategy: Abstraction,Verification and Replication," IEEE Control Systems Magazine, vol.17,no.5, pp.48-61, 1997.

[14] D.L. Kleinman, S. Baron and W. H. Levison, "An Optimal ControlModel of Human Response Part 1: Theory and Validation," Automatica,vol.6, pp.357-369, 1970.

[15] Anil Phatak, Howard Weinert, Ilana Segall and Carroll N. Day, "Identi-fication of a Modified Optimal Control Model for the Human Operator,"Automatica, vol. 12, pp.31-41, 1976.

1561

- - I