119

Proceedings of the 26th Chinese Control ConferenceJuly 26-31, 2007, Zhangjiajie, Hunan, China

Distributed Remote Control System of UAV Based onMan-in-loop Real-time Operation

Zhang Fengqing1, Kong Quancun1,2

1.School of Instrument and Optoelectronic Engineering , Beihang University, Beijing 100083, P. R. ChinaE-mail: luckygirlxue@126.com

2. School of Electronic Information Engineering , Information and Technology University, Beijing 100083, P. R. ChinaE-mail: kongquancun@163.com

Abstract: Unmanned aerial vehicle (UAV) has been widely applied in reconnaissance, strategic bombing, discharging elec-tromagnetic interference. Of paramount significance to completing tasks is control system; the UAV control system shouldhave the ability of formate, revaluating the flight plan timely, making on-line decisions. Such requirements greatly challengepresent control technology. A solution is presented to improve current UAV control system into a remote control system withnetwork distributed computing, three dimensional dynamic display and man-in-loop real-time operation. This system aims toprovide better functionality of real time, visualization, and intelligence in order to meet the above demands.. Besides, the ap-plication of forecasting display control technology in this system can not only mitigate delay problem—one usual problem inremote control system but also realize a high level of intuitive operability. Moreover, the application of .NET remoting light-ens data dropout and makes this system meet the high data-rate communication requirement.Key Words: UAV, remote control system, man-in-loop, .NET remoting, forecasting display control technology

1 INTRODUCTION

When compared with pilot aircraft, UAV has its uniqueadvantages in that we do not need consider the safetyguarantee system, the special design of passenger com-partment and so forth any more. In the situation of fewerlimitations, we can focus on maximizing pneumaticpower and control efficiency. It, however, has its ownfatal drawback��removing the human factor from thesystem so as to be lack of intelligent decision-makingand agile flight control.� Generally, there are three kinds of control meth-

ods:Independent control not only includes inde-pendent accomplishment of previous decidedcourse and task but also includes making suitabledecisions which are based both on previous taskand on the analysis of present flight state, inde-pendently and timely. The high demand of makingdecision to deal with complicated problems inde-pendently even under some uncertain situationsgreatly challenges the control theory. Up to now,there are still many problems not solved.

� Telecontrol technology, from real-timely control-ling the pneumatic rudder panel and the enginestate to intermittently adjusting the flight course,need communications between remote control sta-tion and UAV. With timely flight state information,operators are responsible of surveilling and con-trolling UAV. However, without any independentcontrol, it is sure that operators have large a mountof work.

� The third way is to combine independent controland telecontrol. Under many uncertain situations, itis a good choice to rely on the adaptive, analyzingand decision making capability of human beingespecially when the complexity of flight task andon-line decision-making exceeds the ability ofUAV independent control system. Besides, adding

human intelligence to UAV control system can alsoreduce the cost, lower the risk and improve the ef-ficiency especially when distributing key task,identifying targets precisely or casting weapons.Moreover, adding independent control to telecon-trol could reduce the workload of operators as well.

Up to now, distributed flight tasks for UAVS are moreand more complicated and draw people’s attentionsmostly to the third control methods.

2 UAV CONTROL SYSTEM

2.1 Representation of current UAV control system

In view of efficiency, flight mode of UAV is often co-ordinated aviation. Fig.1 shows the process principle ofa practical control structure.

Task control layer

Tactical layer

Strategic layer

Regulative layer

Target and flightstate sensor

Environmentalsensor

formate

Rudderinstruction Track error

On ground surveillant controlstation

On UAV

Fig.1 a practical control structure

The function of task control layer is to divide the wholetask (such as reconnaissance, bombing and etc.) intodetail aim for each UAV. The kernel function of thislayer is to deploy resource and to rearrange the plantimely. For example it need to optimize the course of theair fleet when executing the task. The strategic layertakes charge of figuring out the detail limitations of

120

executing the course after analyzing real-time informa-tion about appointed lane and target. Also, this layercould adjust coordinated flight plan to avoid collisionand conflict among UAV. The tactical layer is responsi-ble for controlling the change of flight modes and tomake executable lane instructions. The function ofregulative layer is to track the ideal trace by inner loopfeedback controller. The first layer is on the groundsurveillant control station and the next three layers areon the UAV. Therefore, this system is almost based onindependent control with a little telecontrol in the taskcontrol layer.

2.2 Improved UAV control system

This paper discusses a way to improve the above controlstructure into a remote control system of UAV withnetwork distributed computing, three dimensional dy-namic display and man-in-loop real-time operation.As Fig.2 shown, the operation table connects to instruc-tion computer to offer control instruction for UAV inthis system. Mode computer figures out flight math-ematic model. Three dimensional dynamic displaycomputer is to compute in time and display the flightscene.

Instruction computer

UAV

Operationtable

Mode computer

Database

Digitaltransmitter

receiver

Three dimensionaldynamic display computer

Three dimensionaldynamic display

computer

Fig.2 UAV remote control simulation system

To discuss the system further, instruction computer re-ceives control instructions, which are sent by controlpanel and remote devices, transforms them into digitalsignals and then sends them to mode computer withpromissory protocol on internet in order to control flightstate and trace. Mode computer reads UAV parametersfrom database to work out real-time digital simulation.Moreover, after receiving information from instructioncomputer, this computer works out locomotory data ofaircraft by control loop, rudder loop and the loop madeup of UAV itself. Real-time three dimensional dynamicdisplay computer gets data from mode computer peri-odically and displays real-time 3D dynamic flight sceneincluding takeoff, climbing, wheeling, gliding, de-scending and etc. With virtual flight instruments, it sup-plies much real-time and visual flight information ofUAV to the operators. Accordingly, the operators caneasily control the UAV by sending the accordingly or-ders through this system. Fig.3 describes the data flowand logic relationship among different modules in this

simulation system. These modules make up of a networkdistributed computing system so as to increase the run-ning speed.

A C DBControl

instructonCourse

parameters

Flight stateparameters F

G

Navigation parameters

E

Arepresents a system control table.Brepresents a flight control system simulation module.Crepresents control loop.Drepresents rudder loop.Erepresents a flight model simulation module.Frepresents a flight state and instrument 3D dynamic display module.Grepresents a compositive sensor simulation module.

Fig.3 principle framework of UAV remote control system

2.3 Improvements between the former and the latter

control system

Compared with the representation of current UAV con-trol system, the improved UAV control system dis-cussed above are more intelligent and agile withman-in-loop real-time operation. In other words, humanfactor is added not only to the task control layer but alsoto the other three layers.The improvements can be seen clearly from the follow-ing example. In 20th century, the Air Force acquired theAQM-34V UAV to perform strategic bombing ahead ofmanned attack aircraft. When carrying out the appointedtask, these UAV, mainly independent control with sometelecontrol, bombed a wrong place. Since two-year-oldmap used for targeting did not show the numbers ofbuildings on that road, officers mistook the numbers ofbuilding on parallel roads as the arms agency of theenemy. Though officers had soon realized the error aftersending UAV, it is impossible to prevent the task carry-ing out totally only by task control layer. If officers inthe ground surveillant control station rearranged the planentirely by task control layer, they would face manyother problems for example whether the changed courseof the air fleet, based on information from task controllayer, would be better than bombing wrong buildings.However, the improved UAV control system add humanfactor not only to task control layer but also to the otherthree layers and thus can realize man-in-loop real-timeoperation. In this case, the timely changing plan couldbe much more reliable.

3 THREE MAIN PROBLEMS IN THE IM-PROVED UAV CONTROL SYSTEM

3.1 Delay problem

The capability of wireless communication system islimited by wireless channel. In wireless communication,because of reflect and dispersion of electric wave, thereare multi-transmission routes between sender and re-ceiver. Moreover, both transmission delay and fadinggene in each route are time-varying and accordinglymake receiving signal fade. The delay between sendinginstructions and receiving feedback messages prevents

121

operators observing the results of their actions timely,removes the actor from some of the consequences aswell as reduces maneuverability seriously.

3.2 Problems brought by network itself

This remote control system of UAV with network dis-tributed computing and man-in-loop real-time operationis a representative example of Networked Control Sys-tem (NCS). In other words, each part of the feedbackcontrol system makes up of a closed loop by real-timenetwork.As Fig.4 shown, its essential is to exchange informationamong control components (reference input, object out-put, control parameter input and etc.) by network.Though this system has its unique advantages, it is ob-vious that the introduction of network into control sys-tem makes the analysis of control system much morecomplicate and some new problems such as data drop-out, time confusion and time-varying delay.

Sensoroutput

Sensor delay

controller

object

Networkchannel

Reference input

�.

�.�.

.

.

.

.

.

.

.

.

.

.

.

.

Act delay

Act delay

Fig.4 NCS framework

3.3 High demand of transmission rate

Data transmitted between UAV and the main stationinclude telecontrol information, telemetry informationand task sensor information. Telecontrol information isa kind of control information sent by the main station toUAV. Its main function is to control flight state, to leadUAV to required position and to manage various kindsof instruments. Telemetry information, from UAV to themain station, includes flight state, flight parameters,instruments’ working condition and etc. Though thereliability, security and real-time demand of informationtransmission is high, the amount of telecontrol informa-tion and telemetry information is small and generally12.8 kbps transmission rate is enough. Task sensor in-formation is a kind of information obtained by task in-struments such as television camera, infrared scanner,multi-spectrum sensor, compositive aperture radar andetc. The amount of task sensor information relate to thekind of task sensor, the size of image, resolving power,compression ratio and etc. Compared with telecontrolinformation and telemetry information, the amount oftask sensor information is very large and demands highdata-rate communication. For instance, task sensor in-formation which is five-spectrum image, dimension 400pixels × 600 pixels, 25 frames per second, 10 com-

pression ratio, requires 24 Mbps transmission rate.

4 SOLUTIONS FOR ABOVE PROBLEMS

4.1 Forecasting display control technology

In view of delay problem, this paper discusses the fore-casting display control technology in order to meet de-mands of the UAV control system. The developments ofcomputer science, graphics and image technology, soft-ware engineering have pushed the research and applica-tion of system simulation technology into a new era ofvisualization, multimedia and VR (virtual reality). AsFig.5 shown, forecasting display control technologyprovides a scene in which entities move and behave be-lievably and approximate the complexity of the realworld.

Fig.5 three dimension dynamic display of UAV flight

As Fig.6 shown, the idea behind this technology is to“project” actions that are carried out by users in the vir-tual world into the real world, like projective virtualreality in a sense. There is no delay between operatorsand the virtual world. Therefore, operators could worklike they would in the physical world so as to this typeof man machine interface reaches a high level of intui-tive operability. The basic idea of applying forecastingdisplay control technology in UAV control system is tobuild up virtual UAV to simulate real UAV with com-puter graphics, simulation technology and VR. Opera-tors directly control virtual UAV to complete requiredtask in virtual environment and then real UAV will fol-low what the virtual ones have done. With this fore-casting display control technology, operators could con-trol UAV more easily and precisely.

Instructioninput

Displayoutput

operatorVirtual

environmentVirtualUAV

RealUAV

Fig.6 system framework

4.2 .NET remoting

Though forecasting display control technology hasmodified the improved UAV control system in order to

122

mitigate the delay problem, there are still other problemsin actual information transmission process. Consideringthe problems bought by network itself and highrequirement of communication rate, .NET remoting, aframework for developing distributed applications, is agood choice. As the successor to DEC/RPC/DCOM, .NETremoting can help to solve the above problems in that itmakes the communication system more reliable, appli-cable and compatible. This framework not only allowsobjects to interact with one another across applicationdomains but also provides a mass of services. For ex-ample, .NET remoting provides object activation, life-time support, communication channels for transportingmessages to and from remote applications and etc. Itenables objects in different application fields or proc-esses to build up communication channels without limi-tation of transfer protocol. Before messages are trans-ported by the channel, formatters are used for encodingand decoding. When interoperability with other remot-ing framework is critical, applications can use binaryencoding or XML encoding which uses the SOAP pro-tocol in transporting messages. Besides, .NET remotingprovides abstract communication processing approachwhich can separate the remotable object from a specificclient or server application domain and from a specificmechanism of communication. In this case, it greatlysimplifies the accessing of distributed objects.As Fig.7 shown, client side creates one instance ofserver classes and the remoting system creates a proxyobject which represents this class and returns a referenceof the proxy to the client. When a client calls a method,the processing structure of .NET remoting deals with thecalling, verifies the message type and then sends thecalling to the server process by channel. By listening tothe channel, requests are forwarded to the server remot-ing system, which finds and calls the requested object.Then the procedure is reversed and the server remotingsystem transforms the response into messages that are

sent to the client channel by server channel. Finally, theresult of the call is returned to the client object throughthe proxy and any application can provide its services toany client on its network.

5 CONCLUSION

With wider and wider application and more and morecomplicated tasks for UAV, researchers focus on im-proving the UAV control system. In this paper, the im-proved UAV control system has the following advan-tages:� The adaptive, analyzing and decision making capa-

bility of human being make this system more intel-ligent and agile. Abilities of revaluating the flightplan on line, making real-time decisions and man-aging malfunction are powerful.

� This system could lighten data dropout, transmitlarge amount of task sensor information and meetthe high data-rate communication requirement.

� Forecasting display control technology not onlymitigates delay problem—one usual problem inremote control system but also makes the systemreach a high level of intuitive operability.

However, this control system still has space to be furtherimproved. For example the reality of virtual flight in-strument needs to be advanced and the delay betweenvirtual UAV and real UAV still exists. Yet a remotecontrol system with network distributed computing,three dimensional dynamic display and man-in-loopreal-time operation will, of course, be an important re-search direction in future.

REFERENCES

[1] Microsoft Corporation. .NET Framework developer’s guide.Redmond, USA: MSDN library, 2005.

[2] Cheng Peng. Automatic Control Theory. Beijing: HigherEducation Press, 2003.

Remoteobject

Clientobject

Proxy

Channel

RemotingSystem

RemotingSystem

Fig.7 remote process