View
11
Download
0
Category
Preview:
Citation preview
2014 spring lab seminar
Persistent UAV service: Overview
xS3D
Department of Industrial and Systems Engineering
KAIST, South Korea
Thursday, March 6, 2014
2014 spring lab seminar
Presentation Overview
• Motivation for persistent service
• Overall orchestration of UAV service system
• UAV service system : Components and prototype
- Central planning
- UAV guidance system
- Automatic replenishment station
- System demonstration
• Concluding remarks
2014 spring lab seminar
Motivation
• Large expensive UAVs
– Usually military purpose
– Operate for many hours
– Travel long distances
• Small inexpensive UAVs
– A lot of application area such as tracking, communication relay,
environmental / fire / national boundary monitoring, cartography, disaster
relief and so on.
– Limited duration of mission
– Limited distance
• To increase the usability of small UAVs, systems for persistent operation is required
– Collection of UAVs, refueling stations, automatic operation system
– Methods to orchestrate their operations (scheduling issue)
2014 spring lab seminar
Overall orchestration of UAV service system
Service station 1
Service station 2
Object 1
Object 2
Object 3Service station 3
UAV 1
UAV 2
UAV 4
UAV 3
UAV 5
Moving
objective
trajectory
UAV service
system
Persistent UAV service
Random arrival
of customer
information Random path and
duration as
customer request
Vision
technology
Heterogeneous
UAVs
UAV operation
system
Central
planning
Automatic
replenishment
station
2014 spring lab seminar
Overall orchestration of UAV service system
Service station 1
Service station 2
Object 1
Object 2
Object 3Service station 3
UAV 1
UAV 2
UAV 4
UAV 3
UAV 5
Moving
objective
trajectory
UAV service
system
Persistent UAV service
UAV operation
system
Automatic
replenishment
station
Central
planning
Vision
technology
Heterogeneous
UAVs
Random path and
duration as
customer request
2014 spring lab seminar
Overall orchestration of UAV service system
Service station 1
Service station 2
Object 1
Object 2
Object 3Service station 3
UAV 1
UAV 2
UAV 4
UAV 3
UAV 5
Moving
objective
trajectory
UAV service
system
Persistent UAV service
UAV operation
system
Automatic
replenishment
station
Central
planning
Vision
technology
Heterogeneous
UAVs
2014 spring lab seminar
System description
• Persistent UAV system with distributed multiple service stations – Follow deterministic time-space trajectories without interruption
– Capacitated UAVs can use any station and should return after mission
0 100 300 200 400 500 600
100
200
300
Station 1
Station 2
2014 spring lab seminar
System description
Customer & UAV information Optimization algorithm Detail Schedule
2014 spring lab seminar
System Description
■ To follow a time-space trajectory, the trajectory is divided into pieces (split jobs)
Start
End
Service station 1
Service station 2
1
2
3
14 4
5
6 7
∙ ∙ 13
▪ Objective moves
- From point (50,250) to (950,750)
- From 13:06 to 13:20
2 2 2 2( ) ( ) ( ) ( )i j i j j i j i
ij e s e s ji e s e sd x x y y d x x y y
Split
job
Start
point
End
point
Start
time
End
time
1 50,250 150,250 13:06 13:07
2 150,250 250,250 13:07 13:08
3 250,250 350,250 13:08 13:09
4 350,250 450,250 13:09 13:10
5 450,250 550,250 13:10 13:11
6 550,250 650,250 13:11 13:12
7 650,250 750,250 13:12 13:13
8 750,250 850,250 13:13 13:14
9 850,250 950,250 13:14 13:15
10 950,250 950,350 13:15 13:16
11 950,350 950,450 13:16 13:17
12 950,450 950,550 13:17 13:18
13 950,550 950,650 13:18 13:19
14 950,650 950,750 13:19 13:20
2014 spring lab seminar
Mathematical Model
i, j : Indices for jobs
s : Index for stations
k : Index for UAVs
r : Index of a UAV’s rth flight
NJ : Number of split jobs
NUAV : Number of UAVs in the system
NSTA : Number of recharge stations
NR : Maximum number of flight per UAV during the time horizon
M : Large positive number
(xjs, y
js) : Start point of split job j
(xje, y
je) : End point of split job j
Dij : Distance from split job ith finish point to split job jth start point, Dij ≠ Dji
Ei : Start time of split job i
Pi : Processing time or split job i
qk : Maximum traveling time of UAV k
Sok : Initial location(station) of UAV k
TSk : Travel speed of UAV k
• Notations
2014 spring lab seminar
Mathematical Model
ΩJ : = {1, …, NJ}, Set of split jobs
ΩJD : = {1, …, NJ+1}, Set of split jobs and dummy jobs
ΩSS : = {NJ+2, NJ+4, …, NJ+2∙ NSTA}, set of UAV flight start station
ΩSE : = {NJ+3, NJ+5, …, NJ +2∙ NSTA+1}, set of UAV flight end station
ΩA : = (ΩJD U ΩSS U ΩSE) = {1,…, NJ+2∙NSTA+1}, set of all jobs and recharge stations
▪ Xijkr = 1 if UAV k processes split job j or recharges at station j after processing split job i or recharging at station i during the rth flight; 0, otherwise
▪ Cikr is job i’s start time by UAV k during its rth flight or UAV k’s recharge start time at station i; otherwise its value is 0.
▪ Yikr = 1 if UAV k processes split job i during its rth flight; 0, otherwise.
• Notation
• Decision Variables
2014 spring lab seminar
Mathematical Model
1, 1 ( , 1... 1, )JD JD
iskr s ikr R SE
i i
X X k K r N s
1 ( , )SE JD
iskr
s i
X k K r R
1 ( , )SS JD
sjkr
s j
X k K r R
, 1 1 ( )ok
JD
s jk
j
X k K
Subject to
1, 1 ( , 1... 1, )skr s kr R SEC C k K r N s
1 ( )A
ijkr J
k K r R i
X j
0 ( , , )A A
ijkr jikr JD
j j
X X i k K r R
0 ( , , )JD
iskr SS
i
X k K r R s
A A
ij ijkr
k K r R i j
D X
Minimize
Network flow constraints
Initial recharge station constraints
Split job assignment constraints
• Mathematical formulation
2014 spring lab seminar
Mathematical Model
/ (1 ) ( , , , )ikr i ij k jkr ijkr JD SS JD SEC P D TS C M X i j k K r R
( , , )JD SE
ijkr ikr J
j
X Y i k K r R
( , , )ikr ikr JM Y C i k K r R
( )ikr i J
k K r R
C E i
/ ( , )A A JD A
ij k ijkr i ijkr k
i j i j
D TS X P X q k K r R
, 1, ( , , )sdkr d s kr SSX X k K r R s
0 ( , , )dikr idkr JX X k K r R i
0 ( , , )ikr AC k K r R i
{0,1} ( , , , )ijkr A AX k K r R i j
{0,1} ( , , )ikr AY k K r R i
Start time consts
Fuel and battery constraints
Dummy job constraints
Decision variables
• Mathematical formulation
2014 spring lab seminar
UAV guidance system
1280 720 pixel front camera
320 240 pixel belly camera
< AR drone 2.0 >
< Ipad 3>
< WIFI >
2014 spring lab seminar
UAV guidance system
■ Roles of UAV guidance system
1. Receive and implement the schedule from the MILP.
2. Convert the video from the UAV cameras into usable information for directing the motion of
the UAVs
3. Enable a human overseer to monitor the UAV progress via video and adjust feedback control
gain values for various situations
4. Allows for a human overseer to initiate emergency actions such as immediate landing.
2014 spring lab seminar
UAV guidance system
Descriptions
① : Front(Bottom) Camera Video
② : Start Procedure Interface
③ : Color Filtered Video
④ : Control Gain Adjustment Sliders
⑤ : Emergency Landing Button
1. The color video from the camera is acquired via
TCP port and processed using OpenCV framework.
2. The image is separated into three RGB channels.
These three images are used to determine the color
of the targeted image.
3. Control inputs including the longitudinal-lateral tilt
angles, height and yaw angular velocity are calculated
from the number and mean coordinate of target pixels
in the processed image.
2014 spring lab seminar
Automatic replenishment station
▪ Each AR Drone 2.0 uses a three cell lithium
polymer battery
▪ four copper leads (three for each terminal and
one for the ground terminal) were threaded from
the battery inside the UAV to the four feet of the
drone
▪ The service station consists of four pads, one for
each foot of the drone.
▪ Each such pad connects to the UAV battery via
the leads on the drone feet
2014 spring lab seminar
System demonstration
2014 spring lab seminar
System demonstration
UAV Start
station
Assigned
job
End
station
Service
start
time
Service
end
time
1 1 1,2,3,4 2 2 10
2 2 5,6,7,8 3 10 18
■ Demonstration description
Station 1Station 2
Station 3
UAV 1
UAV 2
Hand-off
Split
job 1
Split
job 2
Split
job 8
Split
job 7
∙ ∙ ∙
< Schedule by MILP >
< Demonstration layout >
5m
2014 spring lab seminar
Experiments
■ Demonstration video
2014 spring lab seminar
Concluding remarks
• To increase the usability of small UAVs, prototype systems for persistent operation is developed.
• As a components of persistent UAV service system, MILP for deriving UAV schedules, UAV
guidance system , automatic replenishment station were suggested.
• MILP generates UAV schedules using customer information and system resource information
such as location of station and number & location of UAVs.
• UAV guidance system provides uninterrupted customer tracking service by using vision
technology.
• Demonstration shows the orchestrationof those system components and applicability of proposed
UAV service system.
2014 spring lab seminar
Literature Review
• Scheduling methods without a distance or time restriction – T. Shima and C. Schumacher, “Assignment of cooperating UAVs to simultaneous tasks using genetic algorithm,” In Proc. AIAA
Guidance, Navigation, and Control Conference and Exhibit, San Francisco, 2005 – J. Zeng, X. Yang L. Yang and G. Shen, “Modeling for UAV resource scheduling under mission synchronization,” Journal of Systems
Engineering and Electronics, Vol. 21, No. 5, 2010, pp. 821-826
• Scheduling methods for limited flight duration – A. L. Weinstein and C. Schumacher, “UAV scheduling via the vehicle routing problem with time windows,” In Proc. AIAA
Infotech@Aerospace 2007 Conference and Exhibit, Rohnert Park, California, 2007 – T. Shima, S. Rasmussen and D. Gross, “Assigning micro UAVs to task tours in an urban terrain,” IEEE Transactions on Control Systems
Technology, Vol. 15, No. 4, 2007, pp. 601 – 612 – Y.S. Kim, D.W. Gu and I. Postlethwaite, “Real-time optimal mission scheduling and flight path selection, IEEE Transactions on
Automatic Control, Vol. 52, No. 6, 2007, pp. 1119-1123. – B. Alidaee, H. Wang, and F. Landram, “A note on integer programming formulations of the real-time optimal scheduling and flight
selection of UAVS,” IEEE Transactions of Control Systems Technology, Vol. 17, No. 4, 2009, pp.839-843
• Scheduling method for persistent UAV operation – M. Alighanbari and J. P. How, “Decentralized task assignment for unmanned aerial vehicle”, Proceedings of the 44th IEEE Conference
on Decision and Control, and the European Control Conference 2005 seville, spain, december 12-15, 2005
• Battery recharge/exchange methods – J. How, thesis papers at MIT, 2005, 2007 – A.S. Kurt, B.H. Clarence, R.R. Johnhenri, D.W. Richardson, Z.H. White, Q. Elizabeth and G. Anouck, “Autonomous Battery Swapping
System for Small-scale Helicopters”, 2010 IEEE International Conference on Robotics and Automation – R. Godzdanker, M. J. Rutherford and K. P. Valavanis, “ISLANDS: A self-leveling platform for autonomous miniature UAVs”, 2011
IEEE/ASME International Conference on Advanced Intelligent Mechatronics, pp 170-175 – A.O.S. Koji, K.F. Paulo and James R. Morrison, “Automatic battery replacement system for UAVs: Analysis and design” Journal of
Intelligent and Robotic Systems, Special Issue on Unmanned Aerial Vehicles (Springer), a Special Volume on Selected Papers from ICUAS’11, Vol. 65, No. 1, pp. 563-586, January 2012. First published online September 9, 2011
– M. Valenti, D. Dale, J. P. How and D. P. de Farias, “Mission health management for 24/7 persistent surveillance operations”, AIAA Guidance, Navigation and Control Conference and Exhibit, 20-23 August 2007, Hilton Head, South Carolina
Recommended