Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred

AutonomousHelicopterFlight

PieterAbbeelUCBerkeleyEECS

n  Unstable

n  Nonlinear

n  Complicateddynamicsn  Airflow

n  Coupling

n  Bladedynamics

n  Noisyes>matesofposi>on,orienta>on,velocity,angularrate(andperhapsbladeandenginespeed)

ChallengesinHelicopterControl

n  Justafewexamples:n  Bagnell&Schneider,2001;

n  LaCivita,Papageorgiou,Messner&Kanade,2002;

n  Ng,Kim,Jordan&Sastry2004a(2001);Ngetal.,2004b;

n  Roberts,Corke&Buskey,2003;

n  Saripalli,Montgomery&Sukhatme,2003;

n  Shim,Chung,Kim&Sastry,2003;

n  Dohertyetal.,2004;

n  Gavrilets,Mar>nos,MeWlerandFeron,2002.

n  Varyingcontroltechniques:inner/outerloopPIDwithhandorautoma>ctuning,H1,LQR,…

SuccessStories:HoverandForwardFlight

[Ng,Coates,Tse,etal,2004]

AlanSzabo–SundayattheLake

OneofourfirstaWemptsatautonomousflips[usingsimilarmethodstowhatworkedforihover]

Targettrajectory:me>culouslyhand-engineeredModel:from(commonlyused)frequencysweepsdata

n  Hover/sta>onaryflightregimes:

n  RestrictaWen>ontospecificflightregime

n  Extensivedatacollec>on=collectcontrolinputs,posi>on,orienta>on,velocity,angularrate

n  Buildmodel+model-basedcontroller

à  Successfulautonomousflight.

n  Aggressiveflightmaneuvers---addi>onalchallenges:

n  Taskdescrip7on:Whatisthetargettrajectory?

n  Dynamicsmodel:Howtoobtainaccuratemodel?

Sta>onaryvs.AggressiveFlight

n  Gavrilets,Mar>nos,MeWlerandFeron,2002n  3maneuvers:split-S,snapaxialroll,stall-turn

n  Key:Expertengineeringofcontrollersalerhumanpilotdemonstra>ons

Aggressive,Non-Sta>onaryRegimes

SundayinOpenLoop

n  Ourwork:n  Key:Automa>cengineeringofcontrollersalerhumanpilot

demonstra>onsthroughmachinelearning

n  Widerangeofaggressivemaneuvers

n  Maneuversinrapidsuccession

Aggressive,Non-Sta>onaryRegimes

n  Learningatargettrajectory

n  Learningadynamicsmodel

n  Autonomousflightresults

LearningDynamicManeuvers

n  Difficulttospecifybyhand:n  Requiredformat:posi>on+orienta>onover>me

n  Needstosa>sfyhelicopterdynamics

n  Oursolu>on:n  Collectdemonstra>onsofdesiredmaneuvers

n  Challenge:extractacleantargettrajectoryfrommanysubop>mal/noisydemonstra>ons

TargetTrajectory

Abbeel,Coates,Ng,IJRR2010

ExpertDemonstra>ons

•  HMM-likegenera>vemodel–  DynamicsmodelusedasHMMtransi>onmodel

–  Demosareobserva>onsofhiddentrajectory

•  Problem:howdowealignobserva>onstohiddentrajectory?

LearningaTrajectory

Demo1

Demo2

Hidden


n  DynamicTimeWarping(Needleman&Wunsch1970,Sakoe&Chiba,1978)

n  ExtendedKalmanfilter/smoother

LearningaTrajectory

Demo1

Demo2

Hidden


Results:Time-AlignedDemonstra>ons§ Whitehelicopterisinferred“intended”trajectory.

Results:Loops

Evenwithoutpriorknowledge,theinferredtrajectoryismuchclosertoanidealloop.






StandardModelingApproach


3Gerror!

KeyObserva>on

Errorsobservedinthe“baseline”modelareclearlyconsistentaleraligningdemonstra>ons.


n  Ifweflythesametrajectoryrepeatedly,errorsareconsistentover>meoncewealignthedata.

n  Therearemanyunmodeledvariablesthatwecan’texpectourmodeltocaptureaccurately.

n  Air(!),actuatordelays,etc.

n  Ifweflythesametrajectoryrepeatedly,thehiddenvariablestendtobethesameeach>me.

~musclememoryforhumanpilots

KeyObserva>on


n  Learnlocally-weightedmodelfromaligneddemonstra>ons

n  Sincedataisalignedin>me,wecanweightby!metoexploitrepeatabilityofunmodeledvariables.

n  Formodelat>met:

n  Obtainamodelforeach>metintothemaneuverbyrunningweightedregressionforeach>met

Trajectory-SpecificLocalModels







ExperimentalSetup

Microstrain3DM-GX1@333HzRPMsensor@20-30Hz

Sonar

OxoardCameras1280x960@20HzExtendedKalmanFilterRHDDPcontroller

Controls@20Hz

“Posi>on”

3-axismagnetometer,accelerometer,gyroscope

(“Orienta>on”)

Abbeel,Coates,Quigley,Ng,NIPS2007

1.  Collectsweepstobuildabaselinedynamicsmodel

2.  Ourexpertpilotdemonstratestheairshowseveral>mes.

3.  Learnatargettrajectory.

4.  Learnadynamicsmodel.

5.  Findtheop>malcontrolpolicyforlearnedtargetanddynamicsmodel.

6.  Autonomouslyflytheairshow

7.  Learnanimproveddynamicsmodel.Gobacktostep4.

àLearntoflynewmaneuversin<1hour.

ExperimentalProcedure


Results:AutonomousAirshow

Results:FlightAccuracy

AutonomousAutorota>onFlights

Abbeel,Coates,Hunter,Ng,ISER2008

Chaos[“flip/roll”parameterizedbyyawrate]

ThankYou

Documents

Autonomous Helicopter Flight - EECS at UC Berkeleypabbeel/cs287-fa...§ White helicopter is inferred “intended” trajectory. Results: Loops Even without prior knowledge, the inferred