121 April, 2005
Flight Test Principles Applied to First Robotics: Simple Practices to Help Your Team Succeed
Steven R. JacobsonNASA Dryden Flight Research CenterFlight Controls and Dynamics Branch
221 April, 2005
Steve Jacobson’s Background
Professional Background• BS Aeronautical Engineering Embry-
Riddle Aeronautical University 1992• MS Aeronautics and Astronautics
Purdue University 2000• Daedalus Research, Inc. 1993 – 1997
Design and flight test of small UAVs• NASA Dryden Flight Research Center
1997 – present. Development and flight test of control systems for vehicles ranging from 12’ UAVs to X-37
Autonomous Formation Flight
F-15 Inner Loop Thrust Vectoring
X-37 Approach and Landing Test
Vehicle
Networked UAV Teams (Aerial robots)Advanced Aeroelastic Wing
321 April, 2005
Similarities and Differences Between Coaching FLL and High School Robotics
Similarities between FIRST LEGO League (FLL) and First Robotics (FR) •Short time span•Student Mentor involvement•Coaching/Mentoring strategies very similar•Kids have energy that needs focused in the right direction•The more mentors, the merrier
Differences between FLL and HS• FLL is intended to let the
students do all of the work• First HS robotics is students
and mentors working side by side.
While most of my experience is with Mentoring FLL,
The principles in this presentation apply to both FIRST LEGO League and FIRST High School robotics
421 April, 2005
Steve Jacobson’s Background
FIRST Background• Mentor for Team 585, Tehachapi
High School – 2002, 2003, 2005
• First LEGO League– 2002 City Sights
• Golden Hills School Robotic Eagles
• 5th and 6th grade team• 2nd place at Coppertop
regional Qualifier• 6th place at LEGOLand
– 2003 Mission Mars• Valley Oaks Charter School• 4th, 5th and 6th grade team• 1st place at LA qualifier• 4th place at LEGOLand
– 2004 No Limits• Valley Oaks Charter School• 4th, 5th and 6th grade team• 1st place at Newhall qualifier• 2nd place at LEGOLand
521 April, 2005
Motivation for This Presentation
Each year – My teams have been mostly rookies– Most of my FLL teams have consisted of 4th and 5th grade
students– I start from scratch at the beginning of the season– I run my teams with the philosophy
• The kids come up with the ideas• The kids design, build and program• Let the kids make the mistakes and learn the lessons that are
obvious to the coaches
– Our robots are not very sophisticated– My teams have performed well in competition against older
teams. – I am amazed at what these kids can accomplish in FIRST.– There are accusations of coaches “doing all the work”
Each year is ask myself “What is different between my teams and other teams that makes us successful?”
621 April, 2005
Presentation Abstract
Many robotics teams are coached by people that have a passion for the sport but lack a technical background. Many coaches and mentors have a technical background, but may not have expertise in conducting systems integration, test and operations. Flight testing aircraft and preparing a robot for competition are very similar activities. Both require complex planning, testing, and tying up of loose ends. The flight test community has developed processes for accomplishing flight test and these processes can be applied to FIRST robotics. These include the systems engineering process, configuration management, risk/hazard analysis, system integration, system testing, and operations. When applied they can make an ordinary team into a great team. This paper is intended to help team leaders with technical and non-technical backgrounds prepare there teams at each step in the season. It maps the technical processes applied by the flight test community into simple concepts that can be understood by students and mentors alike. Most of the examples come from my FLL robotics experience but can be applied to high school robotics as well.
721 April, 2005
Johnson Space Center Training
• JSC is NASA’s Lead center for manned space flight• New project managers at JSC are assigned to
mentor a FIRST High School Robotics team• Why?
– Six week systems engineering cycle– Complex problem with infinite solutions– Limited budget– Untrained and unskilled workforce
• If a new project manager can pull this job off with these constraints, they can see what it takes to get any job done at NASA
• NASA Dryden analog: Working with first and applying simple engineering principles in a short time frame has helped me become a better engineer.
821 April, 2005
Phases of a Project
• Project Planning• Design/Building• Testing• Operations/Execution
Systems Engineering Vee
921 April, 2005
Phases of a Project
• Project Planning• Design/Building• Testing• Operations/Execution
1021 April, 2005
Project Planning: Team Leadership
NASA Flight Test leadership• Team leadership and management
skills are the most important skills for the leader
• Intimate knowledge of the technical details is required, but not the most important.
• Empower the technical leaders to get the job done
FIRST team Leaders• The RCA winners are the teams that
have strong student leadership. Team members that focus mostly on planning, delegating and making sure that the details get done.
• Require your parents to get involved
FIRST Mentors (Technical leaders)• Understand the technology
– FLL: LEGO structures and Programming
– FR: C, Fabrication, Pneumatics, etc… • Communicate this knowledge to the
students• Know the game• Know the rules
– Assign a mentor to rule checking
1121 April, 2005
Project Planning: Trained Workforce
FIRST Trained Workforce: Train the students in the technology• FR: Get your team members to trained before the season starts (C
programming, CAD, Pneumatics, drive trains etc.)– Use past years robots or fall to get the students comfortable with the technology – Find a school in your region that offers FIRST Robotics training in the fall– Start a robotics course at your high school– Teach your students how to use sensors
• Go over the game play and rules with your team so everyone understands them.
– Go for the awards not based on robot performance
1221 April, 2005
Project Planning: Trained Workforce
FIRST Trained Workforce: Train the students in the technology• FIRST LEGO League training: Go over the technology with
the students– Use the links on the FIRST web site
http://www.usfirst.org/jrobtcs/flg_tm.htm• Team building• Mindstorms and Robolab Programming
– Teach them how to build robust structures with LEGOs– Burn 10 copies of the programming CD and give it to each team
member• Sensors are your best friend in FLL
– Battery voltage and motor performance vary from hour to hour– Sensors increase repeatability of your robot– Rotation sensor is the most useful sensor in the kit.
• Rotation sensor for turns: Put the rotation sensor on the side that is on the outside turn most of the time
1321 April, 2005
Project Planning: Requirements development
Strategy For the Robot
Robot Base (Drive
System)
Mechan-isms
Software&
Sensors
Speedversus Control
Grasping orHandling
mechanism
TouchLight
Or rotation
Tank Drive Or
Steering
PneumaticsOr motors
Dead Reckoning
or feedback
“Begin with the end in mind”Stephen Covey
1421 April, 2005
Tools for developing requirements
Flight test tool FIRST Robotics First LEGO League
Determine the strategy for your game
•Stewbot•Risk analysis
•Drive a mock robot around the playing field•Look risks analysis associated with each mission
Develop a “Concept of Operations” for each phase of operation. Sketch it out
This will help identify synergy between the missions or modes of operation
•How will you score?•How will you defend?•What kind robustness will the design require?•Robot transportation•Sensor placement
•Go through each mission and strategize several ways on how you will accomplish •Determine what missions you may not want to attempt and focus on the others•Determine the missions that you can combine with others
1521 April, 2005
Tools for developing requirements
Flight test tool FIRST Robotics First LEGO League
Get all of the stakeholders involved
This will prevent you from forgetting important requirements and designing something unobtainable
• Drive system group• Mechanism group • electronics group• sensors group• programming group
• Builders• Programmers • Testers
Identify the interfaces
Defining interface requirements can allow for independent development of systems with integration towards the end of the project.
•Mechanism interface to robot base • Sensor locations•Allow for enough spikes and victors•Special software requirements•Don’t forget access to the robot controller
•Interchangeability of mechanisms•Sensor integration
–Touch sensor–Rotation sensor–Light sensor
•Software training on how to use the sensors
1621 April, 2005
Requirement example for FLL MissionsMechanisms Sensors Wheel or
trackCombination with other missions
CD Fork lift
Bumper
rake
Rotation Wheel or track Glasses
Food Delivery Forklift
Rack
Slide
Rotation
Touch
Wheel Chairs
Chairs Rack/bumper
Forklift
Rotation
Touch
Wheel or track Food
Glasses Hook
Forklift
Rotation Wheel or track CD
Bus Stop Hook or rake
Forklift
Rotation
Light
Touch
Wheel or track Gate, Chair
Pet food
1721 April, 2005
Requirements example: results
• Integrated forklift • Attachable rakes hooks and other
mechanisms• Integrated rotation sensor• Removable light sensor
• Robot required only removal of mechanisms during competition.
• Mechanisms were designed to used for multiple missions
The Requirements design took three practices. All requirements cameFrom the students and before we build the robot we knew exactly
what needed to go on it.
1821 April, 2005
Project Planning: Risk Analysis
Risk table High Risk Medium Risk Low Risk
High payoff Climb the stairs Basketball hoop
Medium payoff Food on table
Gate
Bus Stop
Glasses
Chairs
Low payoff Feed the pets CD
Risk Level
Pay
off
1921 April, 2005
Phases of a Project
• Project Planning
• Design/Building• Testing• Operations/Execution
2021 April, 2005
Design/building phase
• Subsystem Requirements Development• Design • Trade studies/prototyping• Configuration Management
2121 April, 2005
Design/Building: Subsystem Requirements
• Continue on with the requirements development as in the first phase for the subsystem
• Continually look at the interfaces for requirements changes• Have regular “standup” meetings to manage the
configuration• Trade studies:
– Allow competing designs and let the team members decide which is the best design
– Allow the team members to make mistakes. That is how they learn.
2221 April, 2005
Design/Building: Configuration Management
• Make it well understood at the beginning of the project that if you have any changes to your part of the design that you inform the rest of the team.
• Example: A change in a mechanism could effect the robot base, sensor requirements or a software change
• FIRST Robotics: – Hold a regular Configure Control Board (CCB): Discuss any
proposed configuration changes (Major or minor) at each meeting or each time a change is desired
• All groups (stakeholders should be represented)• FIRST LEGO League:
– Instruct team members that they should communicate changes to coaches and other members
– Use a configuration control box where you keep all of the mechanisms (Prevent unintentional disassembly)
– Take digital photographs of the robot and mechanisms to document the construction
– Backup programs before you make a change and after every practice
2321 April, 2005
Phases of a Project
• Project Planning• Design/Building
• Testing• Operations/Execution
Flight research separates “the real from the imagined,” and
makes known the “overlooked and unexpected.”
– Dr. Hugh L. Dryden
2421 April, 2005
Testing, Testing, Testing
• For every hour of flight test there are hundreds or thousands of ground tests
• Testing is where you learn the most about your strengths and weaknesses
• With enough time you can make modifications to improve your robot
• Testing increases the confidence of your team in themselves and their robot
• Plan to start testing with 1-2 weeks to spare• Go to a scrimmage match
Testing is probably one of the most important steps in development. It is usually overlooked and never
given enough time.
2521 April, 2005
Testing: Guidelines
• Write a test plan• Perform end-to-end testing of your robot • Robustness testing: Intentionally try to break the robot to
see if it will withstand the rigors of competition• Perform testing In the order you intend to operate it
– First Robotics: Test the autonomous mode followed by the manual mode
– FIRST LEGO League: Test in the order you intend to operate.• Statistical Process Control
– Run a mission 10-15 times without changing anything– Take notes on the successes and failures– Determine the failure modes based on observations – Modify the robot and repeat the tests
• Regression testing: If you change one thing, however minor, retest everything all over
2621 April, 2005
Phases of a Project
• Project Planning• Design/Building• Testing
• Operations/Execution
2721 April, 2005
Operations and Execution: Flight Cards
• Flight cards include– Checklists for setup before
competition– Steps in which to execute the
missions– Contingency plans
• Flight cards should be used– To prepare for a competition– During practice at the
competition– Only as a reference in case
you “Freeze”
2821 April, 2005
Operations and Execution: General Guidelines
• Practice, Practice, Practice – The flight test analogy is simulation
• Resist the urge to change spontaneously
• “Brief what you fly and fly what you brief”
2921 April, 2005
Conclusion
• Many analogies between getting an airplane ready to fly and a robot ready to play.
• Not every principle may be applicable to your team, but simple discipline in these areas will help your team.
• Most important areas (If you had to only do a few things)– Make sure you have mentors that can teach the
technology– Let the kids “stub their toes”– Spend adequate time on requirements development
early on– Configuration management– Test, Test, Test – Use “Flight cards”– Practice, Practice, Practice